impala3.0参考英文版

所需积分/C币:49 2018-08-28 16:11:23 3.71MB PDF
24
收藏 收藏
举报

impala3.0版本最新版本官方详细参考文档,包括安装使用 ,安全,管理等等所有功能,以及sql语法,升级等
Contents iii Upgrading Impala Impala Upgrade Consideratio ······*· List of Reserved Words updated in Impala 3.0........ Decimal V2 Used by Default in Impala 3.0 30 Behavior of Column Aliases Changed in Impala 3.0 30 Default PARQUET ARRAY RESOLUTION Changed in Impala 3.0. Enable Clustering Hint for Inserts.…… 30 Deprecated Query Options Removed in Impala 3.0. Fine-grained Privileges Added in Impala 3.0 refresh after connect Impala Shell Option Removed in Impala 3.0 Default Setting Changes...... 31 Starting Impala.........…… 31 Starting Impala from the Command Line. 32 Modifying impala startup options Configuring Impala Startup Options through the Command Line…………… Checking the values of Impala Configuration Options 35 Startup Options for impala Daemon 35 Startup options for state d daemon 35 Startup Options for catalog Daemon 5 Impala Tutorials.………,…… 35 Tutorials fo Started 6 Explore a New Im 36 Load Csv Data from Local files 41 Point an Impala table at Existing Data Files 43 Describe the Impala Table Query the Impala table 45 Data Loading and Querying Examples Advanced Tutorials 48 Attaching an External Partitioned Table to an HDFS Directory Structure Switching Back and Forth Between 50 Cross Joins and Cartesian Products with the CrOSs JOIN Operator Dealing with Parquet Files with Unknown Schema .53 Impala administration 67 Admission Control and Query Queuing. 68 Overview of Impala Admission Control...... Concurrent Queries and Admission Control 69 Memory limits and Admission Control How Impala Admission Control Relates to Other Resource Management Tools.....69 How Impala schedules and Enforces Limits on Concurrent Queries………...…... 70 How Admission Control works with Impala Clients(JDBC,ODBC, Hive Server2)……….….0 SQL and schema Considerations for Admission Control Configuring Admission Control 71 Resource management for Impala.... .76 How Resource limits are enforced 76 impala-shell Query Options for Resource Management 77 Limitations of Resource Management for Impala...…..….…......77 Setting Timeout Periods for Daemons, Queries, and Sessions 77 Increasing the state Tir 77 Setting the Idle Query and Idle Session Timeouts for impala 77 Setting Timeout and Retries for Thrift Connections to the Backend Client................78 Contents iv Cancelling a Qu 78 Using Impala through a Proxy for High Availability......,..,.….….…….…..178 Overview of proxy usage and Load balancing for Impala 79 Choosing the Load-Balancing Algorithm Special Proxy Considerations for Clusters Using Kerberos 80 Example of Configuring HAProxy Load Balancer for Impala........81 Managing Disk Space for Impala data.…… Impala security 84 Security Guidelines for Impala 85 Securing Impala Data and Log files 86 Installation Considerations for Impala Security 86 Securing the Hive Metastore Database 86 Securing the Impala Web User Interface Configuring TLS/SSL for Impala Using the Command Line 87 USing TLS/SSL with Business Intelligence Tools 8 Specifying TLS/SSL Minimum Allowed Version and Ciphers 88 Enabling Sentry Authorization for Impala. The Sentry Privilege Model ······ Starting the impala Daemon with Sentry Authorization enabled USing Impala with the Sentry Service(Impala 1. 4 or higher only) 90 USing Impala with the Sentry Policy File ·············· Setting Up Schema Objects for a Secure Impala Deployment 95 Privilege Model and Object Hierarchy Debugging Failed Sentry Authorization Requests The DEFAULT Database in a Secure Deployment Impala authentication ·+·+, Enabling Kerberos Authentication for Impala.,……… 100 Enabling LDAP Authentication for Impala 102 Using Multiple Authentication Methods with Impala 104 Configuring Impala delegation for Hue and bi Tools 105 Auditing Impala Operations. 105 Durability and Performance Considerations for Impala Auditing 105 Format of the Audit Log Files 106 Which Operations Are Audited 106 Viewing Lineage Information for Impala data .107 Impala sQl language reference.…,,…,,…,,,…,…,,,,107 Comments 108 Data t 108 ARRAY Complex Type (Impala 2.3 or higher only) 109 BIGINT Data Type 112 BOOLEAN Data Type……….….….….… ····· .13 CHAR Data Type(Impala 2.0 or higher only) 单·;···· ··“························· 114 DECIMAL Data Type(Impala 3.0 or higher only) .118 DOUBLE Data Type…… 122 FLOAT Data Type 124 INT Data Type.. 125 MAP Complex Type(Impala 2.3 or higher only) 126 REAL Data Type. 130 SMALLINT Data Type 130 STRING Data Type.………… 13 STRUCT Complex Type(Impala 2.3 or higher only) 132 I Contents v TIMESTAMP Data Type 138 TINYINT Data Type 146 VARCHAR Data Type(Impala 2.0 or higher only) 147 Complex Types(Impala 2.3 or higher only) …149 Literals .181 Numeric literals 181 String Literals.…… .182 Boolean literals lestamp literals NULL l84 SQL 5 Arithmetic operat 185 BETWEEN Operator. 188 Comparison operators EⅩ ISTS Operator. ILIKE Operator.… ∴193 IN Operator.…… 194 IREGEXP Operator 197 IS DISTINCT FROM Operator 198 IS NULL Operator.................…….…….…..200 IS TRUE Operator 201 LIKEOperaor....... 20 REGEXP Operator… 205 RLIKE Operator 207 Impala schema Objects and Object Names 208 Overview of Impala aliases 208 Overview of Impala databases .210 Overview of Impala Functions 210 Overview of Impala Identifiers .212 Overview of Impala tables 213 Overview of Impala Views 217 Impala SQL Statements 221 DDL Statements 221 DML Statements ··:····· 22 ALTER TABLE Statement 223 ALTERⅤ EW Statement. .237 COMPUTE STATS Statement 238 CREATE DATABASE Statement CREATE FUNCTION Statement..………248 CREATE ROLE Statement(Impala 2.0 or higher only) 254 CREATE TABLE Statement .255 CREATE VIEW Statement DELETE Statement(Impala 2.8 or higher only) 271 DESCRIBE Statement 273 DROP DATABASE Statement 285 DROP FUNCTION Statement .287 DROP ROLE Statement (Impala 2.0 or higher only) 288 DROP STATS Statement 289 DROP TABLE Statement.………… ········· DROP VIEW Statement 295 eXPLAiN Statement .296 GRANT Statement(Impala 2.0 or higher only INSERT Statement……………… ,301 INVALIDATE METADATA STatement LOAD DATA Statement 313 Contents vi REFRESH Statement ······· .317 REVOKE STatement(Impala 2.0 or higher only) ..........321 SELECT Statement +++++ 322 SET Statement .352 SHOW Statement 392 TRUNCATE TABLE Statement (Impala 2.3 or higher only) 411 UPDATE Statement (Impala 2.8 or higher only) .413 UPSERT Statement(Impala 2. 8 or higher only) 415 USE State 416 Optimizer hints 416 Impala built-In Functions 421 Impala mathenatical functions ……423 Impala bit functio Impala Type Conversion Functions 453 Impala date and Time functions 457 Impala Conditional Functions 508 Impala string Functions .515 Impala miscellaneous Functions Impala Aggregate Functions .542 Impala analytic functions Impala User-Defined Functions(UDFS)……… 592 SQL DIfferences Between Impala and Hive HiveQL Features not Available in Impala .609 Semantic Differences Between Impala and HiveQL Features.……… 610 Porting SqL from Other Database Systems to Impala 611 Porting DDL and DML Statements “+“+++““““ Porting Data Types from Other Database Systems .612 SQL Statements to Remove or Adapt 614 SQL Constructs to Doublecheck....... .615 Next Porting Steps after Verifying Syntax and Semantics……………… ∴616 Using the Impala shell (impala-shell Command).. 曲自自自盘自自自鲁西自西a 617 impala-shell Configuration Options 617 Summary of impala-shell Configuration Options 617 ala-shell Configuration File Connecting to impala through impala-shell 622 Running Commands and sQl Statements in impala-shell............ 623 Rerunning impala-shell commands impala-shell Command Reference 628 Tuning Impala for Performance.. 631 Impala Performance Guidelines and Best Practices 632 Performance Considerations for Join Queries 634 How Joins are processed when Statistics Are Unavailable........................ 635 Overriding Join Reordering with STRAIGHT JOIN 635 Examples of join Order optimization 636 Table and column Statistics………………… 641 Overview of Table statistics Overview of coli Statistics 642 How Table and Column Statistics Work for Partitioned Tables 644 Generating Table and Column Statistics 646 Detecting Missing Statistics 649 Manually Setting Table and Column Statistics with ALTER TABLE 651 Examples of Using Table and Column Statistics with Impala 652 Contents vii Benchmarking Impala Queries... 656 Controlling Impala Resource Usage. …657 Runtime filtering for Impala Queries (Impala 2.5 or higher only) 657 ckground Information for Runtime Filtering....….……657 Runtime Filtering Internals 58 File Format Considerations for Runtime Filtering Wait intervals for runtime filters 659 Query Options for Runtime Filtering.…… 659 Runtime Filtering and Query Plans. …659 Examples of queries that benefit from runtime filtering 660 Tuning and Troubleshooting Queries that Use Runtime Filtering 661 Limitations and restrictions for Runtime filtering +···· 662 Using HDfS Caching with Impala(Impala 2. 1 or higher only) 662 Overview of HDFS Caching for Impala 62 Setting Up HDFS Caching for Impala 663 Enabling HDFS Caching for Impala Tables and Partitions .663 Loading and Removing Data with HDFS Caching Enabled *·······“· 664 Administration for HDFS Caching with Impala 665 Performance Considerations for HDFS Caching with impala Testing Impala performance Understanding Impala Query Performance-EXPLAIN Plans and Query Profiles 668 USing the eXplain Plan for Performance Tuning... Using the SUMMARY Report for Performance Tuning……… 669 Using the Query profile for Performance Tuning……… .671 Detecting and Correcting HDFS Block Skew Conditions 677 Scalability Considerations for Impala.…………678 Impact of Many Tables or Partitions on Impala Catalog Performance and Memory Usage 678 Scalability considerations for the Impala statestore 679 Controlling which Hosts are Coordinators and Executors 680 Effect of Buffer Pool on Memory Usage(Impala 2.10 and higher) .680 SQL Operations that Spill to Disk. 681 Limits on Query Size and Complexity.…… .684 Scalability Considerations for Impala 1/o 684 Scalability considerations for Table layout 685 Kerberos-Related Network Overhead for Large clust 685 Avoiding CPU Hotspots for HDFS Cached Data ·· Scalability Considerations for NameNode Traffic with File Handle Caching 686 Partitioning for Impala tables When to Use Partitioned Tables .687 SQL Statements for Partitioned tables 687 Static and Dynamic Partitioning Clauses .688 Refreshing a Single Partition................…….…..688 Permissions for partition subdirectories 688 Partition Pruning for Queries 689 Checking if Partition Pruning Happens for a Query What SQL Constructs Work with Partition Pruning. 689 Dynamic Partition Pruning…… 690 Partition Key Columns 6 Setting Different File Formats for Partitions 丰·丰 692 Managing Partitions.............. Using Partitioning with Kudu tables 693 Keeping Statistics Up to Date for Partitioned Tables I Contents I How Impala works with Hadoop File Formats Choosing the File Format for a table 697 USing Text Data Files with Impala tables 698 Query performance for Impala text tables 698 Creating Text Tables. 1·申 Data files for text tables …700 Loading Data into Impala Text tables .701 USing LZO-Compressed Text Files.. 702 USing gzip, bzip 2, or Snappy-Compressed Text Files 704 Using the Parquet File Format with Impala Tables 705 Creating Parquet Tables in Impala.………… 705 Loading data into Parquet Tables………… 706 Query Performance for Impala Parquet Tables...... Snappy and gzip compression for Parquet Data Files Parquet Tables for Impala Complex Types 712 Exchanging Parquet Data Files with Other Hadoop Components 712 How Parquet Data Files Are O 715 Compacting Data Files for Parquet tables …·· 716 Schema Evolution for Parquet Tables 717 Data Type Considerations for Parquet tables 718 Using the Avro File Format with Impala tables....,……….… 719 reating 719 Using a Hive- Created Avro table in Impala ing the Avro so through JSON 722 Loading data into an avro table 722 Enabling Compression for Avro tables 722 How Impala handles avro Schema evolution +·+ 722 Data Type Considerations for Avro Tables.....................724 Query Performance for Impala Avro Tables... ..724 Using the rcfile file format with Impala tables 724 Creating RCFile Tables and Loading Data 1725 Enabling Compression for rCFile tables 726 Query Performance for Impala rcFile tables 727 Using the Sequencefile File Format with Impala Tables 727 Creating Sequence File Tables and Loading Data 727 Enabling Compression for SequenceFile Tables Query Performance for Impala SequenceFile Tables. 728 729 Using Impala to Query Kudu tables,…,,…,,,,,…,,,…729 Benefits of Using Kudu Tables with Impala 730 Configuring Impala for Use with Kudu 730 Cluster Topology for Kudu Tables .730 Impala ddl Enhancements for Kudu Tables (CREATE TaBle and ALTER TABLE) Primary Key columns for Kudu Tables 731 Kudu-Specific Column Attributes for CREATE TABLE 731 Partitioning for Kudu tables 735 Handling date, Time, or Timestamp Data with Kudu....... 738 How Impala Handles Kudu metadata......... Loading data into Kudu Tables 741 Impala dml support for Kudu Tables (insert, UPDatE, dElete, UPSERT) 741 Consistency Considerations for Kudu tables 742 Security Considerations for Kudu tables ……742 Impala query performance for Kudu Tables 743 Contents ix Using Impala to Query HBase Tables Overview of Using HBase with Impala “·· 743 Configuring HBase for Use with Impala Supported Data Types for HBase Columns Performance Considerations for the Impala-HBase Integration ……745 Use Cases for Querying HBase through Impala .750 Loading Data into an HBase table 751 Limitations and Restrictions of the Impala and HBase Integration 751 Examples of Querying hBase Tables from impala 751 Using Impala with the Amazon S3 Filesystem.. ●●●●●●·自●●自●D自● 753 How Impala sql statements Work with $3 754 Specifying Impala Credentials to Access Data in S3...….….….….…………75 Loading data into S3 for Impala queries .755 Using Impala dml statements for S3 Data 755 Manually Loading Data into Impala Tables on $3 755 Creating Impala Databases, Tables, and Partitions for Data Stored on $3 756 Internal and External Tables located on S3 .757 Running and Tuning Impala Queries for Data Stored on S3 759 Understanding and Tuning Impala Query Performance for S3 Data Restrictions on Impala support for S3 760 Best Practices for Using Impala with $3 760 Using Impala with the azure Data Lake Store Prerequisites…… .761 How Impala sQl statements Work with ADLs..... Specifying Impala Credentials to Access Data in ADLS... 762 Loading data into ADLS for Impala Queries……… 762 Using Impala dml statements for ADLS Data Manually loading Data into Impala Tables on ADLS 763 Creating Impala Databases, Tables, and Partitions for Data Stored on ADLS ∴763 Internal and External tables located on adls 1765 Running and Tuning Impala Queries for Data Stored on ADLS …767 Understanding and Tuning Impala Query Performance for ADLS Data 767 Restrictions on Impala Support for ADLS .768 Best Practices for Using Impala with ADLS ….768 Using Impala with Isilon Storage……,…,.,,,,…,,…,…,,…768 Using Impala loggingoo.oo..... 769 Locations and Names of Impala log Files 769 770 Rotating Impala logs 70 Reviewing Impala Logs Understanding Impala Log Contents .71 Setting Logging levels Redacting Sensitive Info 772 Troubleshooting Impala.oo. 772 I Contents X Troubleshooting Impala SQL Syntax Issues 772 Troubleshooting I/O Capacity Problens....... .773 Impala troubleshooting quick reference 773 Impala web User Interface for Debugging…… 75 Debug Web UI for impala 776 Breakpad Minidumps for Impala(Impala 2.6 or higher only) 777 Enabling or Disabling Minidump Generation……… 777 Specifying the Location for Minidump files 778 Controlling the Number of Minidump Files 778 Detecting Crash Events 778 Demonstration of Breakpad Feature 778 Ports Used by Impala… 780 Impala reserved words....0. 781 List of reserved words Impala Frequently Asked Questions,…,…,,…,…,…,,…,…,…,…,…,,……,797 Impala release notes……,,…,,…,,,,,,…,,,,798 New Features in Apache Impala 798 New Features in Impala 3.0 798 New Features in Impala 2. 12 798 New Features in Impala 2.11 New Features in Impala 2.10 798 New Features in Impala 2.9. ……………798 New Features in Impala 2.8 798 New Features in Impala 2.7 New Features in Impala 2.6 801 New Features in Impala 2.5 New Features in Impala 2. 4.... 806 New Features in Impala 2 806 New Features in Impala 2.8 .808 New Features in Impala 2.1 810 ew Features in Impala 2.0 810 New Features in Impala 1.4 812 New Features in Impala 1.3.2 813 New Features in Impala 1.3.1 .814 New Features in Impala 1.3..............814 New Features in Impala 1.2.4 8l5 New Features in Impala 1. 2.3 ∴815 New Features in Impala 1.2.2 .815 in impala 1. 2.1 816 ew Features in Impala 1. 2.0(Beta). 817 New Features in Impala 1.1.1 819 New Features in Impala 1 ..819 New Features in Impala 1.0.1 820 New Features in Impala 1.0........... .820 New Features in Version 0.7 of the Impala beta release 820 New Features in Version0.6 of the Impala Beta release………… 821 New Features in Version0.5 of the Impala beta release..……….821 New Features in Version 0.4 of the Impala Beta Release............821

...展开详情
试读 127P impala3.0参考英文版
立即下载 低至0.43元/次 身份认证VIP会员低至7折
一个资源只可评论一次,评论内容不能少于5个字
  • 分享宗师

    成功上传21个资源即可获取
关注 私信 TA的资源
上传资源赚积分or赚钱
    最新推荐
    impala3.0参考英文版 49积分/C币 立即下载
    1/127
    impala3.0参考英文版第1页
    impala3.0参考英文版第2页
    impala3.0参考英文版第3页
    impala3.0参考英文版第4页
    impala3.0参考英文版第5页
    impala3.0参考英文版第6页
    impala3.0参考英文版第7页
    impala3.0参考英文版第8页
    impala3.0参考英文版第9页
    impala3.0参考英文版第10页
    impala3.0参考英文版第11页
    impala3.0参考英文版第12页
    impala3.0参考英文版第13页
    impala3.0参考英文版第14页
    impala3.0参考英文版第15页
    impala3.0参考英文版第16页
    impala3.0参考英文版第17页
    impala3.0参考英文版第18页
    impala3.0参考英文版第19页
    impala3.0参考英文版第20页

    试读结束, 可继续阅读

    49积分/C币 立即下载 >