impala文档

所需积分/C币:17 2017-06-13 14:17:13 3.48MB PDF
11
收藏 收藏
举报

impala官方文档,2.8最新版
I Contents ii Upgrading Impala Starting Impala e。。●●e 31 Starting Impala from the Command Line …31 Modifying Impala Startup Options 32 Configuring Impala Startup Options through the Command Line 32 Checking the Values of Impala Configuration Options 34 Startup Options for impalad Daemon 34 Startup Options for statestored Daemon 34 Startup Options for catalog Daemon 134 Impala Tutorials. ··●b··········●·●●··●··●··●●·● 34 Tutorials for Getting Started 35 Explore a New Impala Instance ·······*:· 35 d csv data from local files Point an Impala table at Existing data Files Describe the Impala Table 43 the Impala table Data Loading and Querying Examples 45 Advanced tutorials 甲·甲····· Attaching an External Partitioned table to an hdfs directory Structure 47 Switching Back and Forth Between Impala and hive Cross Joins and cartesian products with the cross join Operator. ..................................................50 Dealing with Parquet Files with Unknown Schema.……… ∴52 Impala Administration..........…..…………………66 Admission Control and Query Queuing. Overview of Impala Adr n control 67 Concurrent Queries and Admission Control....... .68 Memory limits and Admission Control How Impala admission Control relates to Other Resource management Tools How Impala Schedules and Enforces Limits on Concurrent Queries How Admission Control works with Impala Clients(JDBC, ODBC, Hive Server 2) SQL and Schema Considerations for Admission Control 70 Configuring Admission Control. Resource Management for Impala.... 75 How resource limits are enforced 75 impala-shell Query Options for Resource management .76 Limitations of Resource Management for Impala.………… 76 Setting Timeout Periods for Daemons, Queries, and Sessions... 76 Increasing the Statestore Timeout ··中·· 76 Setting the Idle query and idle session Timeouts for impala....... 76 Sctting timeout and retries for thrift connections to the backend client elling a Query 77 Using Impala through a Proxy for High Availability Overview of Proxy Usage and Load Balancing for Impala 77 Special Proxy Considerations for Clusters Using Kerberos .78 Example of Configuring HAProxy Load Balancer for Impala 79 Managing Disk Space for Impala Data Impala security 82 Security Guidelines for Impala Contents iv Securing Impala Data and Log Files Installation Considerations for Impala Security Securing the hive metastore database Securing the Impala Web User Interface Configuring TLS/SSL for Impala .85 Using the command line 85 Using tls/ssl with business Intelligence Tools Enabling Sentry Authorization for Impala 86 The sentry Privilege Model.... 86 Starting the impalad daemon with Sentry Authorization Enabled 87 Using Impala with the Sentry Service(Impala 1. 4 or higher only)..... 88 USing Impala with the Sentry Policy File Setting Up Schema Objects for a Secure Impala Deployment 93 Pr Model and object Hierarch Debugging Failed Sentry Authorization Requests The DEFAULT Database in a Secure Deployment 97 Impala authentication …97 Enabling Kerberos Authentication for Impala 98 Enabling LDAP Authentication for Impala 100 Using Multiple Authentication Methods with Impala.………….……….……….…….,02 Configuring Impala Delegation for Hue and BI Tools Auditing Impala Operations 103 Durability and Performance Considerations for Impala Auditing .103 Format of the Audit Log Files 104 Which Operations Are Audited Viewing Lineage Information for Impala Data “+“+“+“+““·“* 105 Impala sQl language reference. ●●鲁·●D自●D●●D·。D● 4105 Comments… 中···“··················*······…··· *············ …106 Data Types.. …106 ARRAY Complex Type(Impala 2.3 or higher only .106 BIGINT Data Type…… …10 BOOLEAN Data Type CHAR Data Type(Impala 2.0 or higher only) ·· 112 DECIMAL Data Type(Impala 1.4 or higher only) 116 DOUBLE Data Typc 124 FLOAT Data Type. .125 INT Data Type. 126 MAP Complex Type(Impala 2.3 or higher only) 127 REAL Data Type 131 SMALLINT Data Type.. 131 STRING Data Type 132 STRUCT Complex Type(Impala 2.3 or higher only) 134 TIMESTAMP Data Type 40 TINYINT Data Type 中···中· VARCHAR Data Type(Impala 2.0 or highcr only).... 147 Complex types(Impala 2.3 or higher only) …………150 Literals l81 umeric Literals 181 String literals 182 Boolean literals Timestamp Literals… 184 SQL Operators .++++++++++++ 185 Arithmetic Operators 185 Contents v BETWEEN Operator………… 188 Comparison Operators......... 189 EXISTS Operator ILIKE Operator.…… 194 IN Operator..... 195 IREGEXP Operator 198 IS DISTINCT FROM Operator. IS NULL Operator 200 LIKE Operator 201 Logical Operators 202 REGEXP Operator 205 RLIKE Operator 206 Impala Schema Objects and object Names 207 Overview of Impala aliases 208 Overview of Impala databases 208 Overview of Impala functions of Impala Identifie 210 Overview of Impala tables 211 Overview of Impala views.... 216 Impala sol statements …219 DDL Statements 20 DML Statements ∴21 ALTER TABLE Statement .221 ALTERⅤ IEW Statement 234 COMPUTE STATS Statement 236 CREATE DATABASE Statement 243 CREATE FUNCTION Statement 245 CREATE ROLE Statement(Impala 2.0 or higher only ). .........251 CREATE TABLE Statement CREATE VIEW Statement 265 DELETE Statement(Impala 2.8 or higher only) DESCRIBE Statement 268 DROP DATABASE Statement 280 DROP FUNCTION Statement ··· 282 DROP ROLE Statement(Impala 2.0 or higher only) 284 DROP STATS Statement 284 DROP TABLE Statement 28 DROP VIEW Statement EXPLAIN Statement GRANT Statement (Impala 2.0 or higher only 295 INSERT Statement 296 INVALIDATE METADATA Statement 304 LOAD DATA Statement 307 REfRESH Statement 311 REVOKE Statement(Impala 2.0 or higher only) 中·非非.“ 315 SELECT Statement 315 SEt Statement 341 SHOW Statement TRUNCATE TABLE Statement (Impala 2.3 or higher only) 391 UPDATE Statement(Impala 2. 8 or higher only) ·········:······ .394 UPSERT Statement(Impala 2.8 or higher only) .........395 USE Statement 396 Impala built-In functions Impala Mathematical Functions 398 la bit f 413 Impala Type Conversion Functions 425 Impala Date and Time functions 429 Impala conditional functions ·············*·“· …2473 Impala string Functions 478 Impala Miscellaneous Functions.……… ggregate Impala analytic Functions Impala User- Defined functions(UDFs)………… 542 SQL Diffcrences Between Impala and Hivc 59 HiveQL Features not Available in Impala ∴559 Semantic Differences Between Impala and HiveQL Features Porting SQL from Other Database Systems to Impala 561 Porting DDL and DML Statements Porting Data Types from Other Database Systems SQL Statements to Remove or Adapt 564 SQL Constructs to Doublecheck 565 Next Porting Steps after Verifying Syntax and Semantics 566 Using the impala shell( impala- shell command)……,,,,…567 impala-shell Configuration Options 567 Summary of impala-shell Configuration Options. 567 impala-shell Configuration file .570 Connecting to impalad through impala-shell Running Commands and SQL Statements in impala-shell impala-shell Command Reference 577 Tuning Impala for Performance. 0 Impala Performance〔 Guidelines and best practices……….….….….….….….…….….….…..………....581 Performance Considerations for Join Queries...... 583 How Joins are processed when statistics are unavailable 584 Overriding Join Reordering with STRAIGHT JOIN Examples of Join Order Optimization 584 Table and Column Statistics........................... 590 Overview of table statistics Overview of Column Statistics 591 How table and column statistics Work for Partitioned tables 592 Overview of incremental statistics 594 Generating Table and Column Statistics(COMPUTE STATS Statement) 5 Detecting Missing Statistics 596 Keeping Statistics Up to Date. Setting the numrows Value Manually through ALTER TABLE 598 Setting Column Stats Manually through ALTER TABLE Examples of Using Table and Column Statistics with Impala 600 Benchmarking Impala Queries 604 Controlling Impala resource ∴.604 Runtime Filtering for Impala Queries(Impala 2. 5 or higher only).......604 Background Information for Runtime Filtering )3 Runtime Filtering Internals..... *······· 606 File Format Considerations for Runtime Filtering 606 Wait Intervals for Runtime filters 606 Query Options for Runtime Filtering……… 607 Runtime Filtering and Query Plans 607 Examples of Queries that Benefit from Runtime Filtering 608 Tuning and Troubleshooting Queries that Use Runtime Filtering 609 Limitations and Restrictions for runtime filtering 609 I Contents vii Using HDFS Caching with Impala(Impala 2. 1 or higher only) Overview of HDFS Caching for Impala ·············: 610 Setting Up HDFS Caching for Impala ++++++ 610 Enabling HDFS Caching for Impala Tables and Partitions……… .610 Loading and removing data with hdfs caching enabled 612 Administration for HDFS Caching with Impala 613 Performance Considerations for HDFS Caching with Impala 613 Testing Impala Performance 615 Understanding Impala Query Performance- EXPLAIN Plans and Query Profiles 616 Using the EXPLAiN Plan for Performance Tuning 616 Using the SUMMARY Report for Performance Tuning Using the Query Profile for Performance Tuning,m, . 617 618 Detecting and Correcting HDFS Block Skew Conditions 624 Scalability Considerations for impala Impact of Many Tables or Partitions on Impala Catalog Performance and Memory Usage 626 Scalability Considerations for the Impala SQL Operations that Spill to Disk .627 Scalability Considera tions for Impala I…… Limits on Query Size and Complexity .631 631 Scalability Considerations for Table Layout 631 Kerberos-Related Network Overhead for Large Clusters 631 Avoiding CPU Hotspots for HDFS Cached Data 632 Partitioning for Impala Tables………,....,….,…,,....…………632 When to use partitioned tables 633 SQL Statements for Partitioned Tables........633 Static and Dynamic Partitioning Clauses.…… ····:···…······:·····4:······ 634 Refreshing a Single Partitio 634 Permissions for partition Subdirectories .634 Partition Pruning for Queries.…… Checking if Partition Pruning Happens for a Query.......………..635 What SQL Constructs Work with Partition Pruning .635 Dynamic Partition Pruning... 636 Partition Key Columns 637 Setting different file Formats for partitions 638 Managing Partitions.… .639 Using Partitioning with Kudu tables.......... 639 How Impala works with Hadoop File Formats... ●●。●@@。●看D● 639 Choosing the file Format for a table 640 Using Text Data Files with Impala Tables……………….………………….641 Query Performance for Impala Text Tables. Creating Text Tables 642 Data files for Text Tables 643 Loading Data into Impala Text Tables 644 USing Lzo-Compressed Text Files 645 USing gzip, bzip 2, or Snappy-Compressed Text Files 647 Using the Parquet File Format with Impala Tables 648 Creating Parquet tables in Impala ing nto Parquet tabl 650 Query Performance for Impala Parquet Tables 51 Snappy and gzip compression for Parquet Data Filc 申非非,非非 653 I Contents viii Parquet Tables for Impala Complex Types. Exchanging Parquet Data Files with Other Hadoop Components 655 How Parquet Data Files Are Organized 658 Compacting data Files for Parquet tables 659 Schema evolution for Parquet Tables 660 Data Type Considerations for Parquet tables 662 Using the Avro File format with Impala tables.…………, Creating Avro tables 663 Using a Hive-Created Avro Table in Impal 665 pecifying the Avro Schema through JSON Loading data into an avro table. 665 Enabling Compression for Avro Tables 665 How Impala Handles Avro Schema Evolution.............. 666 Data Type c ns for Avro table 667 Query Performance for Impala Avro Tables 668 Using the rCFile File Format with Impala tables 668 Creating RCFile Tables and Loading data 668 Enabling Compression for rCFile Tables .669 Query Performance for Impala RCFile Tables 670 Using the SequenceFile File Format with Impala Tables……….….….……………670 Creating SequenceFile Tables and Loading Data 671 Enabling Compression for SequenceFile Tables 671 Query Performance for Impala SequenceFile Tables 672 Using Impala to Query Kudu tables………………673 Benefits of Using Kudu Tables with Impala 673 Configuring Impala for Use with Kudu .673 Cluster Topology for Kudu tables 673 Impala ddl enhancements for Kudu Tables(CREatE TaBle and ALTER TABLE) 非 674 Primary Key Columns for Kudu tables 非丰非 ∴674 Kudu-Specific Column Attributes for CREATE TABLE 674 Partitioning for Kudu tables …···“········· ∴679 Handling date, Time, or Timestamp Data with Kudu How Impala Handles Kudu Metadata... Loading Data into Kudu Tables :············ Impala dml support for Kudu Tables(INSErt, UPDATE, DELETE, UPSERt)......684 Consistency Considerations for Kudu tables Security Considerations for Kudu Tables 685 Impala Query Performance for Kudu Tables 685 Using Impala to Query HBase tables.…….….….…….….……..685 Overview of Using HBase with Impala 686 Configuring HBase for Use with Impala 686 Supported Data Types for HBase Columns... 687 Performance Considerations for the Impala- HBase Integration........ Use Cases for Querying HBase through Impala 692 Loading data into an hBase Table Limitations and restrictions of the Impala and hBasc Intcgration........ 693 Examples of Querying HBase Tables from Impala...……… 694 Using impala with the Amazon s3 filesystem.…………696 How Impala S( L Statements Work with s3………… 696 Specifying Impala Credentials to Access Data in $3 697 Contents ix Loading Data into S3 for Impala Queries..... 697 USing Impala dml statements for S3 Data 697 Manually Loading Data into Impala tables on S .697 Creating Impala Databases, Tables, and Partitions for Data Stored on $........ 698 Internal and External tables located on s3 700 Running and tuning Impala Queries for Data Stored on $3 .701 Understanding and Tuning Impala Query Performance for S3Data…………… Restrictions on Impala Support for S3 .702 Best Practices for Using Impala with $..... ∴…...703 Using Impala with Isilon Storage.. 703 Using Impala logging….....,.,,…,704 Locations and Names of Impala Log files.…… 705 M 705 Rotating Impala Logs ∴…..1705 Reviewing Impala Logs 706 Understanding Impala Log Contents .706 Setting Logging Levels 707 Redacting Sensitive Information from Impala Log Files .707 Troubleshooting Impala Troubleshooting Impala SQL Syntax Issues ·:··中······+·+·::+·······++*················ 708 Troubleshooting I/O Capacity Problems........ 708 Impala Troubleshooting Quick Reference 709 Impala Web User Interface for Debugging 710 Debug Web UI for impala 711 Breakpad Minidumps for Impala(Impala 2.6 or higher only) 713 Enabling or Disabling Minidump Generation…… …713 Specifying the location for Minidump files ,,·。 .713 Controlling the number of minidump files…… .713 Detecting Crash Events 713 Demonstration of Breakpad Feature ∴713 Ports Used by impala............………….715 Impala reserved words. 717 List of current reserved words 717 Planning for Future Reserved Words ………720 Impala Frequently Asked Questions 722 Impala release notes....... 722 Impala release Notes ∴722 New Features in Apache Impala(incubating) .722 ew Features in Impala 2.8 722 New Features in Impala 2.7. 725 New Features in Impala 2.6 25 New Features in Impala 2.5 727 Contents x Impala 2.4 730 New Features in Impala 2.3 …730 New Features in Impala 2.8 732 New Features in Impala 2 734 Impala 2.0 734 New Features in Impala 1.4 736 New Features in Impala I.3.2………… +··*·· 737 New Fcatures in Impala 1.3.1 737 New featu Impala 1.3 738 New Features in Impala 1. 2.4 739 New Features in Impala 1.2.3 New Features in Impala I.2.2.¨… 739 中中中··· 739 New Features in Impala 1.2.1 740 New Features in Impala 1. 2.0(Beta)...... 741 742 tures in Impala 1 743 New Features in Impala 1.0.1 …74 New Fcatures in Impala 1.0 .744 New Features in Version 0.7 of the Impala Beta release................744 New Features in Version 0.6 of the Impala beta release New Features in Version 0.5 of the Impala beta release 745 New Features in Version 0.4 of the Impala Beta Release .745 New Features in Version 0.3 of the Impala beta release 745 New Features in Version 0. 2 of the Impala beta release .745 Incompatible Changes and Limitations in Apache Impala(incubating) 745 Incompatible Changes Introduced in Impala 2.8.x 745 Incompatible Changes Introduced in Impala 2.7.x .746 Incompatible Changes Introduced in Impala 2.6.x .746 Incompatible Changes Introduced in Impala 2.5.X 747 Incompatible Changes Introduced in Impala 2.4.x 748 Incompatible Changes Introduced in Impala 2. 3.x 748 Incompatible Changes Introduced in Impala 2.2.x 748 Incompatible Changes Introduced in Impala2.1.x.………… 749 Incompatible Changes Introduced in Impala 2.0.5 749 Incompatible Changes Introduced in Impala 2.0.4 749 Incompatible Changes Introduced in Impala 2.0.3 749 ncom patible Changes Introduced in Impala2.0.2………….….… 749 Incompatible Changes Introduced in Impala 2.0.1 749 Incompatible Changes Introduced in Impala 2.0.0 749 Incompatible Changes Introduced in Impala 1.4.4.….…….….….…....…….175l Incompatible Changes Introduced in Impala 1. 4.3 …751 Incompatible Changes Introduced in Impala 1.4.2 751 Incompatible Changes Introduced in Impala 1. 4.1 5 Incompatible Changes Introduced in Impala 1. 4.0 751 Incompatible Changes Introduced in Impala 1.3.3 752 compatible Changes Introduced in Impala 1.3.2 752 Incompatible Changes Introduced in Impala 1.3.1 .752 Incompatible Changes Introduced in Impala 1.3.0 Incompatible Changes Introduced in Impala 1.2.4 752 Incompatible Changes Introduced in Impala 1. 2.3....................753 Incompatible Changes Introduced in Impala 1.2.2 753 Incompatible Changes Introduced in Impala 1.2 753 Incompatible Changes Introduced in Impala I.2.0(Beta)…………… Incompatible Changes Introduced in Impala 1.1.1 754 Incompatible Change Introduced in Impala 1.1 754 Incompatible Changes Introduced in Impala 1.0 755

...展开详情
试读 127P impala文档
立即下载 低至0.43元/次 身份认证VIP会员低至7折
一个资源只可评论一次,评论内容不能少于5个字
您会向同学/朋友/同事推荐我们的CSDN下载吗?
谢谢参与!您的真实评价是我们改进的动力~
关注 私信
上传资源赚钱or赚积分
最新推荐
impala文档 17积分/C币 立即下载
1/127
impala文档第1页
impala文档第2页
impala文档第3页
impala文档第4页
impala文档第5页
impala文档第6页
impala文档第7页
impala文档第8页
impala文档第9页
impala文档第10页
impala文档第11页
impala文档第12页
impala文档第13页
impala文档第14页
impala文档第15页
impala文档第16页
impala文档第17页
impala文档第18页
impala文档第19页
impala文档第20页

试读结束, 可继续阅读

17积分/C币 立即下载 >