impala-2.9.pdf_Impala文档资源-CSDN文库

impala文档

需积分: 9 5 浏览量 2021-10-12 09:04:20 上传评论收藏 3.57MB PDF 举报

资源推荐

资源详情

资源评论

Apache Impala (incubating) Guide

| Contents | ii

Contents

Introducing Apache Impala (incubating).............................................................13

Impala Benefits................................................................................................................................................... 13

How Impala Works with Apache Hadoop.........................................................................................................13

Primary Impala Features.....................................................................................................................................14

Impala Concepts and Architecture.......................................................................14

Components of the Impala Server......................................................................................................................14

The Impala Daemon............................................................................................................................... 14

The Impala Statestore............................................................................................................................. 15

The Impala Catalog Service................................................................................................................... 15

Developing Impala Applications........................................................................................................................ 16

Overview of the Impala SQL Dialect.................................................................................................... 16

Overview of Impala Programming Interfaces........................................................................................17

How Impala Fits Into the Hadoop Ecosystem................................................................................................... 17

How Impala Works with Hive............................................................................................................... 17

Overview of Impala Metadata and the Metastore..................................................................................18

How Impala Uses HDFS........................................................................................................................ 18

How Impala Uses HBase....................................................................................................................... 18

Planning for Impala Deployment..........................................................................18

Impala Requirements.......................................................................................................................................... 18

Supported Operating Systems................................................................................................................ 18

Hive Metastore and Related Configuration............................................................................................19

Java Dependencies.................................................................................................................................. 19

Networking Configuration Requirements...............................................................................................19

Hardware Requirements..........................................................................................................................20

User Account Requirements................................................................................................................... 20

Cluster Sizing Guidelines for Impala.................................................................................................................20

Guidelines for Designing Impala Schemas........................................................................................................22

Installing Impala..................................................................................................... 24

What is Included in an Impala Installation........................................................................................................24

Managing Impala....................................................................................................25

Post-Installation Configuration for Impala.........................................................................................................25

Configuring Impala to Work with ODBC......................................................................................................... 26

Configuring Impala to Work with JDBC...........................................................................................................27

Configuring the JDBC Port.................................................................................................................... 27

Choosing the JDBC Driver.................................................................................................................... 27

Enabling Impala JDBC Support on Client Systems.............................................................................. 27

Establishing JDBC Connections.............................................................................................................28

Notes about JDBC and ODBC Interaction with Impala SQL Features................................................. 29

Upgrading Impala...................................................................................................30

| Contents | iii

Upgrading Impala............................................................................................................................................... 30

Starting Impala....................................................................................................... 31

Starting Impala from the Command Line..........................................................................................................31

Modifying Impala Startup Options.................................................................................................................... 32

Configuring Impala Startup Options through the Command Line.........................................................32

Checking the Values of Impala Configuration Options.........................................................................34

Startup Options for impalad Daemon.................................................................................................... 34

Startup Options for statestored Daemon................................................................................................ 34

Startup Options for catalogd Daemon....................................................................................................34

Impala Tutorials..................................................................................................... 34

Tutorials for Getting Started.............................................................................................................................. 35

Explore a New Impala Instance............................................................................................................. 35

Load CSV Data from Local Files.......................................................................................................... 40

Point an Impala Table at Existing Data Files........................................................................................ 42

Describe the Impala Table......................................................................................................................43

Query the Impala Table..........................................................................................................................44

Data Loading and Querying Examples.................................................................................................. 45

Advanced Tutorials............................................................................................................................................. 47

Attaching an External Partitioned Table to an HDFS Directory Structure............................................47

Switching Back and Forth Between Impala and Hive...........................................................................49

Cross Joins and Cartesian Products with the CROSS JOIN Operator...................................................50

Dealing with Parquet Files with Unknown Schema.......................................................................................... 52

Impala Administration........................................................................................... 66

Admission Control and Query Queuing.............................................................................................................67

Overview of Impala Admission Control................................................................................................ 67

Concurrent Queries and Admission Control.......................................................................................... 68

Memory Limits and Admission Control................................................................................................ 68

How Impala Admission Control Relates to Other Resource Management Tools..................................68

How Impala Schedules and Enforces Limits on Concurrent Queries....................................................69

How Admission Control works with Impala Clients (JDBC, ODBC, HiveServer2).............................69

SQL and Schema Considerations for Admission Control..................................................................... 70

Configuring Admission Control............................................................................................................. 70

Resource Management for Impala..................................................................................................................... 75

How Resource Limits Are Enforced......................................................................................................75

impala-shell Query Options for Resource Management........................................................................ 76

Limitations of Resource Management for Impala................................................................................. 76

Setting Timeout Periods for Daemons, Queries, and Sessions..........................................................................76

Increasing the Statestore Timeout.......................................................................................................... 76

Setting the Idle Query and Idle Session Timeouts for impalad.............................................................76

Setting Timeout and Retries for Thrift Connections to the Backend Client.......................................... 77

Cancelling a Query................................................................................................................................. 77

Using Impala through a Proxy for High Availability........................................................................................ 77

Overview of Proxy Usage and Load Balancing for Impala...................................................................78

Special Proxy Considerations for Clusters Using Kerberos.................................................................. 78

Example of Configuring HAProxy Load Balancer for Impala.............................................................. 79

Managing Disk Space for Impala Data..............................................................................................................81

Impala Security....................................................................................................... 82

Security Guidelines for Impala.......................................................................................................................... 83

| Contents | iv

Securing Impala Data and Log Files................................................................................................................. 84

Installation Considerations for Impala Security.................................................................................................84

Securing the Hive Metastore Database.............................................................................................................. 84

Securing the Impala Web User Interface...........................................................................................................84

Configuring TLS/SSL for Impala...................................................................................................................... 85

Using the Command Line...................................................................................................................... 85

Using TLS/SSL with Business Intelligence Tools.................................................................................86

Enabling Sentry Authorization for Impala.........................................................................................................86

The Sentry Privilege Model................................................................................................................... 86

Starting the impalad Daemon with Sentry Authorization Enabled........................................................87

Using Impala with the Sentry Service (Impala 1.4 or higher only).......................................................88

Using Impala with the Sentry Policy File..............................................................................................88

Setting Up Schema Objects for a Secure Impala Deployment.............................................................. 93

Privilege Model and Object Hierarchy.................................................................................................. 93

Debugging Failed Sentry Authorization Requests................................................................................. 97

The DEFAULT Database in a Secure Deployment...............................................................................97

Impala Authentication.........................................................................................................................................97

Enabling Kerberos Authentication for Impala....................................................................................... 98

Enabling LDAP Authentication for Impala......................................................................................... 101

Using Multiple Authentication Methods with Impala..........................................................................103

Configuring Impala Delegation for Hue and BI Tools........................................................................103

Auditing Impala Operations............................................................................................................................. 104

Durability and Performance Considerations for Impala Auditing....................................................... 104

Format of the Audit Log Files............................................................................................................. 104

Which Operations Are Audited............................................................................................................105

Viewing Lineage Information for Impala Data............................................................................................... 105

Impala SQL Language Reference.......................................................................106

Comments..........................................................................................................................................................106

Data Types........................................................................................................................................................ 107

ARRAY Complex Type (Impala 2.3 or higher only).......................................................................... 107

BIGINT Data Type...............................................................................................................................110

BOOLEAN Data Type......................................................................................................................... 112

CHAR Data Type (Impala 2.0 or higher only)....................................................................................113

DECIMAL Data Type (Impala 1.4 or higher only).............................................................................116

DOUBLE Data Type............................................................................................................................ 125

FLOAT Data Type............................................................................................................................... 126

INT Data Type......................................................................................................................................127

MAP Complex Type (Impala 2.3 or higher only)............................................................................... 128

REAL Data Type.................................................................................................................................. 132

SMALLINT Data Type........................................................................................................................ 132

STRING Data Type.............................................................................................................................. 133

STRUCT Complex Type (Impala 2.3 or higher only).........................................................................135

TIMESTAMP Data Type..................................................................................................................... 141

TINYINT Data Type............................................................................................................................ 148

VARCHAR Data Type (Impala 2.0 or higher only)............................................................................149

Complex Types (Impala 2.3 or higher only)....................................................................................... 151

Literals...............................................................................................................................................................182

Numeric Literals................................................................................................................................... 183

String Literals........................................................................................................................................184

Boolean Literals.................................................................................................................................... 185

Timestamp Literals............................................................................................................................... 185

NULL.................................................................................................................................................... 185

SQL Operators.................................................................................................................................................. 187

Arithmetic Operators............................................................................................................................ 187

| Contents | v

BETWEEN Operator............................................................................................................................ 190

Comparison Operators.......................................................................................................................... 191

EXISTS Operator..................................................................................................................................192

ILIKE Operator.....................................................................................................................................195

IN Operator........................................................................................................................................... 196

IREGEXP Operator.............................................................................................................................. 199

IS DISTINCT FROM Operator............................................................................................................200

IS NULL Operator................................................................................................................................202

LIKE Operator...................................................................................................................................... 203

Logical Operators..................................................................................................................................203

REGEXP Operator................................................................................................................................206

RLIKE Operator....................................................................................................................................208

Impala Schema Objects and Object Names..................................................................................................... 209

Overview of Impala Aliases.................................................................................................................209

Overview of Impala Databases............................................................................................................ 210

Overview of Impala Functions............................................................................................................. 211

Overview of Impala Identifiers............................................................................................................ 212

Overview of Impala Tables.................................................................................................................. 213

Overview of Impala Views.................................................................................................................. 217

Impala SQL Statements.................................................................................................................................... 221

DDL Statements....................................................................................................................................221

DML Statements................................................................................................................................... 222

ALTER TABLE Statement.................................................................................................................. 223

ALTER VIEW Statement.....................................................................................................................236

COMPUTE STATS Statement............................................................................................................. 237

CREATE DATABASE Statement....................................................................................................... 244

CREATE FUNCTION Statement.........................................................................................................246

CREATE ROLE Statement (Impala 2.0 or higher only)..................................................................... 253

CREATE TABLE Statement................................................................................................................253

CREATE VIEW Statement.................................................................................................................. 267

DELETE Statement (Impala 2.8 or higher only)................................................................................. 269

DESCRIBE Statement.......................................................................................................................... 271

DROP DATABASE Statement............................................................................................................ 283

DROP FUNCTION Statement............................................................................................................. 285

DROP ROLE Statement (Impala 2.0 or higher only)..........................................................................286

DROP STATS Statement..................................................................................................................... 287

DROP TABLE Statement.....................................................................................................................291

DROP VIEW Statement....................................................................................................................... 293

EXPLAIN Statement............................................................................................................................ 294

GRANT Statement (Impala 2.0 or higher only).................................................................................. 297

INSERT Statement................................................................................................................................298

INVALIDATE METADATA Statement..............................................................................................307

LOAD DATA Statement...................................................................................................................... 310

REFRESH Statement............................................................................................................................ 314

REVOKE Statement (Impala 2.0 or higher only)................................................................................318

SELECT Statement............................................................................................................................... 319

SET Statement...................................................................................................................................... 345

SHOW Statement..................................................................................................................................378

TRUNCATE TABLE Statement (Impala 2.3 or higher only)............................................................. 396

UPDATE Statement (Impala 2.8 or higher only)................................................................................ 399

UPSERT Statement (Impala 2.8 or higher only)................................................................................. 400

USE Statement...................................................................................................................................... 401

Impala Built-In Functions.................................................................................................................................402

Impala Mathematical Functions........................................................................................................... 403

Impala Bit Functions............................................................................................................................ 418

Impala Type Conversion Functions..................................................................................................... 430