目录
第一部分 Spark学习 ....................................................................................................................... 6
第1章 Spark介绍 ................................................................................................................... 7
1.1 Spark简介与发展 ...................................................................................................... 7
1.2 Spark特点 .................................................................................................................. 7
1.3 Spark与Hadoop集成 ................................................................................................ 7
1.4 Spark组件 .................................................................................................................. 8
第2章 Spark弹性分布数据集 ............................................................................................... 9
2.1 弹性分布式数据集 .................................................................................................... 9
2.2 MapReduce数据分享效率低..................................................................................... 9
2.3 MapReduce进行迭代操作 ........................................................................................ 9
2.4 MapReduce进行交互操作 ...................................................................................... 10
2.5 Spark RDD数据分享 ............................................................................................... 10
2.6 Spark RDD 迭代操作 .............................................................................................. 10
2.7 Spark RDD交互操作 ............................................................................................... 10
第3章 Spark安装 ................................................................................................................. 11
第4章 Spark CORE编程 ....................................................................................................... 13
4.1 Spark Shell ................................................................................................................ 13
4.2 RDD Transformations ............................................................................................... 13
4.3 Actions ...................................................................................................................... 16
4.4 用RDD编程 ........................................................................................................... 17
4.5 UN-Persist存储 ........................................................................................................ 18
第5章 Spark调度与高级编程 ............................................................................................. 20
5.1 Spark应用程序例子 ................................................................................................ 20
5.2 Spark-submit语法 .................................................................................................... 22
5.3 Spark变量 ................................................................................................................ 23
5.4 数字类型 RDD操作 .............................................................................................. 23
第二部分 ZOOKEEPER学习 .......................................................................................................... 24
第6章 zookeeper介绍 ......................................................................................................... 25
6.1 zookeeper简介 ......................................................................................................... 25
6.2 分布式应用程序 ...................................................................................................... 25
6.3 Apache Zookeeper意味着什么？ ............................................................................ 26
第7章 zookeeper基本组成与工作流程 ............................................................................. 27
第8章 zookeeper的leader节点选择 ................................................................................. 31
第9章 zookeeper安装 ......................................................................................................... 33
第10章 zookeeper 命令行接口 .......................................................................................... 35
第11章 zookeeper应用程序接口 ....................................................................................... 39
第12章 zookeeper应用 ....................................................................................................... 40
第三部分 KAFKA学习 ................................................................................................................... 48
第12章 KAFKA介绍 ............................................................................................................. 49
12.1 KAFKA简介 .......................................................................................................... 49
12.2信息系统 ................................................................................................................. 49
12.3 KAFKA是什么？ .................................................................................................. 50
第13章 KAFKA基本组成与集群架构 ................................................................................. 51
13.1 KAFKA的基本组成 .............................................................................................. 51
13.2 KAFKA集群架构 .................................................................................................. 52
第14章 KAFKA工作流程 ..................................................................................................... 53
14.1 PUB-SUB信息工作流 ........................................................................................... 53
14.2 队列信息工作流/消费者组 ................................................................................... 53
14.3 Zookeeper在KAFKA中扮演的角色 ................................................................... 54
第15章 KAFKA安装 ............................................................................................................. 55
第16章 KAFKA基本操作 ..................................................................................................... 56
16.1 启动zookeeper服务 ............................................................................................. 56
16.2 单个单节点中间件配置 ........................................................................................ 56
16.3 Topics列表 ............................................................................................................. 56
16.4 启动生产者发送信息 ............................................................................................ 57
16.5 启动消费者接收信息 ............................................................................................ 57
16.6 单个多节点中间件配置 ........................................................................................ 57
16.7 创建一个topic ....................................................................................................... 58
16.8 启动生产者发送信息 ............................................................................................ 59
16.9 启动消费者接收信息 ............................................................................................ 59
16.10 基本Topic操作 ................................................................................................... 59
16.11 删除Topic ............................................................................................................ 59
第17章 KAFKA 生产者与消费者群实例 ............................................................................ 60
17.1 生产者实例 ............................................................................................................ 60
17.2 简单消费者实例 .................................................................................................... 63
17.3 消费者群例子 ........................................................................................................ 65
第18章 KAFKA与SPARK集成 ............................................................................................. 67
18.1 Kafka与spark集成 ............................................................................................... 67
18.2 SparkConf API ........................................................................................................ 67
18.3 StreamingContext API ............................................................................................ 67
18.4 KafkaUtils API ........................................................................................................ 67
18.5 建立脚本 ................................................................................................................ 69
18.6 编译/打包 ............................................................................................................... 69
18.7 提交到Spark ......................................................................................................... 69
第四部分HIVE学习 ....................................................................................................................... 70
第19章 HIVE介绍 ................................................................................................................ 71
19.1 HIVE是什么？ ...................................................................................................... 71
19.2 HIVE特点 .............................................................................................................. 71
19.3 HIVE架构 .............................................................................................................. 71
19.5 HIVE工作流 .......................................................................................................... 72
第20章 HIVE 安装 ............................................................................................................... 74
20.1 Hadoop安装 ........................................................................................................... 74
20.2 HIVE安装 .............................................................................................................. 77
20.3 Derby安装与设置 .................................................................................................. 78
第21章 HIVE 数据类型 ....................................................................................................... 80
21.1列类型(Column Type) ............................................................................................ 80
21.2文本类型(Literals) .................................................................................................. 81
21.3 Null 值.................................................................................................................... 81
21.4 复杂类型 ................................................................................................................ 81
第22章 数据库操作 ............................................................................................................. 82
22.1 创建数据库 ............................................................................................................ 82
22.2 删除数据库 ............................................................................................................ 82
第23章 数据表操作 ............................................................................................................. 83
23.1 创建数据表 ............................................................................................................ 83
23.2 load数据(插入数据) .............................................................................................. 85
23.3 修改数据表(Alter table) ........................................................................................ 86
23.4 删除表(Drop table) ................................................................................................ 90
第24章 分区 ......................................................................................................................... 92
24.1 添加分区(Adding a Partition) ................................................................................ 93
24.2 重命名分区(Renaming a Partition) ....................................................................... 93
24.3 删除分区(Droping a Partition) .............................................................................. 93
第25章 内置运算符 ............................................................................................................. 94
25.1 关系运算符 ............................................................................................................ 94
25.2 算术运算符 ............................................................................................................ 96
25.3 逻辑运算符 ............................................................................................................ 97
25.4 复杂运算符(Complex Operators) .......................................................................... 97
第26章 内置函数 ................................................................................................................. 98
26.1 内置函数 ................................................................................................................ 98
26.2 聚合函数(Aggregate Functions) ............................................................................ 99
第27章 视图与索引 ........................................................................................................... 100
27.1 创建视图（Creating a View） ............................................................................ 100
27.2 删除视图(Dropping a View) ................................................................................ 100
27.3 创建索引(Creating an Index)............................................................................... 101
27.4 删除索引(Dropping an Index) ............................................................................. 101
第28章 HIVEQL ................................................................................................................... 102
28.1 查询语句(SELECT ...WHERE) .......................................................................... 102
28.1.1 查询语句实例 ................................................................................................... 102
28.1.2 JDBC查询语句实例 ......................................................................................... 102
28.2 查询语句(SELECT...ORDER BY) ...................................................................... 103
28.2.1 ORDER BY查询语句实例 ............................................................................... 103
28.2.2 JDBC ORDER BY 查询语句实例 ................................................................... 104
28.3 查询语句(GROUP BY) ....................................................................................... 104
28.3.1 GROUP BY查询语句实例 ............................................................................... 104
28.3.2 JDBC GROUP BY查询语句实例 .................................................................... 105
28.4 查询语句(JOIN) .................................................................................................. 106
28.4.1 JOIN查询语句实例 .......................................................................................... 106
第五部分HBASE学习 .................................................................................................................. 111
第29章 HBASE介绍 ........................................................................................................... 112
29.1 HBase是什么？ ................................................................................................... 112
29.2 HBase与HDFS .................................................................................................... 113
29.3 HBase存储机制 ................................................................................................... 113
29.4 面向列和面向行 .................................................................................................. 113
29.5 HBase与RDBMS ................................................................................................ 114
29.6 HBase优点 ........................................................................................................... 114
29.7 HBase使用场景 ................................................................................................... 114
29.8 HBase的应用 ....................................................................................................... 114
第30章 HBASE架构与安装 ............................................................................................... 115
30.1 HBase架构 ........................................................................................................... 115
30.1.1主服务器(Master Server) ................................................................................... 115
30.1.2 分区及分区服务器 ........................................................................................... 115
30.1.3 Zookeeper ........................................................................................................... 116
30.2 HBase安装 ........................................................................................................... 116
第31章 HBASE的SHELL .................................................................................................... 121
31.1 一般命令 .............................................................................................................. 121
31.2 数据定义语言(DDL) ........................................................................................... 121
31.3数据操纵语言(DML) ........................................................................................... 121
第32章 HBASE普通命令 ................................................................................................... 122
32.1 status...................................................................................................................... 122
32.2 version ................................................................................................................... 122
32.3 table_help .............................................................................................................. 122
32.4 whoami .................................................................................................................. 122
第33章 HBASE ADMIN API ................................................................................................. 123
33.1 HBaseAdmin类 .................................................................................................... 123
33.2 Class Descriptor .................................................................................................... 123
33.3 Constructors(构造函数) ........................................................................................ 123
