目 录
1 集群配置说明...............................................................................................................................4
1.1 环境说明...........................................................................................................................4
1.2 目录说明...........................................................................................................................4
2 环境配置与安装...........................................................................................................................5
2.1 硬盘分区配置...................................................................................................................5
2.2 安装配置 JDK....................................................................................................................5
2.2.1 安装 JDK：............................................................................................................5
2.2.2 配置 JDK.................................................................................................................5
2.3 安装配置 Hadoop..............................................................................................................6
2.3.1 安装 Hadoop..........................................................................................................6
2.3.2 配置 Hadoop..........................................................................................................6
2.3.3 配置 master 和 slaves........................................................................................7
2.4 安装配置 SSH....................................................................................................................8
2.4.1 安装 SSH................................................................................................................8
2.4.2 配置 SSH................................................................................................................8
2.5 环境配置检验....................................................................................................................9
3 MapReduce 开发..........................................................................................................................10
3.1 Hadoop Eclipse Plugin 编译......................................................................................10
3.2 Eclipse Plugin 配置....................................................................................................10
3.3 建立 MapReduce 工程.....................................................................................................13
4 Hadoop 性能优化........................................................................................................................15
4.1 配置优化.........................................................................................................................15
4.1.1 内存优化.............................................................................................................15
4.1.2 master 优化........................................................................................................15
4.1.3 文件存储设置.....................................................................................................16
4.1.4 MapReduce 临时存储..........................................................................................16
4.1.5 tasks 配置..........................................................................................................16
4.2 MapReduce 工程优化......................................................................................................17
