没有合适的资源?快使用搜索试试~ 我知道了~
分布式开发基础教程1
资源推荐
资源详情
资源评论
分布式开发基础教程
Document Number: <Feature Identifier>
Document Status: Draft
Document Issue Number: 0.9.0
Issue Date: 2015-12-20
Security Status: Poplar Confidential
Author: 杨立峰(311155@qq.com)
© 2015 POPLAR
All rights reserved
UNCONTROLLED COPY: The master of this document is stored on an electronic database and is “write
protected”; it may be altered only by authorized persons. While copies may be printed, it is not
recommended. Viewing of the master electronically ensures access to the current issue. Any hardcopies
taken must be regarded as uncontrolled copies.
POPLAR CONFIDENTIAL: The information contained in this document is the property of Poplar. Except
as expressly authorized in writing by Poplar, the holder shall keep all information contained herein
confidential, shall disclose the information only to its employees with a need to know, and shall protect the
information from disclosure and dissemination to third parties. Except as expressly authorized in writing by
Poplar, the holder is granted no rights to use the information contained herein. If you have received this
document in error, please notify the sender and destroy it immediately.
Issue 0.9.0 基础教程 <Feature Identifier>
Page 2 of 86 Poplar Confidential Document Status: Draft
分布式开发基础教程 V0.9.0
dddll
Intended left spaces
<Feature Identifier> 基础教程 Issue 0.9.0
Document Status: Draft Poplar Confidential Page 3 of 86
分布式开发基础教程 V0.9.0
dddll
Table of Contents
IDENTIFICATION OF DOCUMENT ..................................................................................................................8
STORAGE LOCATION ..............................................................................................................................................8
LIST OF CONTRIBUTORS..........................................................................................................................................8
APPROVER(S).......................................................................................................................................................8
PUBLICATION HISTORY...........................................................................................................................................9
1 服务器基础环境搭建..............................................................................................................................11
1.1 安装环境..................................................................................................................................................11
1.2 用户配置..................................................................................................................................................11
1.3 SUDO 权限配置 .........................................................................................................................................11
1.4 网络配置..................................................................................................................................................11
1.5 安装配置 SSH...........................................................................................................................................13
1.5.1
配置
Poplar
本机无密码登录
..........................................................................................................13
1.5.2
配置
Poplar
本机无密码登录
POPSlave1~POPSlave3.....................................................................14
1.5.3
配置
POPSlave1~POPSlave3
本机无密码登录
Poplar.....................................................................14
1.6 安装 JDK 工具..........................................................................................................................................15
2 HADOOP 安装 .........................................................................................................................................17
2.1 安装 HADOOP ............................................................................................................................................17
2.2 HADOOP 单机配置.....................................................................................................................................18
2.3 HADOOP 伪分布式配置.............................................................................................................................18
2.4 HADOOP 伪分布式实例-WORDCOUNT ........................................................................................................22
3 HADOOP 集群安装..................................................................................................................................24
3.1 环境..........................................................................................................................................................24
3.2 准备工作..................................................................................................................................................24
3.3 网络配置..................................................................................................................................................24
3.4 SSH 无密码登陆节点...............................................................................................................................24
3.5 配置集群/分布式环境 ............................................................................................................................24
4 ZOOKEEPER 安装配置 .............................................................................................................................28
4.1 ZOOKEEPER 安装.........................................................................................................................................28
4.1.1
下载安装包
......................................................................................................................................28
4.1.2
解压安装包
......................................................................................................................................28
4.1.3
配置环境变量
..................................................................................................................................28
4.2 单机模式..................................................................................................................................................28
4.3 伪集群模式..............................................................................................................................................29
4.4 集群模式..................................................................................................................................................31
5 HBASE 安装.............................................................................................................................................32
Issue 0.9.0 基础教程 <Feature Identifier>
Page 4 of 86 Poplar Confidential Document Status: Draft
分布式开发基础教程 V0.9.0
dddll
5.1 下载安装包..............................................................................................................................................32
5.2 解压安装包..............................................................................................................................................32
5.3 配置安装路径..........................................................................................................................................32
5.4 验证是否安装成功..................................................................................................................................32
6 HBASE 单机模式 .....................................................................................................................................33
6.1 配置 HBASE-ENV.SH .....................................................................................................................................33
6.2 配置 HBASE-SITE.XML...................................................................................................................................33
6.3 启动 HBASE ...............................................................................................................................................33
6.4 进入 SHELL 模式 ........................................................................................................................................33
6.5 停止 HBASE ...............................................................................................................................................34
7 HBASE 伪分布式模式..............................................................................................................................35
7.1 配置 HBASE-ENV.SH .....................................................................................................................................35
7.2 配置 HBASE-SITE.XML...................................................................................................................................35
7.3 启动 HBASE ...............................................................................................................................................35
7.3.1
启动
hadoop
集群
............................................................................................................................35
7.3.2
启动
HBase.......................................................................................................................................36
7.4 进入 SHELL 模式 ........................................................................................................................................36
7.5 查看 HDFS 的 HBASE 数据库文件 ............................................................................................................37
7.6 停止 HBASE ...............................................................................................................................................37
8 HBASE 用户界面 .....................................................................................................................................39
8.1 HBASE 用户界面 .......................................................................................................................................39
8.2 HDFS 主页 ................................................................................................................................................39
8.3 MASTER 页面 .............................................................................................................................................40
8.4 ZOOKEEPER 页面.........................................................................................................................................44
8.5 用户表页面..............................................................................................................................................45
8.6 REGION 服务器页面 ..................................................................................................................................46
9 HBASE 之完全分布式模式安装...............................................................................................................48
9.1 HBASE 集群分布表 ...................................................................................................................................48
9.2 HBASE 集群安装 .......................................................................................................................................48
9.3 配置 HBASE-ENV.SH .....................................................................................................................................48
9.4 配置 HBASE-SITE.XML...................................................................................................................................48
9.5 配置 REGIONSERVERS ...................................................................................................................................49
9.6 分发到其它的机器..................................................................................................................................50
9.7 启动 HBASE ...............................................................................................................................................50
10 HBASE SHELL DDL 操作..........................................................................................................................51
10.1 一般操作................................................................................................................................................51
10.2 DDL 操作 ................................................................................................................................................51
11 HBASE SHELL DML 操作 .........................................................................................................................55
11.1 向表 USER 插入记录 ...............................................................................................................................55
11.2 获取一条记录........................................................................................................................................56
<Feature Identifier> 基础教程 Issue 0.9.0
Document Status: Draft Poplar Confidential Page 5 of 86
分布式开发基础教程 V0.9.0
dddll
11.3 更新一条记录........................................................................................................................................56
11.4 获取指定版本的数据............................................................................................................................57
11.5 全表扫描................................................................................................................................................57
11.6 删除 ID 为”ANDIEGUO”的列为’INFO:AGE’字段..........................................................................................57
11.7 查询表中有多少行................................................................................................................................57
11.8 向 ID 为”ANDIEGUO”添加’INFO:AGE’字段..................................................................................................57
11.9 将表数据清空........................................................................................................................................58
12 HBASE API 访问.....................................................................................................................................59
12.1 HBASE API 介绍 .......................................................................................................................................59
12.1.1
几个相关类与
HBase
数据模型之间的对应关系
........................................................................59
12.1.2 HBaseConfiguration........................................................................................................................59
12.1.3 HBaseAdmin ...................................................................................................................................59
12.1.4 HTableDescriptor ............................................................................................................................60
12.1.5 HColumnDescriptor.........................................................................................................................60
12.1.6 HTable.............................................................................................................................................60
12.1.7 Put ..................................................................................................................................................61
12.1.8 Get ..................................................................................................................................................62
12.2 HBASE API 实战 .......................................................................................................................................62
12.2.1
创建表
............................................................................................................................................62
12.2.2
添加记录
........................................................................................................................................63
12.2.3
获取记录
........................................................................................................................................63
12.2.4
遍历表
............................................................................................................................................64
12.3 部署运行................................................................................................................................................64
12.3.1
启动
Hadoop
集群和
HBase
服务
................................................................................................64
12.3.2
部署源码
........................................................................................................................................65
12.3.3
修改配置文件
................................................................................................................................65
12.3.4
编译文件
........................................................................................................................................66
12.3.5
打包
Jar
文件
.................................................................................................................................66
12.3.6
运行实例
........................................................................................................................................66
13 HBASE 读取 MAPREDUCE 数据写入 HBASE...........................................................................................69
13.1 输入与输出............................................................................................................................................69
13.2 MAPPER 函数实现 ...................................................................................................................................69
13.3 REDUCER 函数实现 ..................................................................................................................................70
13.4 驱动函数实现........................................................................................................................................71
13.5 部署运行................................................................................................................................................72
13.5.1
启动
Hadoop
集群和
HBase
服务
.................................................................................................72
13.5.2
部署源码
........................................................................................................................................72
13.5.3
修改配置文件
................................................................................................................................72
13.5.4
上传输入文件
................................................................................................................................73
13.5.5
编译文件
........................................................................................................................................73
剩余85页未读,继续阅读
资源评论
文润观书
- 粉丝: 22
- 资源: 318
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- 论文(最终)_20240430235101.pdf
- 基于python编写的Keras深度学习框架开发,利用卷积神经网络CNN,快速识别图片并进行分类
- 最全空间计量实证方法(空间杜宾模型和检验以及结果解释文档).txt
- 5uonly.apk
- 蓝桥杯Python组的历年真题
- 2023-04-06-项目笔记 - 第一百十九阶段 - 4.4.2.117全局变量的作用域-117 -2024.04.30
- 2023-04-06-项目笔记 - 第一百十九阶段 - 4.4.2.117全局变量的作用域-117 -2024.04.30
- 前端开发技术实验报告:内含4四实验&实验报告
- Highlight Plus v20.0.1
- 林周瑜-论文.docx
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功