centos7.4下离线安装CDH5.14

所需积分/C币:45 2019-03-22 15:17:22 845KB PDF
116
收藏 收藏
举报

一步一步详细解析centos7.4下离线安装CDH5.14 一步一步详细解析centos7.4下离线安装CDH5.14 一步一步详细解析centos7.4下离线安装CDH5.14
2019/3/22 centos74下离线安CDH514- highyeyeye-博客园 10141.7123 hadoop232颗x1核8GB 10141.7124 hadoop242颗ⅹ1核8GB 10141.7125 hadoop252颗X1核8GB 二)配置服务器 (1)修改主机名 因为虚拟机是通过 vmware的克隆功能拷贝的,4台虚拟机名称相同,我们需要将其修改为规划的名称。使 用" hostnamectl set-hostname主机名"即可修改,例如将hado0p22修改为 hadoop23 hostnamectl set-hostname hadoop23 (2)配置网卡信息 这里我们采用固定P,执行修改配置文件即可,网卡的配置文件在etc/ sysconfig/ network- script/"下面,以 hadoop22 这台主机为例,这里使用的网卡为: iffy-ens33,将其配置信息修改为如下 Trootohadoop22 network-scripts]f vim /etc/sysconfig/network-scripts/ifcfg-ens33 #修改为下面 TYPE=Ethernet NAME=ens 33 UUID=a043a2bb-babb-4864-8e2a-c2c98dbdd8b5 DEVICE=ens33 ONBOOT=yes 工 PADDR=1⊙.141.71.22 NETMASK=255,255,255,日 GATEWAY=10,141.71.254 其它节点一次如上修改。 (3)添加主机节点映射 修改4台主机的"etc/ hosts"文件即可 https://www.cnblogs.com/joneshoway/p/8823640.html 3/16 2019/3/22 centos7.4下离线安CDH514- highyeyeye-博客园 Root@hadoop22-# vim /etc/hosts 127.0.0.1 localhost localhost localdomain localhost localhost localdomain4 1 localhost localhost, localdomain localhost localhost localdomain 6 0.141.71.22 hadoop22 10.141.71.23 hadoop23 10.141.71.24 hadoop24 10.141.71.25 hadoop25 注意,127.0.0.1和:1不能删除。 (4)关闭防火墙 Troot@hadoop22-]# systemctl stop firewall service #plEfirewall [root@ hadoop22-]# systemctl disable firewa1ld. service#禁止 firewa11开机启动,如果不执行,服务器重启之后 firewa11会启动 [root@ hadoop22-]# firewa11-cmd- state#查看防火墙状态 not running (5)关闭 selinux 修改"etc/ selinux/ config"文件,将 SELINUX值改为 disabled [root@hadoop22-]+ vi /etc/selinux/config SELINUX=disabled 6)配置ssh无密码访问 要达到的效果:主节点到其它节点不需要密码就能访问 (61)执行以下命令 [root@ 22 -] ssh-keygen -t rsa 此时会生成ssh文件 [rootghadoop22 -] pwd /root [root@hadoop22 -] ll-a divx 2 root root 80 Apr 615: 48. ssh 里面有2个文件 [root@hadoop22. ssh] ll total 16 rv 1 root root 1679 Apr 6 15: 46 id_rsa rvl-r--r 1 root root 395 Apr 6 15: 46 id_rsa. pub https://www.cnblogs.com/joneshoway/p/8823640.html 4/16 2019/3/22 centos74下离线安CDH514- highyeyeye-博客园 (62)添加公钥到认证文件中 [root@hadoop22. ssh]+ cat -/ssh/id_rsa. pub >>-/ssh/authorized_keys (63)然后把 authorized keys文件Scp到其它节点,在这之前,需要在其它节点先创建sh文件夹 [root@hadoopxx-]# mkdir ssh 在23、24、25节点执行 Trootghadoop22-]# scp -/ ssh/authorized keys root@10.141 71.23: - /ssh/ [root@hadoop22-]# scp -/. ssh/authorized keys root@10. 141.71 24: /. ssh/ Troot@hadoop22-]# scp -/ ssh/authorized keys root@10. 141.71 25:-/ ssh/ 注意,确保ssh文件夹的权限为700, id rsa. pub和 authorized keys权限为64,其他权限都会导致SSH出问题。 (7)时间同步配置 CDH集群的时间必须同步,,否则安装完成会报错:不良:无法找到主机的NTP服务,或该服务未响应时钟偏差请 求。如下图 时钟偏差( hadoop 测试主机时钟与其NTP服务器之间的偏差。 不良无法找到主机的NTP服务,或该服务未响应时钟偏差请求。 (71)NTP规戈 由于我是在局域网内安装,不能访问互联网,故采用 hadoop22的本地时间来做同步。规划如下 NTP属性主机 P地址 NTP server hadoop22 10 141.71.22 hado0p2310.1417123 NTP client hadoop24 10 141.71.24 hadoop2510141.7125 (72)查看是否安装了NTP软件包 [rootghadoop22-]# rpm -ga ntp ntp-4.2.6p5-25,e17. centos.2.X8664 centos74自带了ntp软件包,无需再安装。 https://www.cnblogs.com/joneshoway/p/8823640.html 5/16 2019/3/22 centos74下离线安CDH514- highyeyeye-博客园 (7.3)修改 NTP server 修改"etc/ntp.conf"配文件,如下 driftfile /var/lib/ntp/drift if Permit time synchronization with our time source, but do not permit the source to query or modify the service on this system restrict default ignore#新增行1 restrict10.141.71.0mask255.255.255. notrap nomodify#新增行2:运行71网段的所有IP同步 /var/lib/ntp/ drift#新增行3 fudge127,127,1.0 stratum10#新增行4 #f Permit all access over the loopback interface. This could f be tightened as well, but to do so would effect some of if the administrative functions restrict 127.0,o, 1 restrict : 1 t Hosts on local network are less restricted restrict 192. 168 1.0 mask 255. 255 255.0 nomodify notrap Use public servers from the pool.ntp. org project ipleaseconsiderjoiningthepool(http://www.pool.ntp.org/join.htm1) server127,127,1.日#新增行5: ntp server服务器地址,127.127.1.0是本服务器的IP server18.141.71.22#新增行6: ntp server服务器地址 #broadcast 192.168.1.255 autokey broadcast server #broadcastclient broadcast client (7.4)NTP client 去除原来的 ntp server,添加新的 ntp server Use public servers from the pool.ntp. org project #Pleaseconsiderjoiningthepool(http://www.pool.ntp.org/join.html) server18.141.71.22井.增行1:漆加ntp服务器 (7.5)启动nt服务 在所有节点上执行如下命令启动ntp服务 #/bin/ systemctI restart ntpd. service#启动ntp服务 #/bin/ systemc1enab1 e ntpd. service没置开机启动 查看NTP状态: service ntpd status https://www.cnblogs.com/joneshoway/p/8823640.html 6/16 2019/3/22 centos74下离线安CDH514- highyeyeye-博客园 [rootghadoop22-l# service ntpd status Redirecting to /bin/systemctL status ntpd. service ntpd. service- Network Time Service Loaded Loaded (/usr/Lib/systemd/ system/ntpd. service: enabled: vendor preset: disabled) Active: active (running) since Sat 2018-04-07 16: 19: 44 CST: 4min 455 Process: 26970 Exec Start=/usr/sbin/ntpd -u ntp:ntp soPTIONS (code=exited, status=0/SUCCESS) Main PID: 26971 (ntpd) CGroup: /system. slice/ntpd.service -26971 /usr/sbin/ ntpd -u ntp: ntp-g Apr 07 16: 19: 44 hadoop22 ntpd[26971]: Listen nomally on 3 ens33 10.141.71 22 UDP 123 Apr 07 16: 19: 44 hadoop22 ntpd[26971]: Listen normally on 4 virbre 192.168 122.1 UDP 123 Apr 07 16: 19: 44 hadoop 22 ntpd[26971]: Listen nomally on 5 lo : 1 UDP 123 Apr 07 16: 19: 44 hadoop 22 ntpd[26971]: Listen nomally on 6 ens33 fe80: 20c: 29ff: fega: 9d4b UDP 123 Apr 07 16: 19: 44 hadoop22 ntpd[26971]: Listening on routing socket on fd #23 for interface updates Apre716:19:44 22ntpd[26971]:.0,0c01606 resta Apr 07 16: 19: 44 hadoop22 ntpd [269711: 0.0.0.0 c012 02 freq set kerneL 0.000 PPM Apr 07 16: 19: 44 hadoop 22 ntpd[26971]: 0.0.0.0 cOll 0l freq not set Apr 07 16: 19: 44 hadoop22 systemd[l]: Started Network Time Service. Apr 07 16: 19: 45 hadoop22 ntpdl26971]: 0.0.0.0 c514 04 treq mode (76)查看nt服务时间同步信息 ntdc用于查询ntd守护进程对其当前状态和要求的变化。程序可以运行在交互模式下使用命令行参数或控制。 ntpc np可以看到所使用的时间服务器。 我的nt服务器信息如下: ntpc-np Lrootchadoop 22 -l# ntpd -np renote Local st poll reach de lay offset disp 127,日,D.1 10 643770 000000, 0000000. 03043 =18.141.7 1.141.71.2216 e 0. 000008, e00000.99217 我的nt客户端信息如下: ntpd-np Lrootghadoop23 parcel-cachel# ntpd -np remote Local st poll reach delay offset disp 某10,141,71.221,141.71.2311256377日,00024-0,eee420.13176 (77)进行时间同步 ntpdate-u 10.51.. 12 在 ntp client节点执行 ntpdate hadoop22命令,将所有客户端的时间与服务器同步 rootchadoap25 opt]* ntpdate hadcop22 7 Apr 13: 38: 19 ntpdate[7362]: na server suitable for synchronization found Lrootchadoop 25 opt l* date Sat Apr 73: 46: 49 CsT 2018 Irootchadoop 25 opt]* ntpc -np Intpdc: read: Connection refused [rootghadoop 25 opt]* htpdate hadcop22 7 Apr 13: 55: 02 ntpdate[8102]: adjust time server 10.141.71 22 offset-0 108804 sec 说明:NTP服务启动需要大约五分钟时间,服务启动之前,若客户端同步时间,则会出现错误“ no server suitable for synchronization found (78)配置每天进行一次时间同步 在3台 ntp client.上配置 crontab,执行: Trootghadoop23-]# crontab-e #添加1行 30 12 * xx/usr/sbin/ntpdate 10. 141.71.22>>/tmp/ntdp log https://www.cnblogs.com/joneshoway/p/8823640.html 7/16 2019/3/22 centos74下离线安CDH514- highyeyeye-博客园 该行信息表示,每天12:30执行一次时间同步,并将执行结果输出到/mp/tpd.og"文件中 (8)jdk安装 在所有节点安装jdk,这里现在主节点安装,然后Scp到其它节点 81)查找已安装的 openjdk Root@hadoop22-# rpm -qa grep java javapackages-tools-3.4.1-11. el7 noarch java-1.8.0- openjdk-head1ess-1.8.0.131-11.b12.e17.x8664 tzdata-java-2017b-1.el7noarch python-javapackages-3.4.1-11.el7noarch Java-1,8 θ- openjdk-1.8.0.131-11.b12.e17.x8664 #open jdk java-1.7.0- openjdk-1.7.0.141-2.6.1.5.e17.X8664 #open jdk java-1.7.6- openjdk- headless-1.7.0.141-2.6,16.5,e17.x8664 #open jdk (82)卸载已安装的 openjdk it rpm -e --nodeps java-1.7.0-openjdk-headless-170.141-2.6.10.5.el7X86_64 if rpm -e--nodeps java-1.8.0-open]dk-1.8.0. 131-11 b12 el7X86_64 rpm-e-- nodes java-1.7.0- openjdk-1.7.0.141-2.6.16.5.e17.x8664 83)安装新的j 解压下载好的jk,并重新命名(重新命名只是方便简化路径 [rootghadoop22 opt]# tar -xzvf /root/Desktop/installPackage/jdk-8u161-linux-X64 tar gz [root@hadoop22 opt]# my jdk1.8.0_161 dk1.8 [rootghadoop22 opt]# ls jdk.8 rh (8.4)配置环境变量 rootghadoop22 opt# vim /etc/profile #添加以下: export JAVA_HOME=/opt/jdk18 export CLASSPATH=: JAVA_HOME/lib/dt.jar: SJAVA-_HOME/lib/tools. jar export PAth=PATH: SJAVa HOME/bin (8.5)使生效 [root@hadoop22 opt]# source /etc/profil (8.6)测试 [rootghadoop22 opt]# echo $JAVA_HOME /opt/jdk18 (8.7)复制到其它节点: scp -r /opt/]dk1.8 root@10. 141 71.23: /opt scp -r /opt/jdk.8 root@10.141. 71. 24: /opt https://www.cnblogs.com/joneshoway/p/8823640.html 8/16 2019/3/22 centos74下离线安CDH514- highyeyeye-博客园 scp -r /opt/]dk1.8 root@10. 141. 71. 25: /opt (8.8)修改环境变量,见84 (9)安装mysq数据库 mysq数据库只需要在主节点( hadoop22)安装即可 (91)卸载 mariadb [rootghadoop22 installPackage]# rpm -ga grep mariadb mariadb-1ibs-5.5.56-2.e17.x8664 [rootghadoop22 installPackage]# rpm -e--nodeps mariadb-libs-5556-2el7X86_64 (9.2)解压mysq Lrootghadoop22 opt]# tar -xvf /root/Desktop/instal_Package/mysql-5716-1.el7X86-64.rpm-bundle tar [@hadoop22 opt]# l jdk1.8 mysql-community-libs-5716-1.el7X86_64.rpm mysql-community-client-5.7.16-1.el7X86-64rpm mysql-community-libs-compat-5716 e17,X8664.rpm mysql-community-common-5.7.16-1.el7X86-64rpm mysql-community-minimal-debuginfo-5716 1.e17.X8664.rpm mysql-community-devel-5.7.16-1. el7.X8664 rpm mysql-community-server-57 16 1.e17.x8664.「pm mysql-community-embedded-5.7.16-1.el7x86-64rpm mysql-community-server-minimal-5716 1.e17.X8664.rpm mysql-community-embedded-compat-5.7.16-1.el7.86_64.rpm mysql-community-test-5716 1.e17.x86_64.rpm mysql-community-embedded-devel-5. 7.16-1.017.X86_64.rpm rh (93)只需要按顺序安装4个包即可 [rootohadoop22 opt]# rpm -ivh mysql-community-common-5716-1.el7x8664. rpm [root@hadoop 22 opt]# rpm -ivh mysql-community-libs-5716-1.el7x86-64rpm adoop22 opt]# rpm -ivh mysql-community-client-5716-1.el7X86-64rpm Troot@hadoop22 opt]# rpm -ivh mysql-community-server-5716-1. el7X86-64 rpm 10.4)mysq起停 #查看mysq1是否启动 service mysqld status #启动mysq1 service mysqld start #停止mysq1 service mysqld stop https://www.cnblogs.com/joneshoway/p/8823640.html 9/16 2019/3/22 centos74下离线安CDH514- highyeyeye-博客园 #重启mysq1 service mysqld restart (105)设定开机自启动 [root@hadoop22 opt]# vim /etc/rc. local 添加 service mysqld start (9.6)确认mysq初始密码 [rootghadoop22 log]# more /var/log/mysgld log grep password 2018-04-03T11: 50: 22.6367982 1 [Note a temporary password is generated for root@localhost XMSoLFoa7h= (9.7)登录mysq [root@hadoop22 10g]# mysql-p Enter password: Welcome to the MySQL monitor. Commands end with or g Your MySQL connection id is 2 Server version: 5.7. 16 (9.8)修改密码(大写+小写+特殊字符) mysql> set password password( Mysql_123456); Query OK, o rows affected, 1 warning (0.00 sec) (9.9)使用新密码重新登陆就好了 [root@hadoop22 log]# mysql -uroot - ql_123456 910)创建3个数据库,并授权 create database hive DEFAULT CHaRSet utf8 COLLATE utf8_general_ci create database sentry DEFAULT CHARSET utf8 COLLATE utf8 general_ci create database amon DEFAULT CHARSET utf8 COLLATE utf8 general_ci; grant all privileges on . to 'root '@'hadoop22 identified by 'Mysql_123456 with grant option; prIvileges (三)安装cM (1)在主节点解压 cloudera manager cloudera manager默认安装在"opt"目录下,进入"/opt"目录,解压安装包,生成2个文件夹 cloudera和cm5.7.6 [root@hadoop22-]+cd /opt [root@hadoop22 opt]# tar -xzvf /root/Desktop/installPackage/cloudera-manager-centos7 https://www.cnblogs.com/joneshoway/p/8823640.html 10/16

...展开详情
试读 16P centos7.4下离线安装CDH5.14
立即下载 身份认证后 购VIP低至7折
一个资源只可评论一次,评论内容不能少于5个字
您会向同学/朋友/同事推荐我们的CSDN下载吗?
谢谢参与!您的真实评价是我们改进的动力~
  • 签到新秀

  • 分享王者

关注 私信
上传资源赚钱or赚积分
最新推荐
centos7.4下离线安装CDH5.14 45积分/C币 立即下载
1/16
centos7.4下离线安装CDH5.14第1页
centos7.4下离线安装CDH5.14第2页
centos7.4下离线安装CDH5.14第3页
centos7.4下离线安装CDH5.14第4页

试读结束, 可继续读1页

45积分/C币 立即下载