使用ansible部署CDH 5.15.1大数据集群
使用ansible离线部署CDH 5.15.1大数据集群
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
在此之前，我之前分享过使用shell自定义脚本部署大数据集群，不管是部署CDH集群还是部署HDP集群基本上套路都一样，调优的方式也都大同小异。只是操作界面上可能存在一定的差异。最近赶上公司搞自动化运维项目，我们用到了ansible的一些功能，本篇博客主要记录ansible部署CDH服务。
一.安装ansible
1>.操作环境介绍
注意，请检查你的虚拟机是否可以正常联网，如果不能联网的话，本篇博客估计也就只能成为你的一个参考部署方案了！由于Mac内存有限，我只开了3台虚拟机，使用的操作系统是CentOS 7.2发行版的，内存分配每个节点4G，大家根据自己的机器适当调大或调小内存。详细信息如下：
[root@node101 ~]# hostname
node101.yinzhengjie.org.cn #这是本篇博客所说的主节点，该节点配置必须高于其他节点配置或者配置不能低于2G内存！否则你在后期安装服务时，可能会遇到各种坑！（我测试时，三台主机分配过4，1，1，或者3，3，3基本上都可以正常运行！）
[root@node101 ~]#
[root@node101 ~]# hostname -i
172.30.1.101
[root@node101 ~]#
[root@node101 ~]# cat /etc/redhat-release
CentOS Linux release 7.2. (Core)
[root@node101 ~]#
[root@node101 ~]#
[root@node101 ~]# free -h
total used free shared buff/cache available
Mem: 5.9G 120M 5.5G 6.7M 213M 713M
Swap: .0G 0B .0G
[root@node101 ~]#
[root@node101 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root 50G .8G 43G % /
devtmpfs 480M 480M % /dev
tmpfs 495M 495M % /dev/shm
tmpfs 495M 6.7M 488M % /run
tmpfs 495M 495M % /sys/fs/cgroup
/dev/sda1 477M 133M 315M % /boot
/dev/mapper/VolGroup-lv_home 12G 41M 11G % /home
Home 234G 156G 79G % /media/psf/Home
tmpfs 99M 99M % /run/user/
[root@node101 ~]#
[root@node101 ~]# cat /etc/hosts | grep yinzhengjie
172.30.1.101 node101.yinzhengjie.org.cn
172.30.1.102 node102.yinzhengjie.org.cn
172.30.1.103 node103.yinzhengjie.org.cn
[root@node101 ~]#
[root@node101 ~]#
2>.在主节点（node101.yinzhengjie.org.cn）配置至各个节点的免密钥登录
[root@node101 ~]# ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
Generating public/private rsa key pair.
Created directory '/root/.ssh'.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
7b:9a:3a:cd:0b:7b:b5:5e:da:ad:b8:dd:aa:0a:b1:fb root@node101.yinzhengjie.org.cn
The key's randomart image is:
+--[ RSA ]----+
| |
| |
| |
| |
| . S |
| o .. |
| +o..... |
| .=++.* o |
| +=E+*o=oo |
+-----------------+
[root@node101 ~]#
生成密钥（[root@node101 ~]# ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa）
[root@node101 ~]# ssh-copy-id root@node101.yinzhengjie.org.cn
The authenticity of host 'node101.yinzhengjie.org.cn (172.30.1.101)' can't be established.
ECDSA key fingerprint is :2c::::ba:4a::6d:5b::c4:bd:ab:c2:.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@node101.yinzhengjie.org.cn's password: Number of key(s) added: Now try logging into the machine, with: "ssh 'root@node101.yinzhengjie.org.cn'"
and check to make sure that only the key(s) you wanted were added. [root@node101 ~]#
[root@node101 ~]# who
root pts/ -- : (172.30.1.2)
[root@node101 ~]#
[root@node101 ~]# ssh node101.yinzhengjie.org.cn
Last login: Thu Jan :: from 172.30.1.2
[root@node101 ~]#
[root@node101 ~]# who
root pts/ -- : (172.30.1.2)
root pts/ -- : (node101.yinzhengjie.org.cn)
[root@node101 ~]#
[root@node101 ~]# exit
logout
Connection to node101.yinzhengjie.org.cn closed.
[root@node101 ~]#
[root@node101 ~]# who
root pts/ -- : (172.30.1.2)
[root@node101 ~]#
[root@node101 ~]#
与本机配置免密登录（[root@node101 ~]# ssh-copy-id root@node101.yinzhengjie.org.cn）
[root@node101 ~]# ssh-copy-id root@node102.yinzhengjie.org.cn
The authenticity of host 'node102.yinzhengjie.org.cn (172.30.1.102)' can't be established.
ECDSA key fingerprint is :f4::e1:7a:::::7b:::5d:1b::ec.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@node102.yinzhengjie.org.cn's password: Number of key(s) added: Now try logging into the machine, with: "ssh 'root@node102.yinzhengjie.org.cn'"
and check to make sure that only the key(s) you wanted were added. [root@node101 ~]#
[root@node101 ~]# ssh node102.yinzhengjie.org.cn
Last login: Thu Jan :: from 172.30.1.2
[root@node102 ~]#
[root@node102 ~]# who
root pts/ -- : (172.30.1.2)
root pts/ -- : (172.30.1.101)
[root@node102 ~]#
[root@node102 ~]# exit
logout
Connection to node102.yinzhengjie.org.cn closed.
[root@node101 ~]#
[root@node101 ~]#
与node102.yinzhengjie.org.cn节点配置免密登录（[root@node101 ~]# ssh-copy-id root@node102.yinzhengjie.org.cn）
[root@node101 ~]# ssh-copy-id root@node103.yinzhengjie.org.cn
The authenticity of host 'node103.yinzhengjie.org.cn (172.30.1.103)' can't be established.
ECDSA key fingerprint is :e7:::ce::0d::9d:e5::9f::ab:4c:e7.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@node103.yinzhengjie.org.cn's password: Number of key(s) added: Now try logging into the machine, with: "ssh 'root@node103.yinzhengjie.org.cn'"
and check to make sure that only the key(s) you wanted were added. [root@node101 ~]#
[root@node101 ~]# ssh node103.yinzhengjie.org.cn
Last login: Thu Jan :: from 172.30.1.2
[root@node103 ~]#
[root@node103 ~]# who
root pts/ -- : (172.30.1.2)
root pts/ -- : (172.30.1.101)
[root@node103 ~]#
[root@node103 ~]# exit
logout
Connection to node103.yinzhengjie.org.cn closed.
[root@node101 ~]#
[root@node101 ~]#
与node103.yinzhengjie.org.cn节点配置免密登录（[root@node101 ~]# ssh-copy-id root@node103.yinzhengjie.org.cn）
3>.使用yum工具包安装ansible
安装ansible依赖python2-pyasn1，PyYAML，python-jinja2等模块，使用yum方式安装，他会自动帮我们处理依赖关系，我们配置好yum源即可。
[root@node101 ~]# yum -y install wget
Loaded plugins: fastestmirror
base | 3.6 kB 00:00:00
extras | 3.4 kB 00:00:00
updates | 3.4 kB 00:00:00
updates/7/x86_64/primary_db FAILED
http://mirrors.cqu.edu.cn/CentOS/7.6.1810/updates/x86_64/repodata/1c8b4fadfe3ee3539b33616064daaae4a96b2d659235607042e44c7cc389e0dc-primary.sqlite.bz2: [Errno 14] curl#56 - "Recv failure: Connection reset by peer"
Trying other mirror.
(1/4): extras/7/x86_64/primary_db | 156 kB 00:00:00
(2/4): base/7/x86_64/group_gz | 166 kB 00:00:02
(3/4): updates/7/x86_64/primary_db | 2.4 MB 00:00:03
(4/4): base/7/x86_64/primary_db | 6.0 MB 00:00:08
Determining fastest mirrors
* base: mirrors.tuna.tsinghua.edu.cn
* extras: mirrors.aliyun.com
* updates: mirrors.tuna.tsinghua.edu.cn
Resolving Dependencies
--> Running transaction check
---> Package wget.x86_64 0:1.14-18.el7 will be installed
--> Finished Dependency Resolution Dependencies Resolved ===================================================================================================================================================================
Package Arch Version Repository Size
===================================================================================================================================================================
Installing:
wget x86_64 1.14-18.el7 base 547 k Transaction Summary
===================================================================================================================================================================
Install 1 Package Total download size: 547 k
Installed size: 2.0 M
Downloading packages:
warning: /var/cache/yum/x86_64/7/base/packages/wget-1.14-18.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
Public key for wget-1.14-18.el7.x86_64.rpm is not installed
wget-1.14-18.el7.x86_64.rpm | 547 kB 00:00:00
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Importing GPG key 0xF4A80EB5:
Userid : "CentOS-7 Key (CentOS 7 Official Signing Key) <security@centos.org>"
Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5
Package : centos-release-7-2.1511.el7.centos.2.10.x86_64 (@anaconda)
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : wget-1.14-18.el7.x86_64 1/1
Verifying : wget-1.14-18.el7.x86_64 1/1 Installed:
wget.x86_64 0:1.14-18.el7 Complete!
[root@node101 ~]#
安装下载工具（[root@node101 ~]# yum -y install wget）
[root@node101 ~]# wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
---- ::-- http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
Resolving dl.fedoraproject.org (dl.fedoraproject.org)... 209.132.181.23, 209.132.181.24
Connecting to dl.fedoraproject.org (dl.fedoraproject.org)|209.132.181.23|:... connected.
HTTP request sent, awaiting response... OK
Length: (15K) [application/x-rpm]
Saving to: ‘epel-release-latest-.noarch.rpm’ %[================================================================================================================================>] , .2KB/s in .2s -- :: (78.2 KB/s) - ‘epel-release-latest-.noarch.rpm’ saved [/] [root@node101 ~]#
下载epel源（[root@node101 ~]# wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm）
[root@node101 ~]# ll
total
-rw-r--r--. root root Jan : epel-release-latest-.noarch.rpm
[root@node101 ~]#
[root@node101 ~]# rpm -ivh epel-release-latest-.noarch.rpm
warning: epel-release-latest-.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY
Preparing... ################################# [%]
Updating / installing...
:epel-release-- ################################# [%]
[root@node101 ~]#
[root@node101 ~]# ll /etc/yum.repos.d/ | grep epel
-rw-r--r--. root root Oct epel.repo
-rw-r--r--. root root Oct epel-testing.repo
[root@node101 ~]#
安装epel源（[root@node101 ~]# rpm -ivh epel-release-latest-7.noarch.rpm ）
[root@node101 ~]# cat /etc/yum.conf | grep -v ^# |grep -v ^$
[main]
cachedir=/var/cache/yum/$basearch/$releasever
keepcache=
debuglevel=
logfile=/var/log/yum.log
exactarch=
obsoletes=
gpgcheck=
plugins=
installonly_limit=
bugtracker_url=http://bugs.centos.org/set_project.php?project_id=23&ref=http://bugs.centos.org/bug_report_page.php?category=yum
distroverpkg=centos-release
[root@node101 ~]#
[root@node101 ~]# sed -i 's#keepcache=0#keepcache=1#' /etc/yum.conf
[root@node101 ~]#
[root@node101 ~]# cat /etc/yum.conf | grep -v ^# |grep -v ^$
[main]
cachedir=/var/cache/yum/$basearch/$releasever
keepcache=
debuglevel=
logfile=/var/log/yum.log
exactarch=
obsoletes=
gpgcheck=
plugins=
installonly_limit=
bugtracker_url=http://bugs.centos.org/set_project.php?project_id=23&ref=http://bugs.centos.org/bug_report_page.php?category=yum
distroverpkg=centos-release
[root@node101 ~]#
修改yum的配置文件，使得下载rpm安装包后不删除，方便咱们以后自建内网的yum仓库（[root@node101 ~]# sed -i 's#keepcache=0#keepcache=1#' /etc/yum.conf）
[root@node101 ~]# yum clean all
Loaded plugins: fastestmirror, langpacks
Cleaning repos: epel
Cleaning up everything
[root@node101 ~]#
清空之前的缓存（[root@node101 ~]# yum clean all）
[root@node101 ~]# yum makecache
Loaded plugins: fastestmirror, langpacks
base | 3.6 kB ::
epel/x86_64/metalink | 8.1 kB ::
epel | 3.2 kB ::
extras | 3.4 kB ::
updates | 3.4 kB ::
(/): epel/x86_64/group_gz | kB ::
(/): base//x86_64/other_db | 2.6 MB ::
(/): base//x86_64/group_gz | kB ::
(/): epel/x86_64/prestodelta | B ::
(/): epel/x86_64/updateinfo | kB ::
(/): epel/x86_64/filelists | MB ::
(/): epel/x86_64/other | 2.3 MB ::
(/): base//x86_64/filelists_db | 7.1 MB ::
(/): extras//x86_64/primary_db | kB ::
(/): extras//x86_64/other_db | kB ::
(/): base//x86_64/primary_db | 6.0 MB ::
(/): updates//x86_64/prestodelta | kB ::
(/): extras//x86_64/filelists_db | kB ::
(/): updates//x86_64/filelists_db | 1.4 MB ::
(/): updates//x86_64/primary_db | 1.3 MB ::
extras//x86_64/prestodelta FAILED ========================================================-] kB/s | MB :: ETA
http://mirrors.njupt.edu.cn/centos/7.6.1810/extras/x86_64/repodata/9718255a5fa57675311c5ac73719bee470f94b10d38b5aa988511b9ddd860025-prestodelta.xml.gz: [Errno 12] Timeout on http://10.10.254.10/cache/4/02/mirrors.njupt.edu.cn/4f9b5de52f938c8378fba16c668a0a55/9718255a5fa57675311c5ac73719bee470f94b10d38b5aa988511b9ddd860025-prestodelta.xml.gz: (28, 'Connection timed out after 30884 milliseconds')
Trying other mirror.
(/): extras//x86_64/prestodelta | kB ::
(/): epel/x86_64/primary | 3.6 MB ::
updates//x86_64/other_db FAILED
http://mirrors.njupt.edu.cn/centos/7.6.1810/updates/x86_64/repodata/1fab468ed066d5469823091845179165040deb4951923d854a025e3b5c903051-other.sqlite.bz2: [Errno 12] Timeout on http://10.10.254.10/cache/5/02/mirrors.njupt.edu.cn/08d3e890295133ac7140ae59fc9dee3f/1fab468ed066d5469823091845179165040deb4951923d854a025e3b5c903051-other.sqlite.bz2: (28, 'Connection timed out after 30866 milliseconds')
Trying other mirror.
(/): updates//x86_64/other_db | kB ::
Determining fastest mirrors
* base: mirror.jdcloud.com
* epel: mirrors.yun-idc.com
* extras: mirror.jdcloud.com
* updates: mirrors.tuna.tsinghua.edu.cn
epel /
epel /
epel /
Metadata Cache Created
[root@node101 ~]#
创建yum缓存（[root@node101 ~]# yum makecache）
[root@node101 ~]# yum -y install ansible
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: mirror.jdcloud.com
* epel: mirrors.yun-idc.com
* extras: mirror.jdcloud.com
* updates: mirrors.tuna.tsinghua.edu.cn
Resolving Dependencies
--> Running transaction check
---> Package ansible.noarch :2.7.-.el7 will be installed
--> Processing Dependency: sshpass for package: ansible-2.7.-.el7.noarch
--> Processing Dependency: python2-jmespath for package: ansible-2.7.-.el7.noarch
--> Processing Dependency: python-paramiko for package: ansible-2.7.-.el7.noarch
--> Processing Dependency: python-keyczar for package: ansible-2.7.-.el7.noarch
--> Processing Dependency: python-jinja2 for package: ansible-2.7.-.el7.noarch
--> Processing Dependency: python-httplib2 for package: ansible-2.7.-.el7.noarch
--> Processing Dependency: python-crypto for package: ansible-2.7.-.el7.noarch
--> Processing Dependency: PyYAML for package: ansible-2.7.-.el7.noarch
--> Running transaction check
---> Package PyYAML.x86_64 :3.10-.el7 will be installed
--> Processing Dependency: libyaml-.so.()(64bit) for package: PyYAML-3.10-.el7.x86_64
---> Package python-httplib2.noarch :0.9.-.el7 will be installed
---> Package python-jinja2.noarch :2.7.-.el7 will be installed
--> Processing Dependency: python-babel >= 0.8 for package: python-jinja2-2.7.-.el7.noarch
--> Processing Dependency: python-markupsafe for package: python-jinja2-2.7.-.el7.noarch
---> Package python-keyczar.noarch :.71c-.el7 will be installed
---> Package python-paramiko.noarch :2.1.-.el7 will be installed
--> Processing Dependency: python2-pyasn1 for package: python-paramiko-2.1.-.el7.noarch
---> Package python2-crypto.x86_64 :2.6.-.el7 will be installed
--> Processing Dependency: libtomcrypt.so.()(64bit) for package: python2-crypto-2.6.-.el7.x86_64
---> Package python2-jmespath.noarch :0.9.-.el7 will be installed
---> Package sshpass.x86_64 :1.06-.el7 will be installed
--> Running transaction check
---> Package libtomcrypt.x86_64 :1.17-.el7 will be installed
--> Processing Dependency: libtommath >= 0.42. for package: libtomcrypt-1.17-.el7.x86_64
--> Processing Dependency: libtommath.so.()(64bit) for package: libtomcrypt-1.17-.el7.x86_64
---> Package libyaml.x86_64 :0.1.-.el7_0 will be installed
---> Package python-babel.noarch :0.9.-.el7 will be installed
---> Package python-markupsafe.x86_64 :0.11-.el7 will be installed
---> Package python-pyasn1.noarch :0.1.-.el7 will be obsoleted
---> Package python2-pyasn1.noarch :0.1.-.el7 will be obsoleting
--> Running transaction check
---> Package libtommath.x86_64 :0.42.-.el7 will be installed
--> Finished Dependency Resolution Dependencies Resolved ==========================================================================================================================================================================
Package Arch Version Repository Size
==========================================================================================================================================================================
Installing:
ansible noarch 2.7.-.el7 epel M
python2-pyasn1 noarch 0.1.-.el7 base k
replacing python-pyasn1.noarch 0.1.-.el7
Installing for dependencies:
PyYAML x86_64 3.10-.el7 base k
libtomcrypt x86_64 1.17-.el7 extras k
libtommath x86_64 0.42.-.el7 extras k
libyaml x86_64 0.1.-.el7_0 base k
python-babel noarch 0.9.-.el7 base 1.4 M
python-httplib2 noarch 0.9.-.el7 extras k
python-jinja2 noarch 2.7.-.el7 base k
python-keyczar noarch .71c-.el7 epel k
python-markupsafe x86_64 0.11-.el7 base k
python-paramiko noarch 2.1.-.el7 updates k
python2-crypto x86_64 2.6.-.el7 extras k
python2-jmespath noarch 0.9.-.el7 extras k
sshpass x86_64 1.06-.el7 extras k Transaction Summary
==========================================================================================================================================================================
Install Packages (+ Dependent packages) Total download size: M
Downloading packages:
(/): PyYAML-3.10-.el7.x86_64.rpm | kB ::
(/): libtomcrypt-1.17-.el7.x86_64.rpm | kB ::
(/): python-httplib2-0.9.-.el7.noarch.rpm | kB ::
(/): python-jinja2-2.7.-.el7.noarch.rpm | kB ::
(/): libtommath-0.42.-.el7.x86_64.rpm | kB ::
(/): python-markupsafe-0.11-.el7.x86_64.rpm | kB ::
(/): python-paramiko-2.1.-.el7.noarch.rpm | kB ::
(/): python2-crypto-2.6.-.el7.x86_64.rpm | kB ::
(/): python2-jmespath-0.9.-.el7.noarch.rpm | kB ::
(/): python2-pyasn1-0.1.-.el7.noarch.rpm | kB ::
(/): sshpass-1.06-.el7.x86_64.rpm | kB ::
(/): python-babel-0.9.-.el7.noarch.rpm | 1.4 MB ::
(/): libyaml-0.1.-.el7_0.x86_64.rpm | kB ::
warning: /var/cache/yum/x86_64//epel/packages/python-keyczar-.71c-.el7.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY7 kB/s | 9.4 MB :: ETA
Public key for python-keyczar-.71c-.el7.noarch.rpm is not installed
(/): python-keyczar-.71c-.el7.noarch.rpm | kB ::
(/): ansible-2.7.-.el7.noarch.rpm | MB ::
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total kB/s | MB ::
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
Importing GPG key 0x352C64E5:
Userid : "Fedora EPEL (7) <epel@fedoraproject.org>"
Fingerprint: 91e9 7d7c 4a5e 96f1 7f3e 888f 6a2f aea2 352c 64e5
Package : epel-release--.noarch (installed)
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
Installing : python2-pyasn1-0.1.-.el7.noarch /
Installing : python-paramiko-2.1.-.el7.noarch /
Installing : python-babel-0.9.-.el7.noarch /
Installing : python-httplib2-0.9.-.el7.noarch /
Installing : sshpass-1.06-.el7.x86_64 /
Installing : libtommath-0.42.-.el7.x86_64 /
Installing : libtomcrypt-1.17-.el7.x86_64 /
Installing : python2-crypto-2.6.-.el7.x86_64 /
Installing : python-keyczar-.71c-.el7.noarch /
Installing : python-markupsafe-0.11-.el7.x86_64 /
Installing : python-jinja2-2.7.-.el7.noarch /
Installing : python2-jmespath-0.9.-.el7.noarch /
Installing : libyaml-0.1.-.el7_0.x86_64 /
Installing : PyYAML-3.10-.el7.x86_64 /
Installing : ansible-2.7.-.el7.noarch /
Erasing : python-pyasn1-0.1.-.el7.noarch /
Verifying : python-keyczar-.71c-.el7.noarch /
Verifying : libyaml-0.1.-.el7_0.x86_64 /
Verifying : python-jinja2-2.7.-.el7.noarch /
Verifying : python2-jmespath-0.9.-.el7.noarch /
Verifying : python-paramiko-2.1.-.el7.noarch /
Verifying : python-markupsafe-0.11-.el7.x86_64 /
Verifying : python2-crypto-2.6.-.el7.x86_64 /
Verifying : libtommath-0.42.-.el7.x86_64 /
Verifying : sshpass-1.06-.el7.x86_64 /
Verifying : python-httplib2-0.9.-.el7.noarch /
Verifying : ansible-2.7.-.el7.noarch /
Verifying : PyYAML-3.10-.el7.x86_64 /
Verifying : python-babel-0.9.-.el7.noarch /
Verifying : libtomcrypt-1.17-.el7.x86_64 /
Verifying : python2-pyasn1-0.1.-.el7.noarch /
Verifying : python-pyasn1-0.1.-.el7.noarch / Installed:
ansible.noarch :2.7.-.el7 python2-pyasn1.noarch :0.1.-.el7 Dependency Installed:
PyYAML.x86_64 :3.10-.el7 libtomcrypt.x86_64 :1.17-.el7 libtommath.x86_64 :0.42.-.el7 libyaml.x86_64 :0.1.-.el7_0
python-babel.noarch :0.9.-.el7 python-httplib2.noarch :0.9.-.el7 python-jinja2.noarch :2.7.-.el7 python-keyczar.noarch :.71c-.el7
python-markupsafe.x86_64 :0.11-.el7 python-paramiko.noarch :2.1.-.el7 python2-crypto.x86_64 :2.6.-.el7 python2-jmespath.noarch :0.9.-.el7
sshpass.x86_64 :1.06-.el7 Replaced:
python-pyasn1.noarch :0.1.-.el7 Complete!
[root@node101 ~]#
安装ansible（[root@node101 ~]# yum -y install ansible）
4>.配置ansible的配置文件
我根据我的需求，将主机分为2个角色，大家都知道，hdfs集群又NameNode和DataNode之分，因此我将配置相对较高的那台机器配置为NameNode节点，然后把DataNode服务安置在所有的节点上。
[root@node101 ~]# tail - /etc/ansible/hosts
#Add by yinzhengjie
[namenode]
node101.yinzhengjie.org.cn [datanode]
node101.yinzhengjie.org.cn
node102.yinzhengjie.org.cn
node103.yinzhengjie.org.cn
[root@node101 ~]#
[root@node101 ~]#
5>.安装相关依赖插件，解决cryptography 0.8.2 版本兼容性（如果该步骤不做的话，在你执行“ansible all -m ping”命令时可能会报错！）
5.1>.安装pip
[root@node101 ~]# wget https://files.pythonhosted.org/packages/45/ae/8a0ad77defb7cc903f09e551d88b443304a9bd6e6f124e75c0fbbf6de8f7/pip-18.1.tar.gz
---- ::-- https://files.pythonhosted.org/packages/45/ae/8a0ad77defb7cc903f09e551d88b443304a9bd6e6f124e75c0fbbf6de8f7/pip-18.1.tar.gz
Resolving files.pythonhosted.org (files.pythonhosted.org)... 151.101.109.63, 2a04:4e42:::
Connecting to files.pythonhosted.org (files.pythonhosted.org)|151.101.109.63|:... connected.
HTTP request sent, awaiting response... OK
Length: (1.2M) [binary/octet-stream]
Saving to: ‘pip-18.1.tar.gz’ %[================================================================================================================================>] ,, .7KB/s in 22s -- :: (55.4 KB/s) - ‘pip-18.1.tar.gz’ saved [/] [root@node101 ~]#
下载pip（[root@node101 ~]# wget https://files.pythonhosted.org/packages/45/ae/8a0ad77defb7cc903f09e551d88b443304a9bd6e6f124e75c0fbbf6de8f7/pip-18.1.tar.gz）
[root@node101 ~]# tar -zxf pip-18.1.tar.gz
[root@node101 ~]# ll
total
-rw-r--r--. root root Oct epel-release-latest-.noarch.rpm
drwxr-xr-x. games Jan : pip-18.1
-rw-r--r--. root root Oct : pip-18.1.tar.gz
[root@node101 ~]#
[root@node101 ~]# cd pip-18.1/
[root@node101 pip-18.1]#
[root@node101 pip-18.1]# python setup.py install
.........(内容输出过多，我这里就统一复制了！)
Installing pip2. script to /usr/bin
Installing pip2 script to /usr/bin Installed /usr/lib/python2./site-packages/pip-18.1-py2..egg
Processing dependencies for pip==18.1
Finished processing dependencies for pip==18.1
[root@node101 pip-18.1]#
安装pip工具（[root@node101 pip-18.1]# python setup.py install）
5.2>.安装wheel
[root@node101 ~]# wget https://files.pythonhosted.org/packages/d8/55/221a530d66bf78e72996453d1e2dedef526063546e131d70bed548d80588/wheel-0.32.3.tar.gz
---- ::-- https://files.pythonhosted.org/packages/d8/55/221a530d66bf78e72996453d1e2dedef526063546e131d70bed548d80588/wheel-0.32.3.tar.gz
Resolving files.pythonhosted.org (files.pythonhosted.org)... 151.101.109.63, 2a04:4e42:1a::
Connecting to files.pythonhosted.org (files.pythonhosted.org)|151.101.109.63|:... connected.
HTTP request sent, awaiting response... OK
Length: (18K) [binary/octet-stream]
Saving to: ‘wheel-0.32..tar.gz’ %[================================================================================================================================>] , --.-K/s in .1s -- :: ( KB/s) - ‘wheel-0.32..tar.gz’ saved [/] [root@node101 ~]# ll
total
-rw-r--r--. root root Oct epel-release-latest-.noarch.rpm
drwxr-xr-x. games Jan : pip-18.1
-rw-r--r--. root root Oct : pip-18.1.tar.gz
-rw-r--r--. root root Nov : wheel-0.32..tar.gz
[root@node101 ~]#
[root@node101 ~]#
下载wheel（[root@node101 ~]# wget https://files.pythonhosted.org/packages/d8/55/221a530d66bf78e72996453d1e2dedef526063546e131d70bed548d80588/wheel-0.32.3.tar.gz）
[root@node101 ~]# ll
total
-rw-r--r--. root root Oct epel-release-latest-.noarch.rpm
drwxr-xr-x. games Jan : pip-18.1
-rw-r--r--. root root Oct : pip-18.1.tar.gz
-rw-r--r--. root root Nov : wheel-0.32..tar.gz
[root@node101 ~]#
[root@node101 ~]# tar -zxf wheel-0.32..tar.gz
[root@node101 ~]#
[root@node101 ~]# ll
total
-rw-r--r--. root root Oct epel-release-latest-.noarch.rpm
drwxr-xr-x. games Jan : pip-18.1
-rw-r--r--. root root Oct : pip-18.1.tar.gz
drwxrwxr-x. Nov : wheel-0.32.
-rw-r--r--. root root Nov : wheel-0.32..tar.gz
[root@node101 ~]#
[root@node101 ~]# cd wheel-0.32./
[root@node101 wheel-0.32.]# ll
total
-rw-rw-r--. Nov : LICENSE.txt
-rw-rw-r--. Nov : MANIFEST.in
-rw-rw-r--. Nov : PKG-INFO
-rw-rw-r--. Nov : README.rst
-rw-rw-r--. Nov : setup.cfg
-rw-rw-r--. Nov : setup.py
drwxrwxr-x. Nov : wheel
drwxrwxr-x. Nov : wheel.egg-info
[root@node101 wheel-0.32.]#
[root@node101 wheel-0.32.]# python setup.py install
/usr/lib64/python2./distutils/dist.py:: UserWarning: Unknown distribution option: 'project_urls'
warnings.warn(msg)
/usr/lib64/python2./distutils/dist.py:: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
running install
running bdist_egg
running egg_info
writing requirements to wheel.egg-info/requires.txt
writing wheel.egg-info/PKG-INFO
writing top-level names to wheel.egg-info/top_level.txt
writing dependency_links to wheel.egg-info/dependency_links.txt
writing entry points to wheel.egg-info/entry_points.txt
reading manifest file 'wheel.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'wheel/*.txt'
warning: no files found matching '*.sh'
warning: no files found matching '*.py' under directory 'wheel/test'
warning: no files found matching 'wheel/test/test-1.0-py2.py3-none-win32.whl'
warning: no files found matching 'wheel/test/headers.dist/header.h'
warning: no files found matching 'wheel/test/pydist-schema.json'
no previously-included directories found matching 'wheel/test/*/dist'
no previously-included directories found matching 'wheel/test/*/build'
writing manifest file 'wheel.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/wheel
copying wheel/wheelfile.py -> build/lib/wheel
copying wheel/pep425tags.py -> build/lib/wheel
copying wheel/__init__.py -> build/lib/wheel
copying wheel/util.py -> build/lib/wheel
copying wheel/metadata.py -> build/lib/wheel
copying wheel/__main__.py -> build/lib/wheel
copying wheel/bdist_wheel.py -> build/lib/wheel
copying wheel/pkginfo.py -> build/lib/wheel
creating build/lib/wheel/cli
copying wheel/cli/install.py -> build/lib/wheel/cli
copying wheel/cli/pack.py -> build/lib/wheel/cli
copying wheel/cli/__init__.py -> build/lib/wheel/cli
copying wheel/cli/convert.py -> build/lib/wheel/cli
copying wheel/cli/unpack.py -> build/lib/wheel/cli
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/wheel
copying build/lib/wheel/wheelfile.py -> build/bdist.linux-x86_64/egg/wheel
copying build/lib/wheel/pep425tags.py -> build/bdist.linux-x86_64/egg/wheel
copying build/lib/wheel/__init__.py -> build/bdist.linux-x86_64/egg/wheel
copying build/lib/wheel/util.py -> build/bdist.linux-x86_64/egg/wheel
copying build/lib/wheel/metadata.py -> build/bdist.linux-x86_64/egg/wheel
copying build/lib/wheel/__main__.py -> build/bdist.linux-x86_64/egg/wheel
copying build/lib/wheel/bdist_wheel.py -> build/bdist.linux-x86_64/egg/wheel
copying build/lib/wheel/pkginfo.py -> build/bdist.linux-x86_64/egg/wheel
creating build/bdist.linux-x86_64/egg/wheel/cli
copying build/lib/wheel/cli/install.py -> build/bdist.linux-x86_64/egg/wheel/cli
copying build/lib/wheel/cli/pack.py -> build/bdist.linux-x86_64/egg/wheel/cli
copying build/lib/wheel/cli/__init__.py -> build/bdist.linux-x86_64/egg/wheel/cli
copying build/lib/wheel/cli/convert.py -> build/bdist.linux-x86_64/egg/wheel/cli
copying build/lib/wheel/cli/unpack.py -> build/bdist.linux-x86_64/egg/wheel/cli
byte-compiling build/bdist.linux-x86_64/egg/wheel/wheelfile.py to wheelfile.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/pep425tags.py to pep425tags.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/util.py to util.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/metadata.py to metadata.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/__main__.py to __main__.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/bdist_wheel.py to bdist_wheel.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/pkginfo.py to pkginfo.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/cli/install.py to install.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/cli/pack.py to pack.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/cli/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/cli/convert.py to convert.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/cli/unpack.py to unpack.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying wheel.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying wheel.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying wheel.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying wheel.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying wheel.egg-info/not-zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO
copying wheel.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying wheel.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
creating dist
creating 'dist/wheel-0.32.3-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing wheel-0.32.-py2..egg
creating /usr/lib/python2./site-packages/wheel-0.32.-py2..egg
Extracting wheel-0.32.-py2..egg to /usr/lib/python2./site-packages
Adding wheel 0.32. to easy-install.pth file
Installing wheel script to /usr/bin Installed /usr/lib/python2./site-packages/wheel-0.32.-py2..egg
Processing dependencies for wheel==0.32.
Finished processing dependencies for wheel==0.32.
[root@node101 wheel-0.32.]#
安装wheel工具（[root@node101 wheel-0.32.3]# python setup.py install）
5.3>.安装cryptography
[root@node101 ~]# wget https://files.pythonhosted.org/packages/7f/ba/383b51cc26e3141c689ce988814385c7659f5ba01c4b5f2de38233010b5f/cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl
---- ::-- https://files.pythonhosted.org/packages/7f/ba/383b51cc26e3141c689ce988814385c7659f5ba01c4b5f2de38233010b5f/cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl
Resolving files.pythonhosted.org (files.pythonhosted.org)... 151.101.73.63, 2a04:4e42:::
Connecting to files.pythonhosted.org (files.pythonhosted.org)|151.101.73.63|:... connected.
HTTP request sent, awaiting response... OK
Length: (2.0M) [binary/octet-stream]
Saving to: ‘cryptography-2.4.-cp27-cp27mu-manylinux1_x86_64.whl’ %[================================================================================================================================>] ,, .2KB/s in 24s -- :: (85.8 KB/s) - ‘cryptography-2.4.-cp27-cp27mu-manylinux1_x86_64.whl’ saved [/] [root@node101 ~]#
下载cryptography（[root@node101 ~]# wget https://files.pythonhosted.org/packages/7f/ba/383b51cc26e3141c689ce988814385c7659f5ba01c4b5f2de38233010b5f/cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl）
[root@node101 ~]# pip install cryptography-2.4.-cp27-cp27mu-manylinux1_x86_64.whl
Processing ./cryptography-2.4.-cp27-cp27mu-manylinux1_x86_64.whl
Collecting idna>=2.1 (from cryptography==2.4.)
Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)
% |████████████████████████████████| 61kB 248kB/s
Requirement already satisfied: enum34; python_version < "" in /usr/lib/python2./site-packages (from cryptography==2.4.) (1.0.)
Collecting cffi!=1.11.,>=1.7 (from cryptography==2.4.)
Downloading https://files.pythonhosted.org/packages/14/dd/3e7a1e1280e7d767bd3fa15791759c91ec19058ebe31217fe66f3e9a8c49/cffi-1.11.5-cp27-cp27mu-manylinux1_x86_64.whl (407kB)
% |████████████████████████████████| 409kB 147kB/s
Requirement already satisfied: six>=1.4. in /usr/lib/python2./site-packages (from cryptography==2.4.) (1.9.)
Collecting ipaddress; python_version < "" (from cryptography==2.4.)
Downloading https://files.pythonhosted.org/packages/fc/d0/7fc3a811e011d4b388be48a0e381db8d990042df54aa4ef4599a31d39853/ipaddress-1.0.22-py2.py3-none-any.whl
Collecting asn1crypto>=0.21. (from cryptography==2.4.)
Downloading https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl (101kB)
% |████████████████████████████████| 102kB 292kB/s
Requirement already satisfied: pycparser in /usr/lib/python2./site-packages (from cffi!=1.11.,>=1.7->cryptography==2.4.) (2.14)
Installing collected packages: idna, cffi, ipaddress, asn1crypto, cryptography
Found existing installation: cffi 0.8.
Uninstalling cffi-0.8.:
Successfully uninstalled cffi-0.8.
Found existing installation: cryptography 0.8.
Uninstalling cryptography-0.8.:
Successfully uninstalled cryptography-0.8.
Successfully installed asn1crypto-0.24. cffi-1.11. cryptography-2.4. idna-2.8 ipaddress-1.0.
[root@node101 ~]#
安装cryptography（[root@node101 ~]# pip install cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl ）
6>.使用ansible同步文件到其他节点
[root@node101 ~]# ansible all -m ping
node102.yinzhengjie.org.cn | SUCCESS => {
"changed": false,
"ping": "pong"
}
node103.yinzhengjie.org.cn | SUCCESS => {
"changed": false,
"ping": "pong"
}
node101.yinzhengjie.org.cn | SUCCESS => {
"changed": false,
"ping": "pong"
}
[root@node101 ~]#
检查所有节点是否在线（[root@node101 ~]# ansible all -m ping ）
[root@node101 ~]# ansible datanode -m copy -a 'src=/etc/hosts dest=/etc/hosts'
node101.yinzhengjie.org.cn | SUCCESS => {
"changed": false,
"checksum": "6b35ffc3c04d3ec2d3464b74ea0f78c20f27131a",
"dest": "/etc/hosts",
"gid": ,
"group": "root",
"mode": "",
"owner": "root",
"path": "/etc/hosts",
"secontext": "system_u:object_r:net_conf_t:s0",
"size": ,
"state": "file",
"uid":
}
node102.yinzhengjie.org.cn | CHANGED => {
"changed": true,
"checksum": "6b35ffc3c04d3ec2d3464b74ea0f78c20f27131a",
"dest": "/etc/hosts",
"gid": ,
"group": "root",
"md5sum": "494c828fc321572368ec771a390f90c1",
"mode": "",
"owner": "root",
"secontext": "system_u:object_r:net_conf_t:s0",
"size": ,
"src": "/root/.ansible/tmp/ansible-tmp-1547116266.5-263369209847793/source",
"state": "file",
"uid":
}
node103.yinzhengjie.org.cn | CHANGED => {
"changed": true,
"checksum": "6b35ffc3c04d3ec2d3464b74ea0f78c20f27131a",
"dest": "/etc/hosts",
"gid": ,
"group": "root",
"md5sum": "494c828fc321572368ec771a390f90c1",
"mode": "",
"owner": "root",
"secontext": "system_u:object_r:net_conf_t:s0",
"size": ,
"src": "/root/.ansible/tmp/ansible-tmp-1547116266.52-44859007354579/source",
"state": "file",
"uid":
}
[root@node101 ~]#
[root@node101 ~]# ansible all -m shell -a 'cat /etc/hosts|grep yinzhengjie'
node103.yinzhengjie.org.cn | CHANGED | rc= >>
172.30.1.101 node101.yinzhengjie.org.cn
172.30.1.102 node102.yinzhengjie.org.cn
172.30.1.103 node103.yinzhengjie.org.cn node101.yinzhengjie.org.cn | CHANGED | rc= >>
172.30.1.101 node101.yinzhengjie.org.cn
172.30.1.102 node102.yinzhengjie.org.cn
172.30.1.103 node103.yinzhengjie.org.cn node102.yinzhengjie.org.cn | CHANGED | rc= >>
172.30.1.101 node101.yinzhengjie.org.cn
172.30.1.102 node102.yinzhengjie.org.cn
172.30.1.103 node103.yinzhengjie.org.cn [root@node101 ~]#
将namenode节点的数据同步到其他节点中（[root@node101 ~]# ansible datanode -m copy -a 'src=/etc/hosts dest=/etc/hosts'）
二.安装jdk环境
1>.卸载jdk1.7版本
[root@node101 ~]# ansible all -m shell -a 'rpm -qa | grep java'
[WARNING]: Consider using the yum, dnf or zypper module rather than running rpm. If you need to use command because yum, dnf or zypper is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node102.yinzhengjie.org.cn | CHANGED | rc= >>
javapackages-tools-3.4.-.el7.noarch
tzdata-java-2015g-.el7.noarch
java-1.7.-openjdk-headless-1.7.0.91-2.6.2.3.el7.x86_64
python-javapackages-3.4.-.el7.noarch
java-1.7.-openjdk-1.7.0.91-2.6.2.3.el7.x86_64 node103.yinzhengjie.org.cn | CHANGED | rc= >>
javapackages-tools-3.4.-.el7.noarch
tzdata-java-2015g-.el7.noarch
java-1.7.-openjdk-headless-1.7.0.91-2.6.2.3.el7.x86_64
python-javapackages-3.4.-.el7.noarch
java-1.7.-openjdk-1.7.0.91-2.6.2.3.el7.x86_64 node101.yinzhengjie.org.cn | CHANGED | rc= >>
javapackages-tools-3.4.-.el7.noarch
tzdata-java-2015g-.el7.noarch
java-1.7.-openjdk-headless-1.7.0.91-2.6.2.3.el7.x86_64
python-javapackages-3.4.-.el7.noarch
java-1.7.-openjdk-1.7.0.91-2.6.2.3.el7.x86_64 [root@node101 ~]#
使用ansible检查各个节点是否安装jdk（[root@node101 ~]# ansible all -m shell -a 'rpm -qa | grep java'）
[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps javapackages-tools-3.4.1-11.el7.noarch'
[WARNING]: Consider using the yum, dnf or zypper module rather than running rpm. If you need to use command because yum, dnf or zypper is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> node103.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 ~]#
卸载javapackages软件包（ [root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps javapackages-tools-3.4.1-11.el7.noarch'）
[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps tzdata-java-2015g-1.el7.noarch'
[WARNING]: Consider using the yum, dnf or zypper module rather than running rpm. If you need to use command because yum, dnf or zypper is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node103.yinzhengjie.org.cn | CHANGED | rc= >> node101.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 ~]#
卸载tzdata软件包（[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps tzdata-java-2015g-1.el7.noarch' ）
[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps java-1.7.0-openjdk-headless-1.7.0.91-2.6.2.3.el7.x86_64'
[WARNING]: Consider using the yum, dnf or zypper module rather than running rpm. If you need to use command because yum, dnf or zypper is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> node103.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 ~]#
卸载openjdk软件包（[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps java-1.7.0-openjdk-headless-1.7.0.91-2.6.2.3.el7.x86_64'）
[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps python-javapackages-3.4.1-11.el7.noarch'
[WARNING]: Consider using the yum, dnf or zypper module rather than running rpm. If you need to use command because yum, dnf or zypper is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> node103.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 ~]#
[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps python-javapackages-3.4.1-11.el7.noarch'
[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps java-1.7.0-openjdk-1.7.0.91-2.6.2.3.el7.x86_64'
[WARNING]: Consider using the yum, dnf or zypper module rather than running rpm. If you need to use command because yum, dnf or zypper is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> node103.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 ~]#
[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps java-1.7.0-openjdk-1.7.0.91-2.6.2.3.el7.x86_64'
[root@node101 ~]# ansible all -m shell -a 'rpm -qa | grep java'
[WARNING]: Consider using the yum, dnf or zypper module rather than running rpm. If you need to use command because yum, dnf or zypper is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node103.yinzhengjie.org.cn | FAILED | rc= >>
non-zero return code node102.yinzhengjie.org.cn | FAILED | rc= >>
non-zero return code node101.yinzhengjie.org.cn | FAILED | rc= >>
non-zero return code [root@node101 ~]#
检查jdk1.7相关软件包是否卸载完成（[root@node101 ~]# ansible all -m shell -a 'rpm -qa | grep java' ）
2>.安装jdk1.8版本几以上版本
大家可以自行去Oracle官网下载相应的jdk，我这里给出下载地址：https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html（推荐下载tar.gz的安装包）。
[root@node101 ~]# ansible all -m shell -a 'mkdir -pv /yinzhengjie/softwares'
[WARNING]: Consider using the file module with state=directory rather than running mkdir. If you need to use command because file is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node103.yinzhengjie.org.cn | CHANGED | rc= >>
mkdir: created directory ‘/yinzhengjie’
mkdir: created directory ‘/yinzhengjie/softwares’ node101.yinzhengjie.org.cn | CHANGED | rc= >>
mkdir: created directory ‘/yinzhengjie’
mkdir: created directory ‘/yinzhengjie/softwares’ node102.yinzhengjie.org.cn | CHANGED | rc= >>
mkdir: created directory ‘/yinzhengjie’
mkdir: created directory ‘/yinzhengjie/softwares’ [root@node101 ~]#
创建指定的软件存放目录，这个目录大家可以自定义创建目录（[root@node101 ~]# ansible all -m shell -a 'mkdir -pv /yinzhengjie/softwares'）
[root@node101 ~]# ansible namenode -m shell -a 'tar -zxf ~/jdk-8u131-linux-x64.tar.gz -C /yinzhengjie/softwares'
[WARNING]: Consider using the unarchive module rather than running tar. If you need to use command because unarchive is insufficient you can add warn=False to this
command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 ~]# ls /yinzhengjie/softwares/
jdk1..0_131
[root@node101 ~]#
[root@node101 ~]# ll /yinzhengjie/softwares/jdk1..0_131/
total
drwxr-xr-x. Mar bin
-r--r--r--. Mar COPYRIGHT
drwxr-xr-x. Mar db
drwxr-xr-x. Mar include
-rwxr-xr-x. Mar javafx-src.zip
drwxr-xr-x. Mar jre
drwxr-xr-x. Mar lib
-r--r--r--. Mar LICENSE
drwxr-xr-x. Mar man
-r--r--r--. Mar README.html
-rw-r--r--. Mar release
-rw-r--r--. Mar src.zip
-rwxr-xr-x. Mar THIRDPARTYLICENSEREADME-JAVAFX.txt
-r--r--r--. Mar THIRDPARTYLICENSEREADME.txt
[root@node101 ~]#
解压jdk软件（[root@node101 ~]# ansible namenode -m shell -a 'tar -zxf ~/jdk-8u131-linux-x64.tar.gz -C /yinzhengjie/softwares'）
[root@node101 ~]# ansible datanode -m copy -a 'src=/yinzhengjie/softwares/jdk1.8.0_131 dest=/yinzhengjie/softwares'
node101.yinzhengjie.org.cn | CHANGED => {
"changed": true,
"dest": "/yinzhengjie/softwares/",
"src": "/yinzhengjie/softwares/jdk1.8.0_131"
}
node103.yinzhengjie.org.cn | CHANGED => {
"changed": true,
"dest": "/yinzhengjie/softwares/",
"src": "/yinzhengjie/softwares/jdk1.8.0_131"
}
node102.yinzhengjie.org.cn | CHANGED => {
"changed": true,
"dest": "/yinzhengjie/softwares/",
"src": "/yinzhengjie/softwares/jdk1.8.0_131"
}
[root@node101 ~]#
将主节点的java目录同步到其他节点（[root@node101 ~]# ansible datanode -m copy -a 'src=/yinzhengjie/softwares/jdk1.8.0_131 dest=/yinzhengjie/softwares'）
[root@node101 ~]# ansible datanode -m shell -a 'ln -s /yinzhengjie/softwares/jdk1.8.0_131 /yinzhengjie/softwares/jdk'
[WARNING]: Consider using the file module with state=link rather than running ln. If you need to use command because file is insufficient you can add warn=False to
this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc= >> node103.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 ~]#
创建符号链接，即软连接（[root@node101 ~]# ansible datanode -m shell -a 'ln -s /yinzhengjie/softwares/jdk1.8.0_131 /yinzhengjie/softwares/jdk' ）
3>.配置环境java的系统环境变量
[root@node101 ~]# tail - /etc/profile
#ADD JAVA_PATH
JAVA_HOME=/yinzhengjie/softwares/jdk
PATH=$PATH:$JAVA_HOME/bin
[root@node101 ~]#
[root@node101 ~]# ansible datanode -m copy -a 'src=/etc/profile dest=/etc/profile'
node103.yinzhengjie.org.cn | CHANGED => {
"changed": true,
"checksum": "5b3a42ee16d78583b834dc0dea67d282b5fa5323",
"dest": "/etc/profile",
"gid": ,
"group": "root",
"md5sum": "cd04b3ecdde83a7c250b923167dc0c01",
"mode": "",
"owner": "root",
"secontext": "system_u:object_r:etc_t:s0",
"size": ,
"src": "/root/.ansible/tmp/ansible-tmp-1547122009.37-153584791304684/source",
"state": "file",
"uid":
}
node102.yinzhengjie.org.cn | CHANGED => {
"changed": true,
"checksum": "5b3a42ee16d78583b834dc0dea67d282b5fa5323",
"dest": "/etc/profile",
"gid": ,
"group": "root",
"md5sum": "cd04b3ecdde83a7c250b923167dc0c01",
"mode": "",
"owner": "root",
"secontext": "system_u:object_r:etc_t:s0",
"size": ,
"src": "/root/.ansible/tmp/ansible-tmp-1547122009.34-130374655280496/source",
"state": "file",
"uid":
}
node101.yinzhengjie.org.cn | SUCCESS => {
"changed": false,
"checksum": "5b3a42ee16d78583b834dc0dea67d282b5fa5323",
"dest": "/etc/profile",
"gid": ,
"group": "root",
"mode": "",
"owner": "root",
"path": "/etc/profile",
"secontext": "system_u:object_r:etc_t:s0",
"size": ,
"state": "file",
"uid":
}
[root@node101 ~]#
[root@node101 ~]# ansible datanode -m copy -a 'src=/etc/profile dest=/etc/profile'
[root@node101 ~]# ansible datanode -m shell -a 'chmod -R 777 /yinzhengjie/softwares/jdk/bin'
[WARNING]: Consider using the file module with mode rather than running chmod. If you need to use command because file is insufficient you can add warn=False to this
command task or set command_warnings=False in ansible.cfg to get rid of this message. node103.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> node101.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 ~]#
取保拷贝到其他节点到java脚本拥有相应的执行权限（[root@node101 ~]# ansible datanode -m shell -a 'chmod -R 777 /yinzhengjie/softwares/jdk/bin'）
[root@node101 ~]# ansible datanode -m shell -a 'source /etc/profile;java -version'
node101.yinzhengjie.org.cn | CHANGED | rc= >>
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) -Bit Server VM (build 25.131-b11, mixed mode) node103.yinzhengjie.org.cn | CHANGED | rc= >>
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) -Bit Server VM (build 25.131-b11, mixed mode) node102.yinzhengjie.org.cn | CHANGED | rc= >>
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) -Bit Server VM (build 25.131-b11, mixed mode) [root@node101 ~]#
查看每个节点的环境是否可用（[root@node101 ~]# ansible datanode -m shell -a 'source /etc/profile;java -version' ）
[root@node101 ~]# ansible datanode -m shell -a 'mkdir /usr/java'
[WARNING]: Consider using the file module with state=directory rather than running mkdir. If you need to use command because file is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc= >> node103.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 ~]#
[root@node101 ~]# ansible datanode -m shell -a 'mkdir /usr/java'
[root@node101 ~]# ansible datanode -m shell -a 'ln -s /yinzhengjie/softwares/jdk1.8.0_131/ /usr/java/jdk1.8'
[WARNING]: Consider using the file module with state=link rather than running ln. If you need to use command because file is insufficient you can add warn=False to
this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc= >> node103.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 ~]#
[root@node101 ~]# ansible datanode -m shell -a 'ln -s /yinzhengjie/softwares/jdk1.8.0_131/ /usr/java/jdk1.8'
[root@node101 ~]# ansible datanode -m shell -a 'ln -s /yinzhengjie/softwares/jdk1.8.0_131/ /usr/java/defalut'
[WARNING]: Consider using the file module with state=link rather than running ln. If you need to use command because file is insufficient you can add warn=False to
this command task or set command_warnings=False in ansible.cfg to get rid of this message. node103.yinzhengjie.org.cn | CHANGED | rc= >> node101.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 ~]#
[root@node101 ~]# ansible datanode -m shell -a 'ln -s /yinzhengjie/softwares/jdk1.8.0_131/ /usr/java/defalut'
[root@node101 ~]# ansible datanode -m shell -a 'ls -l /usr/java'
node101.yinzhengjie.org.cn | CHANGED | rc= >>
total
lrwxrwxrwx. root root Jan : defalut -> /yinzhengjie/softwares/jdk1..0_131/
lrwxrwxrwx. root root Jan : jdk1. -> /yinzhengjie/softwares/jdk1..0_131/ node103.yinzhengjie.org.cn | CHANGED | rc= >>
total
lrwxrwxrwx. root root Jan : defalut -> /yinzhengjie/softwares/jdk1..0_131/
lrwxrwxrwx. root root Jan : jdk1. -> /yinzhengjie/softwares/jdk1..0_131/ node102.yinzhengjie.org.cn | CHANGED | rc= >>
total
lrwxrwxrwx. root root Jan : defalut -> /yinzhengjie/softwares/jdk1..0_131/
lrwxrwxrwx. root root Jan : jdk1. -> /yinzhengjie/softwares/jdk1..0_131/ [root@node101 ~]#
[root@node101 ~]# ansible datanode -m shell -a 'ls -l /usr/java'
三.安装mysql
1>.下载mysql的yum源
[root@node101 ~]# wget http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
---- ::-- http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
Resolving dev.mysql.com (dev.mysql.com)... 137.254.60.11
Connecting to dev.mysql.com (dev.mysql.com)|137.254.60.11|:... connected.
HTTP request sent, awaiting response... Moved Permanently
Location: https://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm [following]
---- ::-- https://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
Connecting to dev.mysql.com (dev.mysql.com)|137.254.60.11|:... connected.
HTTP request sent, awaiting response... Found
Location: https://repo.mysql.com//mysql-community-release-el7-5.noarch.rpm [following]
---- ::-- https://repo.mysql.com//mysql-community-release-el7-5.noarch.rpm
Resolving repo.mysql.com (repo.mysql.com)... 104.76.8.253
Connecting to repo.mysql.com (repo.mysql.com)|104.76.8.253|:... connected.
HTTP request sent, awaiting response... OK
Length: (.0K) [application/x-redhat-package-manager]
Saving to: ‘mysql-community-release-el7-.noarch.rpm’ %[================================================================================================================================>] , --.-K/s in 0s -- :: ( MB/s) - ‘mysql-community-release-el7-.noarch.rpm’ saved [/] [root@node101 ~]#
[root@node101 ~]# ll
total
-rw-r--r--. root root Nov mysql-community-release-el7-.noarch.rpm
[root@node101 ~]#
[root@node101 ~]# wget http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
2>.安装mysql的yum源
[root@node101 ~]# ll
total
-rw-r--r--. root root Nov mysql-community-release-el7-.noarch.rpm
[root@node101 ~]#
[root@node101 ~]# rpm -ivh mysql-community-release-el7-.noarch.rpm
Preparing... ################################# [%]
Updating / installing...
:mysql-community-release-el7- ################################# [%]
[root@node101 ~]#
[root@node101 ~]# ll /etc/yum.repos.d/ | grep mysql
-rw-r--r--. root root Jan mysql-community.repo
-rw-r--r--. root root Jan mysql-community-source.repo
[root@node101 ~]#
[root@node101 ~]# rpm -ivh mysql-community-release-el7-5.noarch.rpm
3>.安装mysql服务
[root@node101 ~]# yum -y install mysql-community-server
Loaded plugins: fastestmirror, langpacks
mysql-connectors-community | 2.5 kB ::
mysql-tools-community | 2.5 kB ::
mysql56-community | 2.5 kB ::
(/): mysql-tools-community/x86_64/primary_db | kB ::
(/): mysql-connectors-community/x86_64/primary_db | kB ::
(/): mysql56-community/x86_64/primary_db | kB ::
Loading mirror speeds from cached hostfile
* base: mirror.jdcloud.com
* epel: mirrors.yun-idc.com
* extras: mirror.jdcloud.com
* updates: mirrors.tuna.tsinghua.edu.cn
Resolving Dependencies
--> Running transaction check
---> Package mysql-community-server.x86_64 :5.6.-.el7 will be installed
--> Processing Dependency: mysql-community-common(x86-) = 5.6.-.el7 for package: mysql-community-server-5.6.-.el7.x86_64
--> Processing Dependency: mysql-community-client(x86-) >= 5.6. for package: mysql-community-server-5.6.-.el7.x86_64
--> Running transaction check
---> Package mysql-community-client.x86_64 :5.6.-.el7 will be installed
--> Processing Dependency: mysql-community-libs(x86-) >= 5.6. for package: mysql-community-client-5.6.-.el7.x86_64
---> Package mysql-community-common.x86_64 :5.6.-.el7 will be installed
--> Running transaction check
---> Package mariadb-libs.x86_64 :5.5.-.el7.centos will be obsoleted
---> Package mysql-community-libs.x86_64 :5.6.-.el7 will be obsoleting
--> Finished Dependency Resolution Dependencies Resolved ==========================================================================================================================================================================
Package Arch Version Repository Size
==========================================================================================================================================================================
Installing:
mysql-community-libs x86_64 5.6.-.el7 mysql56-community 2.0 M
replacing mariadb-libs.x86_64 :5.5.-.el7.centos
mysql-community-server x86_64 5.6.-.el7 mysql56-community M
Installing for dependencies:
mysql-community-client x86_64 5.6.-.el7 mysql56-community M
mysql-community-common x86_64 5.6.-.el7 mysql56-community k Transaction Summary
==========================================================================================================================================================================
Install Packages (+ Dependent packages) Total download size: M
Downloading packages:
warning: /var/cache/yum/x86_64//mysql56-community/packages/mysql-community-common-5.6.-.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY02: ETA
Public key for mysql-community-common-5.6.-.el7.x86_64.rpm is not installed
(/): mysql-community-common-5.6.-.el7.x86_64.rpm | kB ::
(/): mysql-community-libs-5.6.-.el7.x86_64.rpm | 2.0 MB ::
(/): mysql-community-client-5.6.-.el7.x86_64.rpm | MB ::
(/): mysql-community-server-5.6.-.el7.x86_64.rpm | MB ::
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total kB/s | MB ::
Retrieving key from file:/etc/pki/rpm-gpg/RPM-GPG-KEY-mysql
Importing GPG key 0x5072E1F5:
Userid : "MySQL Release Engineering <mysql-build@oss.oracle.com>"
Fingerprint: a4a9 76fc bd3c 70c8 8c71 8d3b e1f5
Package : mysql-community-release-el7-.noarch (installed)
From : file:/etc/pki/rpm-gpg/RPM-GPG-KEY-mysql
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
** Found pre-existing rpmdb problem(s), 'yum check' output follows:
:libreoffice-core-4.3.7.2-.el7.x86_64 has missing requires of java-headless >= ('', '1.6', None)
:libreoffice-ure-4.3.7.2-.el7.x86_64 has missing requires of libjvm.so()(64bit)
Installing : mysql-community-common-5.6.-.el7.x86_64 /
Installing : mysql-community-libs-5.6.-.el7.x86_64 /
Installing : mysql-community-client-5.6.-.el7.x86_64 /
Installing : mysql-community-server-5.6.-.el7.x86_64 /
Erasing : :mariadb-libs-5.5.-.el7.centos.x86_64 /
Verifying : mysql-community-libs-5.6.-.el7.x86_64 /
Verifying : mysql-community-common-5.6.-.el7.x86_64 /
Verifying : mysql-community-client-5.6.-.el7.x86_64 /
Verifying : mysql-community-server-5.6.-.el7.x86_64 /
Verifying : :mariadb-libs-5.5.-.el7.centos.x86_64 / Installed:
mysql-community-libs.x86_64 :5.6.-.el7 mysql-community-server.x86_64 :5.6.-.el7 Dependency Installed:
mysql-community-client.x86_64 :5.6.-.el7 mysql-community-common.x86_64 :5.6.-.el7 Replaced:
mariadb-libs.x86_64 :5.5.-.el7.centos Complete!
[root@node101 ~]#
[root@node101 ~]# yum -y install mysql-community-server
4>.启动mysql服务
[root@node101 ~]# systemctl enable mysqld
[root@node101 ~]#
[root@node101 ~]# systemctl start mysqld
[root@node101 ~]#
[root@node101 ~]#
[root@node101 ~]# systemctl status mysqld
● mysqld.service - MySQL Community Server
Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
Active: active (running) since Thu -- :: HKT; 8s ago
Process: ExecStartPost=/usr/bin/mysql-systemd-start post (code=exited, status=/SUCCESS)
Process: ExecStartPre=/usr/bin/mysql-systemd-start pre (code=exited, status=/SUCCESS)
Main PID: (mysqld_safe)
CGroup: /system.slice/mysqld.service
├─ /bin/sh /usr/bin/mysqld_safe --basedir=/usr
└─ /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --log-error=/var/log/mysqld.log --pid-file=/var/run/my... Jan :: node101.yinzhengjie.org.cn mysql-systemd-start[]: PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
Jan :: node101.yinzhengjie.org.cn mysql-systemd-start[]: To do so, start the server, then issue the following commands:
Jan :: node101.yinzhengjie.org.cn mysql-systemd-start[]: /usr/bin/mysqladmin -u root password 'new-password'
Jan :: node101.yinzhengjie.org.cn mysql-systemd-start[]: /usr/bin/mysqladmin -u root -h node101.yinzhengjie.org.cn password 'new-password'
Jan :: node101.yinzhengjie.org.cn mysql-systemd-start[]: Alternatively you can run:
Jan :: node101.yinzhengjie.org.cn mysql-systemd-start[]: /usr/bin/mysql_secure_installation
Jan :: node101.yinzhengjie.org.cn mysql-systemd-start[]: which will also give you the option of removing the test
Jan :: node101.yinzhengjie.org.cn mysqld_safe[]: :: mysqld_safe Logging to '/var/log/mysqld.log'.
Jan :: node101.yinzhengjie.org.cn mysqld_safe[]: :: mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
Jan :: node101.yinzhengjie.org.cn systemd[]: Started MySQL Community Server.
[root@node101 ~]#
[root@node101 ~]#
[root@node101 ~]# systemctl start mysqld
5>.为mysql配置管理用户
[root@node101 ~]# mysql -uroot #安装后的操作系统，默认是密钥root密码的，我们直接可以用root用户登陆即可。
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is
Server version: 5.6. MySQL Community Server (GPL) Copyright (c) , , Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> set password for 'root'@'localhost' = password('yinzhengjie'); #修改root本地用户的登陆密码为“yinzhengjie”
Query OK, rows affected (0.00 sec) mysql> grant all privileges on *.* to 'root'@'172.30.1.10%' identified by 'yinzhengjie'; #我这里值允许172.30.1.100～172.30.1.109这10个服务器IP使用root用户登陆我这台mysql。其他的机器不允许。
Query OK, rows affected (0.00 sec) mysql> FLUSH PRIVILEGES; #别忘记刷新权限哟！
Query OK, rows affected (0.00 sec) mysql> quit
Bye
[root@node101 ~]#
6>.配置一个cdh超级用户权限（这个步骤如果你调过了，在初始化数据库时可能与报错哟！）
[root@node101 mysql]# systemctl start mysqld
[root@node101 mysql]#
[root@node101 mysql]# systemctl enable mysqld
Created symlink from /etc/systemd/system/mysql.service to /usr/lib/systemd/system/mysqld.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/mysqld.service to /usr/lib/systemd/system/mysqld.service.
[root@node101 mysql]#
[root@node101 mysql]# mysql
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is
Server version: 5.6. MySQL Community Server (GPL) Copyright (c) , , Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> GRANT ALL PRIVILEGES ON *.* TO 'cdh'@'172.30.1.10%' IDENTIFIED BY 'yinzhengjie' WITH GRANT OPTION;
Query OK, rows affected (0.00 sec) mysql> GRANT ALL PRIVILEGES ON *.* TO 'cdh'@'node101.yinzhengjie.org.cn' IDENTIFIED BY 'yinzhengjie' WITH GRANT OPTION;
Query OK, rows affected (0.00 sec) mysql> FLUSH PRIVILEGES;
Query OK, rows affected (0.00 sec) mysql> quit
Bye
[root@node101 mysql]#
四.部署cloudrea manager软件
1>.安装相应的依赖包（每个节点都需要安装哟～）
[root@node101 ~]# ansible datanode -m shell -a 'yum -y install chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb'
2>.安装Cloudera Manager Server，Agent（在所有机器）
2.1>.CDH的选择
下载地址：http://archive.cloudera.com/cdh5/parcels/5.15.1/
2.2>.CM的选择
下载地址：http://archive.cloudera.com/cm5/cm/5/
2.3>.温馨提示：
如果网速较慢的小伙伴可以用不建议使用wget下载，推荐使用迅雷下载再上传到服务器，如果有百度云超级会员的话也可以使用我给大家体现下载好的软件。百度云链接：https://pan.baidu.com/s/148t70GenClGQRtJYX8WjvQ 密码：i2yd
[root@node101 CDH-5.15.]# ll
total
-rw-r--r--. root root Jan : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel
-rw-r--r--. root root Jan : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha1
-rw-r--r--. root root Jan : cloudera-manager-centos7-cm5..1_x86_64.tar.gz
-rw-r--r--. root root Jan : jdk-8u131-linux-x64.tar.gz
-rw-r--r--. root root Jan : manifest.json
-rw-r--r--. root root Jan : mysql-connector-java-5.1..jar
[root@node101 CDH-5.15.]#
[root@node101 CDH-5.15.]# ansible all -m shell -a 'ls /yinzhengjie/softwares'
node103.yinzhengjie.org.cn | CHANGED | rc= >>
jdk
jdk1..0_131 node101.yinzhengjie.org.cn | CHANGED | rc= >>
CDH-5.15.
jdk
jdk1..0_131 node102.yinzhengjie.org.cn | CHANGED | rc= >>
jdk
jdk1..0_131 [root@node101 CDH-5.15.]# ansible all -m copy -a 'src=/yinzhengjie/softwares/CDH-5.15.1/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz dest=/yinzhengjie/softwares/' node101.yinzhengjie.org.cn | CHANGED => {
"changed": true,
"checksum": "4227988280e0f09164bbfc6c59f291c6a814b8be",
"dest": "/yinzhengjie/softwares/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz",
"gid": ,
"group": "root",
"md5sum": "ca0be616f62668915d9ceeb22bdcdd4b",
"mode": "",
"owner": "root",
"secontext": "system_u:object_r:default_t:s0",
"size": ,
"src": "/root/.ansible/tmp/ansible-tmp-1547172382.51-139484190159118/source",
"state": "file",
"uid":
}
node103.yinzhengjie.org.cn | CHANGED => {
"changed": true,
"checksum": "4227988280e0f09164bbfc6c59f291c6a814b8be",
"dest": "/yinzhengjie/softwares/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz",
"gid": ,
"group": "root",
"md5sum": "ca0be616f62668915d9ceeb22bdcdd4b",
"mode": "",
"owner": "root",
"secontext": "system_u:object_r:default_t:s0",
"size": ,
"src": "/root/.ansible/tmp/ansible-tmp-1547172382.57-61195539797705/source",
"state": "file",
"uid":
}
node102.yinzhengjie.org.cn | CHANGED => {
"changed": true,
"checksum": "4227988280e0f09164bbfc6c59f291c6a814b8be",
"dest": "/yinzhengjie/softwares/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz",
"gid": ,
"group": "root",
"md5sum": "ca0be616f62668915d9ceeb22bdcdd4b",
"mode": "",
"owner": "root",
"secontext": "system_u:object_r:default_t:s0",
"size": ,
"src": "/root/.ansible/tmp/ansible-tmp-1547172382.54-248038987426177/source",
"state": "file",
"uid":
}
[root@node101 CDH-5.15.]#
[root@node101 CDH-5.15.]#
[root@node101 CDH-5.15.]#
[root@node101 CDH-5.15.]# ansible all -m shell -a 'ls /yinzhengjie/softwares'
node101.yinzhengjie.org.cn | CHANGED | rc= >>
CDH-5.15.
cloudera-manager-centos7-cm5..1_x86_64.tar.gz
jdk
jdk1..0_131 node103.yinzhengjie.org.cn | CHANGED | rc= >>
cloudera-manager-centos7-cm5..1_x86_64.tar.gz
jdk
jdk1..0_131 node102.yinzhengjie.org.cn | CHANGED | rc= >>
cloudera-manager-centos7-cm5..1_x86_64.tar.gz
jdk
jdk1..0_131 [root@node101 CDH-5.15.]#
分发Cloudera Manager到各个节点（[root@node101 CDH-5.15.1]# ansible all -m copy -a 'src=/yinzhengjie/softwares/CDH-5.15.1/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz dest=/yinzhengjie/softwares/'）
[root@node101 CDH-5.15.]# ansible all -m shell -a 'mkdir /opt/cloudera-manager'
[WARNING]: Consider using the file module with state=directory rather than running mkdir. If you need to use command because file is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node103.yinzhengjie.org.cn | CHANGED | rc= >> node101.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 CDH-5.15.]#
创建解压CM默认解压的目录，如果你想要使用别的解压目录别忘了修改配置文件哟（[root@node101 CDH-5.15.1]# ansible all -m shell -a 'mkdir /opt/cloudera-manager'）
[root@node101 softwares]# ansible all -m shell -a 'tar -zxf /yinzhengjie/softwares/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz -C /opt/cloudera-manager'
[WARNING]: Consider using the unarchive module rather than running tar. If you need to use command because unarchive is insufficient you can add warn=False to this
command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> node103.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 softwares]#
[root@node101 softwares]# ansible all -m shell -a 'ls -l /opt/cloudera-manager'
node101.yinzhengjie.org.cn | CHANGED | rc= >>
total
drwxr-xr-x. Jul : cloudera
drwxr-xr-x. Jul : cm-5.15. node102.yinzhengjie.org.cn | CHANGED | rc= >>
total
drwxr-xr-x. Jul : cloudera
drwxr-xr-x. Jul : cm-5.15. node103.yinzhengjie.org.cn | CHANGED | rc= >>
total
drwxr-xr-x. Jul : cloudera
drwxr-xr-x. Jul : cm-5.15. [root@node101 softwares]#
将CM解压到我们刚刚创建的目录（[root@node101 softwares]# ansible all -m shell -a 'tar -zxf /yinzhengjie/softwares/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz -C /opt/cloudera-manager'）
[root@node101 softwares]# ansible all -m shell -a 'useradd --system --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm'
node102.yinzhengjie.org.cn | CHANGED | rc= >> node101.yinzhengjie.org.cn | CHANGED | rc= >> node103.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 softwares]#
[root@node101 softwares]#
[root@node101 softwares]# ansible all -m shell -a 'id cloudera-scm'
node101.yinzhengjie.org.cn | CHANGED | rc= >>
uid=(cloudera-scm) gid=(cloudera-scm) groups=(cloudera-scm) node102.yinzhengjie.org.cn | CHANGED | rc= >>
uid=(cloudera-scm) gid=(cloudera-scm) groups=(cloudera-scm) node103.yinzhengjie.org.cn | CHANGED | rc= >>
uid=(cloudera-scm) gid=(cloudera-scm) groups=(cloudera-scm) [root@node101 softwares]#
在所有的节点上创建clodera-scm用户（[root@node101 softwares]# ansible all -m shell -a 'useradd --system --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm'）
3>.配置CM的Agent端
[root@node101 softwares]# ansible all -m shell -a 'grep server_host /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini'
node103.yinzhengjie.org.cn | CHANGED | rc= >>
server_host=localhost node102.yinzhengjie.org.cn | CHANGED | rc= >>
server_host=localhost node101.yinzhengjie.org.cn | CHANGED | rc= >>
server_host=localhost [root@node101 softwares]#
[root@node101 softwares]# ansible all -m shell -a 'sed -i 's/server_host=localhost/server_host=node101.yinzhengjie.org.cn/g' /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini'
[WARNING]: Consider using the replace, lineinfile or template module rather than running sed. If you need to use command because replace, lineinfile or template is
insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node103.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> node101.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 softwares]#
[root@node101 softwares]# ansible all -m shell -a 'grep server_host /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini' node101.yinzhengjie.org.cn | CHANGED | rc= >>
server_host=node101.yinzhengjie.org.cn node103.yinzhengjie.org.cn | CHANGED | rc= >>
server_host=node101.yinzhengjie.org.cn node102.yinzhengjie.org.cn | CHANGED | rc= >>
server_host=node101.yinzhengjie.org.cn [root@node101 softwares]#
我们指定CM的Server端([root@node101 softwares]# ansible all -m shell -a 'sed -i 's/server_host=localhost/server_host=node101.yinzhengjie.org.cn/g' /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini')
4>.初始化CM Server的数据库
[root@node101 CDH-5.15.]# ansible namenode -m shell -a 'mkdir /usr/share/java'
[WARNING]: Consider using the file module with state=directory rather than running mkdir. If you need to use command because file is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 CDH-5.15.]#
创建java环境目录（[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'mkdir /usr/share/java'）
[root@node101 CDH-5.15.]# ansible namenode -m shell -a 'ls -l /usr/share/java'
node101.yinzhengjie.org.cn | CHANGED | rc= >>
total [root@node101 CDH-5.15.]#
[root@node101 CDH-5.15.]# ansible namenode -m shell -a 'mv /yinzhengjie/softwares/CDH-5.15.1/mysql-connector-java-5.1.26.jar /usr/share/java/mysql-connector-java.jar'
node101.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 CDH-5.15.]# ansible namenode -m shell -a 'ls -l /usr/share/java'
node101.yinzhengjie.org.cn | CHANGED | rc= >>
total
-rw-r--r--. root root Jan : mysql-connector-java.jar [root@node101 CDH-5.15.]#
将mysql的驱动拷贝到上一步创建的java环境目录，CM数据库初始化脚本默认会去该目录查找相应的驱动（[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'mv /yinzhengjie/softwares/CDH-5.15.1/mysql-connector-java-5.1.26.jar /usr/share/java/mysql-connector-java.jar'）
[root@node101 CDH-5.15.]# ansible namenode -m shell -a '/opt/cloudera-manager/cm-5.15.1/share/cmf/schema/scm_prepare_database.sh mysql cdh -h node101.yinzhengjie.org.cn -ucdh -pyinzhengjie --scm-host node101.yinzhengjie.org.cn scm scm scm'
node101.yinzhengjie.org.cn | CHANGED | rc= >>
JAVA_HOME=/usr/java/jdk1.
Verifying that we can write to /opt/cloudera-manager/cm-5.15./etc/cloudera-scm-server
Creating SCM configuration file in /opt/cloudera-manager/cm-5.15./etc/cloudera-scm-server
Executing: /usr/java/jdk1./bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera-manager/cm-5.15./share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
[ main] DbCommandExecutor INFO Successfully connected to database.
All done, your SCM database is configured correctly! [root@node101 CDH-5.15.1]#
执行数据库初始化脚本（[root@node101 CDH-5.15.1]# ansible namenode -m shell -a '/opt/cloudera-manager/cm-5.15.1/share/cmf/schema/scm_prepare_database.sh mysql cdh -h node101.yinzhengjie.org.cn -ucdh -pyinzhengjie --scm-host node101.yinzhengjie.org.cn scm scm scm' #其格式为：数据库类型，数据库，数据库服务器，用户名，密码，CM服务器，后面的三个scm请不要改动！）
5>.制作CDH本地源
[root@node101 CDH-5.15.]# ansible namenode -m shell -a 'mkdir -p /opt/cloudera/parcel-repo'
[WARNING]: Consider using the file module with state=directory rather than running mkdir. If you need to use command because file is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 CDH-5.15.]#
Server端创建Parcel目录（[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'mkdir -p /opt/cloudera/parcel-repo'）
[root@node101 CDH-5.15.]# ansible namenode -m shell -a 'chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo/'
[WARNING]: Consider using the file module with owner rather than running chown. If you need to use command because file is insufficient you can add warn=False to this
command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 CDH-5.15.]#
[root@node101 CDH-5.15.]# ansible namenode -m shell -a 'ls -ld /opt/cloudera/parcel-repo/'
node101.yinzhengjie.org.cn | CHANGED | rc= >>
drwxr-xr-x. cloudera-scm cloudera-scm Jan : /opt/cloudera/parcel-repo/ [root@node101 CDH-5.15.]#
将上一步创建的目录所有者和所属组改为cloudera-scm用户（[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo/'）
[root@node101 CDH-5.15.]# ansible datanode -m shell -a 'mkdir -p /opt/cloudera/parcels'
[WARNING]: Consider using the file module with state=directory rather than running mkdir. If you need to use command because file is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> node103.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 CDH-5.15.]#
[root@node101 CDH-5.15.]# ansible datanode -m shell -a 'ls -ld /opt/cloudera/parcels'
node102.yinzhengjie.org.cn | CHANGED | rc= >>
drwxr-xr-x. root root Jan : /opt/cloudera/parcels node103.yinzhengjie.org.cn | CHANGED | rc= >>
drwxr-xr-x. root root Jan : /opt/cloudera/parcels node101.yinzhengjie.org.cn | CHANGED | rc= >>
drwxr-xr-x. root root Jan : /opt/cloudera/parcels [root@node101 CDH-5.15.]#
agent端创建parcels目录（[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'mkdir -p /opt/cloudera/parcels' ）
[root@node101 CDH-5.15.]# ansible datanode -m shell -a 'chown cloudera-scm:cloudera-scm /opt/cloudera/parcels'
[WARNING]: Consider using the file module with owner rather than running chown. If you need to use command because file is insufficient you can add warn=False to this
command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc= >> node103.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 CDH-5.15.]#
[root@node101 CDH-5.15.]# ansible datanode -m shell -a 'ls -ld /opt/cloudera/parcels'
node101.yinzhengjie.org.cn | CHANGED | rc= >>
drwxr-xr-x. cloudera-scm cloudera-scm Jan : /opt/cloudera/parcels node103.yinzhengjie.org.cn | CHANGED | rc= >>
drwxr-xr-x. cloudera-scm cloudera-scm Jan : /opt/cloudera/parcels node102.yinzhengjie.org.cn | CHANGED | rc= >>
drwxr-xr-x. cloudera-scm cloudera-scm Jan : /opt/cloudera/parcels [root@node101 CDH-5.15.]#
别忘了把parcels目录权限授权给我们之前创建的cloudera-scm用户！（[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'chown cloudera-scm:cloudera-scm /opt/cloudera/parcels'）
[root@node101 CDH-5.15.]#
[root@node101 CDH-5.15.]# cp CDH-5.15.-.cdh5.15.1.p0.-el7.parcel /opt/cloudera/parcel-repo/
[root@node101 CDH-5.15.]#
[root@node101 CDH-5.15.]# cp manifest.json /opt/cloudera/parcel-repo/ #别小瞧这个文件，尽管它不是很大，但是它却记录着CDH和Hadoop生态圈组件的版本依赖关系！
[root@node101 CDH-5.15.]#
[root@node101 CDH-5.15.]# cp CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha1 /opt/cloudera/parcel-repo/CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha #注意，我在拷贝到时候重命名该文件了！
[root@node101 CDH-5.15.]#
[root@node101 CDH-5.15.]# ll /opt/cloudera/parcel-repo/
total
-rw-r--r-- root root Sep : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel
-rw-r--r-- root root Sep : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha
-rw-r--r-- root root Sep : manifest.json
[root@node101 CDH-5.15.]# 温馨提示：
如果你没有下载到“CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha”文件是，可以找到“manifest.json”文件中"parcelName": "CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel"对应的"hash": "deff00898e410a34cf0a1e66c5dbe87546608f0c"复制到该文件即可。这个方法也适用于其他的版本！
五.启动集群
1>.启动cloudera manager的Server 端
[root@node101 CDH-5.15.]# ansible namenode -m shell -a '/opt/cloudera-manager/cm-5.15.1/etc/init.d/cloudera-scm-server start'
node101.yinzhengjie.org.cn | CHANGED | rc= >>
Starting cloudera-scm-server: [ OK ] [root@node101 CDH-5.15.]#
[root@node101 CDH-5.15.]# netstat -untalp | grep
tcp 0.0.0.0: 0.0.0.0:* LISTEN /java
[root@node101 CDH-5.15.]#
............（信息会有很多，你会看到日志中启动了很多端口，如果7180默认端口启动成功，那么你就可以访问WebUI界面了，当然这个端口咱们是可以修改的！）
-- ::, INFO WebServerImpl:com.cloudera.server.web.cmon.JobDetailGatekeeper: ActivityMonitor configured to allow job details for all jobs.
-- ::, INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Discovered parcel on CM server: CDH-5.15.-.cdh5.15.1.p0.-el7.parcel
-- ::, INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Created torrent file: /opt/cloudera/parcel-repo/CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.torrent
-- ::, INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Creating single-file torrent for CDH-5.15.-.cdh5.15.1.p0.-el7.parcel...
-- ::, INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Hashing data from CDH-5.15.-.cdh5.15.1.p0.-el7.parcel with threads ( pieces)...
-- ::, INFO WebServerImpl:com.cloudera.server.web.cmf.AggregatorController: AggregateSummaryScheduler started.
-- ::, INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... % complete
-- ::, ERROR SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: The server storage directory [/var/lib/cloudera-scm-server] doesn't exist.
-- ::, ERROR SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: No read permission to the server storage directory [/var/lib/cloudera-scm-server]
-- ::, ERROR SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: No write permission to the server storage directory [/var/lib/cloudera-scm-server]
-- ::, INFO WebServerImpl:org.mortbay.log: jetty-6.1..cloudera.
-- ::, INFO WebServerImpl:org.mortbay.log: Started SelectChannelConnector@0.0.0.0:
-- ::, INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
-- ::, INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... % complete
-- ::, INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... % complete
-- ::, INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... % complete
-- ::, INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... % complete
服务端启动时，我们应该查看相应的日志信息（[root@node101 CDH-5.15.1]# tail -f /opt/cloudera-manager/cm-5.15.1/log/cloudera-scm-server/cloudera-scm-server.log）
2>.启动cloudera manager的Agent 端
[root@node101 CDH-5.15.]# ansible datanode -m shell -a '/opt/cloudera-manager/cm-5.15.1/etc/init.d/cloudera-scm-agent start'
node102.yinzhengjie.org.cn | CHANGED | rc= >>
Starting cloudera-scm-agent: [ OK ] node101.yinzhengjie.org.cn | CHANGED | rc= >>
Starting cloudera-scm-agent: [ OK ] node103.yinzhengjie.org.cn | CHANGED | rc= >>
Starting cloudera-scm-agent: [ OK ] [root@node101 CDH-5.15.]#
[root@node101 CDH-5.15.]# ansible datanode -m shell -a '/opt/cloudera-manager/cm-5.15.1/etc/init.d/cloudera-scm-agent status'
node103.yinzhengjie.org.cn | CHANGED | rc= >>
cloudera-scm-agent (pid ) is running... node102.yinzhengjie.org.cn | CHANGED | rc= >>
cloudera-scm-agent (pid ) is running... node101.yinzhengjie.org.cn | CHANGED | rc= >>
cloudera-scm-agent (pid ) is running... [root@node101 CDH-5.15.]#
3>.关闭防火墙和selinux
温馨提示：
在实际生产环境中，如果是内网环境咱们可以关闭防火墙和selinux，风向毕竟很小。而且很多公司都是有硬件防火墙的，如果咱们在开启防火墙会影响服务器性能。由于我是测试机环境，为了实验方便就直接禁用这2个服务了，实际生产中，根据你的需求选择是否开启防火墙，到时候你只需要定义一些列端口开放规则即可。
[root@node101 CDH-5.15.]# ansible datanode -m shell -a 'systemctl status firewalld'
node101.yinzhengjie.org.cn | CHANGED | rc= >>
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since Thu -- :: HKT; 17h ago
Main PID: (firewalld)
CGroup: /system.slice/firewalld.service
└─ /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid Jan :: node101.yinzhengjie.org.cn systemd[]: Starting firewalld - dynamic firewall daemon...
Jan :: node101.yinzhengjie.org.cn systemd[]: Started firewalld - dynamic firewall daemon. node103.yinzhengjie.org.cn | CHANGED | rc= >>
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since Thu -- :: HKT; 17h ago
Main PID: (firewalld)
CGroup: /system.slice/firewalld.service
└─ /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid Jan :: node103.yinzhengjie.org.cn systemd[]: Starting firewalld - dynamic firewall daemon...
Jan :: node103.yinzhengjie.org.cn systemd[]: Started firewalld - dynamic firewall daemon. node102.yinzhengjie.org.cn | CHANGED | rc= >>
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since Thu -- :: HKT; 17h ago
Main PID: (firewalld)
CGroup: /system.slice/firewalld.service
└─ /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid Jan :: node102.yinzhengjie.org.cn systemd[]: Starting firewalld - dynamic firewall daemon...
Jan :: node102.yinzhengjie.org.cn systemd[]: Started firewalld - dynamic firewall daemon. [root@node101 CDH-5.15.]#
检查集群的防火墙状态（[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'systemctl status firewalld' ）
[root@node101 CDH-5.15.]# ansible datanode -m shell -a 'systemctl stop firewalld'
node102.yinzhengjie.org.cn | CHANGED | rc= >> node103.yinzhengjie.org.cn | CHANGED | rc= >> node101.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 CDH-5.15.]#
停用防火墙（[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'systemctl stop firewalld' ）
[root@node101 CDH-5.15.]# ansible datanode -m shell -a 'systemctl disable firewalld'
node103.yinzhengjie.org.cn | CHANGED | rc= >>
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service. node102.yinzhengjie.org.cn | CHANGED | rc= >>
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. node101.yinzhengjie.org.cn | CHANGED | rc= >>
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service. [root@node101 CDH-5.15.]#
禁用防火墙开机自启动（[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'systemctl disable firewalld'）
[root@node101 CDH-5.15.]# ansible datanode -m shell -a 'setenforce 0'
node103.yinzhengjie.org.cn | CHANGED | rc= >> node101.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 CDH-5.15.]#
临时禁用selinux（[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'setenforce 0' ）
[root@node101 CDH-5.15.]# ansible datanode -m shell -a 'getenforce'
node101.yinzhengjie.org.cn | CHANGED | rc= >>
Permissive node102.yinzhengjie.org.cn | CHANGED | rc= >>
Permissive node103.yinzhengjie.org.cn | CHANGED | rc= >>
Permissive [root@node101 CDH-5.15.]#
查看selinux的状态（[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'getenforce' ）
[root@node101 CDH-5.15.]# ansible datanode -m shell -a 'sed -i 's#SELINUX=enforcing#SELINUX=disabled#' /etc/selinux/config'
[WARNING]: Consider using the replace, lineinfile or template module rather than running sed. If you need to use command because replace, lineinfile or template is
insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc= >> node103.yinzhengjie.org.cn | CHANGED | rc= >> node102.yinzhengjie.org.cn | CHANGED | rc= >> [root@node101 CDH-5.15.]#
[root@node101 CDH-5.15.]#
[root@node101 CDH-5.15.]# ansible datanode -m shell -a 'cat /etc/selinux/config | grep SELINUX= | grep -v ^#'
node103.yinzhengjie.org.cn | CHANGED | rc= >>
SELINUX=disabled node101.yinzhengjie.org.cn | CHANGED | rc= >>
SELINUX=disabled node102.yinzhengjie.org.cn | CHANGED | rc= >>
SELINUX=disabled [root@node101 CDH-5.15.]#
永久禁用selinux，重启操作系统后生效（[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'sed -i 's#SELINUX=enforcing#SELINUX=disabled#' /etc/selinux/config'）
4>.如下图所示，访问webUI界面（http://node101.yinzhengjie.org.cn:7180/）
我们安装CM的过程通过WebUI的安装向导来进行安装，推荐使用谷歌浏览器，不要使用容易崩溃的浏览器，这样会影响你安装进度的！如果你也出现了以上界面，恭喜你CM部署成功，关于CDH的部署详情请参考：https://www.cnblogs.com/yinzhengjie/p/9638360.html。
使用ansible部署CDH 5.15.1大数据集群的更多相关文章
- CDH版本大数据集群下搭建Hue（hadoop-2.6.0-cdh5.5.4.gz + hue-3.9.0-cdh5.5.4.tar.gz）（博主推荐）
不多说,直接上干货! 我的集群机器情况是 bigdatamaster(192.168.80.10).bigdataslave1(192.168.80.11)和bigdataslave2(192.168 ...
- centos7 ambari2.6.1.5+hdp2.6.4.0 大数据集群安装部署
前言 本文是讲如何在centos7(64位) 安装ambari+hdp,如果在装有原生hadoop等集群的机器上安装,需要先将集群服务停掉,然后将不需要的环境变量注释掉即可,如果不注释掉,后面虽然可以 ...
- 基于Docker搭建大数据集群（一）Docker环境部署
本篇文章是基于Docker搭建大数据集群系列的开篇之作 主要内容 docker搭建 docker部署CentOS 容器免密钥通信 容器保存成镜像 docker镜像发布 环境 Linux 7.6 一.D ...
- 使用Ansible部署etcd 3.2高可用集群
之前写过一篇手动搭建etcd 3.1集群的文章<etcd 3.1 高可用集群搭建>,最近要初始化一套新的环境,考虑用ansible自动化部署整套环境, 先从部署etcd 3.2集群开始. ...
- Ubuntu14.04下Ambari安装搭建部署大数据集群（图文分五大步详解）（博主强烈推荐）
不多说,直接上干货! 写在前面的话 (1) 最近一段时间,因担任我团队实验室的大数据环境集群真实物理机器工作,至此,本人秉持负责.认真和细心的态度,先分别在虚拟机上模拟搭建ambari(基于CentO ...
- Ubuntu14.04下Cloudera安装搭建部署大数据集群（图文分五大步详解）（博主强烈推荐）（在线或离线）
第一步: Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一) 第二步: Cloudera Manager安装之时间服务器和时间客户端(Ub ...
- CDH版本大数据集群下搭建的Hue详细启动步骤（图文详解）
关于安装请见 CDH版本大数据集群下搭建Hue(hadoop-2.6.0-cdh5.5.4.gz + hue-3.9.0-cdh5.5.4.tar.gz)(博主推荐) Hue的启动 也就是说,你Hue ...
- 基于Docker搭建大数据集群（七）Hbase部署
基于Docker搭建大数据集群(七)Hbase搭建 一.安装包准备 Hbase官网下载 微云下载 | 在 tar 目录下 二.版本兼容 三.角色分配 节点 Master Regionserver cl ...
- 大数据集群Linux CentOS 7.6 系统调优篇
大数据集群Linux CentOS 7.6 系统调优篇 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 一.设置主机hosts文件 1>.修改主机名 [root@node100 ...
随机推荐
- win10找不到hosts文件(设置显示隐藏文件也找不到)解决方法
Win10系统中的Hosts文件有很多作用,屏蔽网址,指定解析,跳转等等,所以我们经常会通过编辑Hosts文件来达成一些目的,一般来说hosts文件是隐藏的,我们需要显示受保护的文件才可以,但是有一些 ...
- Centos7.3使用脚本自动静默安装oracle11.2.0.4数据库
一直想着写一个脚本实现自动化安装oracle数据库.以下内容实验过几次了,可能还存在些小问题,如果在跑以下脚本中遇到问题,自己仔细排查即可 挣扎了好久,总算还是没实现,目前只能通过依次执行多个脚本来安 ...
- 【视频版】PDF合并器破解视频教程
无聊顺手录了一下,需要的拿去. 下载地址: 链接:https://pan.baidu.com/s/1TtK1JNzNw0BIl0eRPS_nlw 提取码复制可见:pqgi
- 2017ACM/ICPC广西邀请赛 1007 Duizi and Shunzi
Duizi and Shunzi Time Limit: 6000/3000 MS (Java/Others) Memory Limit: 32768/32768 K (Java/Others) ...
- python 可变数据类型和不可变数据类型（7）
python数据类型分别有整数int / 浮点数float / 布尔值bool / 元组tuple / 列表list / 字典dict,其中数据类型分为两个大类,一种是可变数据类型:一种是不可变数据类 ...
- Centos7挂载新硬盘
1.查看系统是否检测到新的硬盘设备 ls /dev/ |grep sd linux 中所有外设都会在这个目录下,对应一个文件,其中第一块硬盘是sda,第二块硬盘是sdb,第三块硬盘是sdc.其中sda ...
- LeetCode 240. 搜索二维矩阵 II(Search a 2D Matrix II) 37
240. 搜索二维矩阵 II 240. Search a 2D Matrix II 题目描述 编写一个高效的算法来搜索 m x n 矩阵 matrix 中的一个目标值 target.该矩阵具有以下特性 ...
- Linux基础系统优化（二）
SELinux功能 SELinux(Security-Enhanced Linux) 是美国国家安全局(NSA)对于强制访问控制的实现,这个功能管理员又爱又恨,大多数生产环境也是关闭的做法,安全手段使 ...
- Magic Line（思维+计算几何问题）（2019牛客暑期多校训练营（第三场））
示例: 输入: 140 1-1 01 00 -1 输出:-1 999000000 1 -999000001 题意:给定平面上一系列的点,求一条以(x1,y1),(x2,y2)两点表示的直线将平面分为包 ...
- centos7 设置静态ip , 并开机联网
修改 /etc/sysconfig/network-scripts 下的文件 我的是ens32 (不同系统这个文件名字不一样) 内容如下 TYPE=Ethernet PROXY_METHOD=none ...