最新版大数据平台安装部署指南，HDP-2.6.5.0，ambari-2.6.2.0

一、服务器环境配置

1 系统要求

名称	地址	操作系统	root密码
Master1	10.1.0.30	Centos 7.7	Root@bidsum1
Master2	10.1.0.105	Centos 7.7
Worker1	10.1.0.127	Centos 7.7
Worker2	10.1.0.106	Centos 7.7
Worker3	10.1.0.3	Centos 7.7
系统要求	Centos 7.3以上 64bit 最小化安装
浏览器	Google Chrome

2 SSH免密登录配置

ssh-keygen

cat .ssh/*.pub > .ssh/authorized_keys

scp -r .ssh root@10.1.0.30:/root/

scp -r .ssh root@10.1.0.105:/root/

3 主机名IP映射配置

在master 机器上执行如下命令

vi /etc/hosts

192.168.217.130 hdp-0001

192.168.217.131 hdp-0002

192.168.217.132 hdp-0003

scp /etc/hosts root@hdp-0002:/etc/hosts

scp /etc/hosts root@hdp-0003:/etc/hosts

4 关闭及禁止防火墙

systemctl stop firewalld

systemctl disable firewalld

5 配置阿里yum源(可选)

mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.bak

wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo

yum makecache

6 开启时钟同步

yum install -y ntp

systemctl enable ntpd

systemctl start ntpd

7 关闭SELinux

vi /etc/sysconfig/selinux

#修改以下内容

SELINUX=disabled

8 安装mysql数据库

mysql服务安装在master节点，也可选择其他节点或机器。

具体步骤如下：

8.1数据库安装

wget http://repo.mysql.com/mysql57-community-release-el7-8.noarch.rpm

yum install mysql-server

[root@master ~]# service mysqld  start

[root@master ~]# grep "password" /var/log/mysqld.log

[root@master ~]# mysql -u root -p

Enter password:

mysql>set global validate_password_policy=LOW;

mysql> set password = password("mysql@Edmp.com");

mysql> grant all privileges on *.* to root@"%" identified by " mysql@Edmp.com" ;

mysql> flush privileges;

8.2安装成功后，创建ambari数据库及用户

#进入mysql shell

mysql -uroot –p

#创建ambari用户

create database ambari character set utf8 ;

CREATE USER 'ambari'@'%'IDENTIFIED BY 'ambari123';

GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';

FLUSH PRIVILEGES;

#创建hive用户

create database hive character set utf8 ;

CREATE USER 'hive'@'%'IDENTIFIED BY 'hive123';

GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%';

FLUSH PRIVILEGES;

#创建oozie用户

create database oozie character set utf8 ;

CREATE USER 'oozie'@'%'IDENTIFIED BY 'oozie123';

GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%';

FLUSH PRIVILEGES;

9 安装JDK

9.1解压安装包并配置环境变量

mkdir -p /opt/java

tar -zxvf jdk-8u181-linux-x64.tar.gz -C /opt/java/

vi /etc/profile

export JAVA_HOME=/opt/java/jdk1.8.0_181

export JRE_HOME=$JAVA_HOME/jre

export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib

export PATH=$PATH:$JAVA_HOME/bin

9.2分发到其他服务器

scp -r /opt/java/jdk1.8.0_181/ root@hdc-data2:/opt/java/

scp -r /opt/java/jdk1.8.0_181/ root@hdc-data3:/opt/java/

scp /etc/profile root@hdc-data2:/etc/

scp /etc/profile root@hdc-data3:/etc/

source /etc/profile

二、配置本地yum库

1 服务安装配置

yum install yum-utils createrepo

yum install httpd

service httpd start

2 磁盘挂载

磁盘文件系统可用ext4或者xfs，挂载时需添加noatime选项

mkdir /data

vi /etc/fstab

/dev/vdb /data xfs defaults,noatime 0 0

mkfs -t xfs /dev/vdb

mount /dev/vdb /data

3下载镜像包，解压到指定目录

操作系统	版本	库名称	下载链接
CentOS 7	2.6.5.0	HDP	http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/HDP-2.6.5.0-centos7-rpm.tar.gz
			Md5: 1775e80b7fcc16ff1f908f3d81693854
		HDP-UTILS	http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
			Md5: ebf4c5519640b6a62a3def461ea66f13
		HDP-GPL	http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.5.0/HDP-GPL-2.6.5.0-centos7-gpl.tar.gz
			Md5: c1e0b4b6517142bf61f7bbdef967558f
CentOS 7	2.6.2.0	Ambari	http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.0/ambari-2.6.2.0-centos7.tar.gz
			Md5: 6d87af3788b8fdf4a6abaa759b1dc84e
		Jdk	jdk-8u281-linux-x64.tar.gz
			Md5: e85f591a370400c99fc314e5c43015a7

下载后把镜像包解压缩，并且移动到/var/www/html/hdp 目录下；

mkdir /var/www/html/ambari

tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/ambari/

tar -zxvf ambari-2.6.5.0-centos7.tar.gz -C /var/www/html/ambari/

tar -zxvf HDP-2.6.5.0-centos7-rpm.tar.gz -C /var/www/html/ambari/

tar -zxvf HDP-GPL-2.6.5.0-centos7-gpl.tar.gz -C /var/www/html/ambari/

三、安装配置ambari Server

1 配置ambari 的yum源（在master节点）

wget  -nv  http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.5.0.3/ambari.repo -O /etc/yum.repos.d/ambari.repo

wget -nv http://public-repo-1.hortonworks.com/HDP/centos7/2.x/

updates/2.6.0.3/hdp.repo -O /etc/yum.repos.d/hdp.repo

2 修改ambari.repo配置信息

#VERSION_NUMBER=2.5.0.3-7

[ambari-2.5.0.3]

name=ambari Version - ambari-2.5.0.3

baseurl=http://10.1.0.30/ambari/ambari/centos7

gpgcheck=0

gpgkey=http://10.1.0.30/ambari/ambari/centos7/2.x/updates/2.5.0.3/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

enabled=1

priority=1

3 执行ambari server安装

yum install ambari-server

4 安装mysql-java驱动

yum install –y mysql-connector-java.noarch

找到/usr/share/java/mysql-connector-java.jar

将mysql-connector-java.jar复制到/var/lib/ambari-server/resources目录下

cp mysql-connector-java-5.1.40.jar /var/lib/ambari-server/resources/mysql-jdbc-driver.jar

5 ambaria初始化

#设置mysql驱动

ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar

ambari-server setup

[root@master yum.repos.d]# ambari-server setup

Checking SELinux...

SELinux status is 'enabled'

SELinux mode is 'permissive'

WARNING: SELinux is set to 'permissive' mode and temporarily disabled.

OK to continue [y/n] (y)? y

Customize user account for ambari-server daemon [y/n] (n)?

Adjusting ambari-server permissions and ownership...

Checking firewall status...

Checking JDK...

[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8

[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7

[3] Custom JDK

==============================================================================

Enter choice (1): 1

…………

Successfully downloaded JDK distribution to /var/lib/ambari-server/resources/jdk-8u112-linux-x64.tar.gz

Installing JDK to /usr/jdk64/

Successfully installed JDK to /usr/jdk64/

Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/jce_policy-8.zip

Installing JCE policy...

Checking GPL software agreement...

GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html

Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)?

Completing setup...

Configuring database...

Enter advanced database configuration [y/n] (n)? y

Configuring database...

==============================================================================

Choose one of the following options:

[1] - PostgreSQL (Embedded)

[2] - Oracle

[3] - MySQL / MariaDB

[4] - PostgreSQL

[5] - Microsoft SQL Server (Tech Preview)

[6] - SQL Anywhere

[7] - BDB

==============================================================================

Enter choice (1): 3

Hostname (localhost): master.bigdata.com

Port (3306):

Database name (ambari):

Username (ambari):

Enter Database Password (bigdata): BigData123_

Configuring ambari database...

Configuring remote database connection properties...

WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql (此处转下表的数据库操作)

Proceed with configuring remote database connection properties [y/n] (y)? y

Extracting system views...

ambari-admin-2.6.1.5.3.jar

Ambari Server 'setup' completed successfully.

说明：

1：提示是否自定义设置。输入：y 或 按回车继续

2：ambari-server 账号。使用root（推荐）用户直接回车
若使用ambari用户输入：ambari

3：设置JDK,输入数字3，选客户端安装jdk，如果服务器可以访问外网，则可以选1，自动下载jdk1.8，默认下载的安装目录是/usr/java/default

4：如果上面选择3自定义JDK,则需要设置JAVA_HOME

5：输入：y 或 按回车继续

6：输入“y“，进行数据库的高级配置

7：输入“3“，选择mysql/mariadb数据库

8：设置数据库的配置参数：host,port,database,username,password。根据实际情况输入，如果和括号内相同，则可以直接回车。若端口改6033则输入port:6033。

9：输入：y 或 按回车继续

（若设置更改，可以再次执行ambari-server setup进行设置）

数据库操作

[root@master ~]# mysql -u root -p

Enter password:

mysql> use ambari;

Database changed

mysql> source
/var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

Query OK, 0 rows affected (0.00 sec)

6 服务启停

启动：ambari-server start

停止：ambari-server stop

启动成功后可以通过浏览器访问管理界面。

URL:
192.168.217.130:8080(IP为master节点IP)

用户名：admin 密码：admin

四、HDP集群部署

1
登录ambari
server 管理界面

1.在浏览器打开http://192.168.217.130:8080

2.输入用户名密码  默认admin/admin

2
进入集群安装向导

3
输入集群名称

建议设置为EDMP，标信数据管理平台简称

4
选择hdp版本

点击HDP-2.6，选择上传文件

在HDP/centos7/2.6.4.0-91/HDP-2.6.4.0-91.xml文件基础上，修改baseurl，修改完后如下：

选择增加版本、上传修改后的版本文件。

增加版本完成后，直接下一步

5
确认主机列表（并上传id_rsa文件）

6
主机注册、主机环境检查

  ERROR 2021-02-20 00:17:28,352 NetUtil.py:96 - EOF occurred in violation of protocol (_ssl.c:618)

ERROR 2021-02-20 00:17:28,352 NetUtil.py:97 - SSLError: Failed to connect. Please check openssl library versions.

这一步会出现一些问题，具体解决方案在第六部分安装过程中出现的问题中查找

7 选择安装部署组件

选择安装组件，（最小化安装HDFS+YARN+MAPREDUCE2+Ambari Metrics+SmartSense+ZooKeeper+Hbase）不必要的组件可以暂时不安装节省安装时间，后续可以再添加安装组件

8
分配服务角色

9
组件配置信息修改

通常数据目录修改到/data/目录下，

有红色信息提示的组件表示要输入相应的账号密码。

以下示例，可根据自己的实际情况配置。

9.1 HDFS

修改3个数据盘路径

9.2 HIVE

修改元数据库类型及地址，填写账号密码hive/hive123

9.3 oozie

填写元数据库账号密码，修改访问地址oozie/oozie123，路径参考hive

9.4 zookeeper

修改数据存放路径

9.5 ambari Metrics/ smartsence

设置账号密码，默认为admin/admin

附：暂时修改了以下属性，具体配置信息可根据实际情况后续相应修改

属性	修改值
HDFS
NameNode	/data/hadoop/hdfs/namenode
DataNode	/data/hadoop/hdfs/data
SecondaryNameNode Checkpoint directories	/data/hadoop/hdfs/namesecondary
Hadoop PID Dir Prefix	/data/var/run/hadoop
Hadoop Log Dir Prefix	/data/var/log/hadoop
dfs.journalnode.edits.dir	/data/hadoop/hdfs/journalnode
Yarn
yarn.nodemanager.local-dirs	/data/hadoop/yarn/local
yarn.nodemanager.log-dirs	/data/hadoop/yarn/log
yarn.timeline-service.leveldb-state-store.path	/data/hadoop/yarn/timeline
yarn.timeline-service.leveldb-timeline-store.path	/data/hadoop/yarn/timeline
YARN Log Dir Prefix	/data/var/log/hadoop-yarn
YARN PID Dir Prefix	/data/var/run/hadoop-yarn
Mapreduce
Mapreduce Log Dir Prefix	/data/var/log/hadoop-mapreduce
Mapreduce PID Dir Prefix	/data/var/run/hadoop-mapreduce
mapreduce.jobhistory.recovery.store.leveldb.path	/data/hadoop/mapreduce/jhs
Hive
Hive Log Dir	/data/var/log/hive
Hive PID Dir	/data/var/run/hive
HBase
HBase Log Dir Prefix	/data/var/log/hbase
HBase PID Dir	/data/var/run/hbase
Oozie
Oozie Data Dir	/data/hadoop/oozie/data
Oozie Log Dir	/data/var/log/oozie
Oozie PID Dir	/data/var/run/oozie
zookeeper
ZooKeeper directory	/data/hadoop/zookeeper
ZooKeeper Log Dir	/data/var/log/zookeeper
ZooKeeper PID Dir	/data/var/run/zookeeper
ambari-infra
Metrics Collector log dir	/data/var/log/ambari-metrics-collector
Metrics Collector pid dir	/data/var/run/ambari-metrics-collector
Metrics Monitor log dir	/data/var/log/ambari-metrics-monitor
Metrics Monitor pid dir	/data/var/run/ambari-metrics-monitor
Aggregator checkpoint directory	/data/var/lib/ambari-metrics-collector/checkpoint
Metrics Grafana data dir	/data/var/lib/ambari-metrics-grafana
Metrics Grafana log dir	/data/var/log/ambari-metrics-grafana
Metrics Grafana pid dir	/data/var/run/ambari-metrics-grafana
hbase_log_dir	/data/var/log/ambari-metrics-collector
hbase_pid_dir	/data/var/run/ambari-metrics-collector/
hbase.tmp.dir	/data/var/lib/ambari-metrics-collector/hbase-tmp
ambari-infra
Infra Solr Client log dir	/data/var/log/ambari-infra-solr-client
Infra Solr log dir	/data/var/log/ambari-infra-solr
Infra Solr pid dir	/data/var/run/ambari-infra-solr
spark
livy2_log_dir	/data/var/log/livy2
livy2_pid_dir	/data/var/run/livy2
spark_log_dir	/data/var/log/spark2
spark_pid_dir	/data/var/run/spark2

10
执行安装

11 安装成功，点击完成进入图形界面

五  HDFS启用HA(高可用)（暂时忽略）

1停止hbase hive相关服务，

2

在hdfs界面，点击 enable namenode ha

3 输入namenode ha serverid

六 安装过程中出现的问题

安装过程中排查问题具体思路：2个路径，第1个为安装页面message目录，点击蓝色进度信息跳转查看；第二个ambari-server后台日志，具体路径为

1）UID错误问题

具体报错信息如下

# 修改文件：/var/lib/ambari-server/resources/stacks/HDP/2.0.6/hooks/before-ANY/files/changeToSecureUid.sh

[root@manager ~]# vim
/var/lib/ambari-server/resources/stacks/HDP/2.0.6/hooks/before-ANY/files/changeToSecureUid.sh

function find_available_uid() {

for ((i=8001;
i<=9000; i++))

do

grep -q $i
/etc/passwd

if [
"$?" -ne 0 ]

then

newUid=$i

break

fi

done

}

# 修改原来的  for ((i=1001; i<=2000; i++)) 为 
for ((i=8001; i<=9000; i++))

# 8001，9000可以取值为暂时未使用的UID，可以使用id UID验证UID是否被使用，找一段未使用的UID即可

修改完之后需要重启ambari服务

修改了ambari脚本之后，需要执行以下步骤重启ambari

• 删除ambari-agent缓存：rm
-rf /var/lib/ambari-agent/cache/
• 重启ambari-server：ambari-server
restart
• 重启ambari-agent：ambari-agent
restart

2)8080端口报错

报错现象：

Ambari Agent host cannot reach Ambari Server 'localhost:8080'.

错误原因：

客户端调用setupAgent.py函数， ambari-server所在IP传参失败，从而导致了问题的发生。

解决办法：

找到文件：/usr/lib/ambari-server/lib/ambari_server/setupAgent.py

手动修改ambari-server hostname

160   expected_hostname = onlyargs[0]

161   passPhrase = onlyargs[1]

162   #hostname = onlyargs[2]

163   hostname = "192.168.31.150"

164   projectVersion = None

3）SSL HTTPS连接错误

注册ambari-agents时failed

NetUtil.py:96 - EOF occurred in violation of protocol (_ssl.c:579)

SSLError: Failed to connect. Please check openssl library versions.

解决：如下第三个方案，/etc/ambari-agent/conf/ambari-agent.ini文件只有执行该步骤进行注册才生成。修改后点击“Retry Failed”按钮，等待注册成功进入下一步。

# 1、yum upgrade openssl 已是最新本版，未解决

# 2、vi /etc/python/cert-verification.cfg 修改 verify=disable，未解决

[https]

#verify=platform_default

verify=disable

# 3、最后解决方案 在ambari-agent的配置文件/etc/ambari-agent/conf/ambari-agent.ini

 在 ［security］标签下面增加一项

[security]

force_https_protocol=PROTOCOL_TLSv1_2