Ubuntu16.04安装Ambari 2.7.3
概念了解
Ambair介绍
Apache Ambari是一个用于支持大数据软件供应 管理与监控软件.它也是一个分布式软件,分为Ambair-Server与Ambari-Client两个部分.在生产环境下一般单独用一台服务器安装ambari-server来确保服务的稳定性.然后需要安装大数据服务组件的服务器上均安装ambari-client组件,server发送命令与client进行交互完成任务.
HDP介绍
HDP是hortonworks公司的Hadoop发行版,其中包括大多数Hadoop生态下的重要组件,可以作为Ambari进行部署与安装.从官网上可以查看HDP中版本与其他组件的版本对应关系:https://zh.hortonworks.com/products/data-platforms/hdp/
准备工作
硬件准备
这里使用的是阿里云中Ubuntu16.06版本服务器,预装有Python2.7环境,机器三台,配置为一台2H8G与两台2H4G服务器作为测试搭建环境.系统准备好后防火墙及端口是全开放的,直接启用的root用户.若环境与之不符作为测试环境可以先匹配一下.选用配置最好的作为master,剩下的两台作为两个slave节点.
准备安装包
这边因为是要测试Spark的一些新特性,所以所准备的安装包均为当前时间点最新的,直接从hortonworks官网进行下载;
其中要安装的组件有JDK-1.8 确认一下Python版本为2.7 然后下载Ambari2.7.3对应的组件,其中包括:ambari-2.7.3.0 HDP HDP-UTILS HDP-GPL
首先是Ambari安装包位置:
选择下载最新版,这样子在版本控制方面选择余地更大
主页:
https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/ambari_repositories.html
https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/hdp_31_repositories.html
下载链接:
ambari-2.7.3.0 http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.7.3.0/ambari-2.7.3.0-ubuntu16.tar.gz
HDP http://public-repo-1.hortonworks.com/HDP/ubuntu16/3.x/updates/3.1.0.0/HDP-3.1.0.0-ubuntu16-deb.tar.gz
HDP-UTILS http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu16/HDP-UTILS-1.1.0.22-ubuntu16.tar.gz
HDP-GPL http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu16/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-ubuntu16-gpl.tar.gz
查看配置
首先查看机器硬件配置信息,包括内存及CPU配置信息,根据配置分配不同的角色:
cat /proc/meminfo |grep MemTotal
cat /proc/cpuinfo |grep “model name” && cat /proc/cpuinfo |grep “physical id”
主机命名 master.example.com node1.example.com node2.example.com(生产环境下建议使用域名服务器进行解析)
三步操作:1 )hostname 主机名 2)修改/etc/hosts 文件对应DNS解析 3)修改/etc/hostname文件 最后使用hostname hostname -A对结果进行检测,要求与设置值一致
修改/etc/hosts 添加对应的域名配置解析,将主机名与IP地址相对应(注意IP地址与DNS要做到一一对应,会存在反解析操作,否则有坑)
ssh免密码登录 ssh-keygen -t rsa 然后id_rsa.pub 导入到authorized_keys文件中 scp将文件发送到远程服务器对应目录上
配置环境
配置所有机器的JDK环境 首先是安装Ambari Server服务器,这边因为墙的原因需要自己配置本地源 : mkdir -p /var/www/html 将上述安装包解压到该目录下,通过Python服务器在该目录启动访问链接:( 以静默的方式启动Python服务器 nohup python -m SimpleHTTPServer 1>out.log 2>&1 &)
mkdir -p /var/www/html
ls /var/www/html
ambari HDP HDP-GPL HDP-UTILS
将安装包解压后放入上述目录
配置Ubuntu启动源
指定仓库地址:
https://www.jianshu.com/p/e33e6d9a0655
/etc/apt/sources.list.d cat /etc/apt/sources.lost.d/ambari.list
deb http://10.1.51.17:8000/ambari/ubuntu16/2.7.3.0-139/ Ambari main
etc/apt/sources.list.d/ambari-hdp.list
deb http://10.1.51.17:8000/HDP/ubuntu16/3.1.0.0-78/ HDP main
deb http://10.1.51.17:8000/HDP-GPL/ubuntu16/3.1.0.0-78/ HDP-GPL main
deb http://10.1.51.17:8000/HDP-UTILS/ubuntu16/1.1.0.22/ HDP-UTILS main apt-key adv --recv-keys --keyserver keyserver.ubuntu.com B9733A7A07513CAD
apt-get update
apt-cache showpkg ambari-server
最后在apt-server服务器上执行:
apt-get install ambari-server
可以顺利安装ambari-server;
之后执行ambari-server setup对其进行安装配置操作,包括设置jdk与存储数据库等配置
最后ambari-server start启动,启动后可以通过该机器的8080端口访问Ambari页面
当打开以下页面后,Ambari的安装流程就告一段落,可以先打一个镜像了;
当打开以上界面后证明Ambari Server已经可以成功启动了,后边现在需要利用它进行Ambari agent配置工作,同时包括大数据的一些服务组件配置:
首先在以上界面对该集群进行命名;
选择对应的HDP版本,同时配置一些源服务.这个地方如果不需要修改源码可以初略处理,后边我想要更换HDP源还需要进行一些替换操作;
填好需要安装Ambari agent服务器的hostname,同时下边贴上ambari server服务器的id_rsa文件,注意没有.pub后缀,否则在下一步安装的时候会提示ssh连接失败的错误.(如下图所示)
填好之后下一步就会进行Ambari agent安装工作了.
安装完成后选择自己所需要的服务组件,进行安装,注意要将服务分配均匀 同时中间如果有出错需要及时解决错误;
出坑1:
Ubuntu16.04安装Ambari2.6.2.0配置HDP2.5.5.0-292遇见大坑,安装Ambari Client有个坎一直过不去.
[code=csharp]
==========================
Creating target directory...
==========================
Command start time 2019-01-04 12:38:12
chmod: cannot access '/var/lib/ambari-agent/data': No such file or directory
Warning: Permanently added 'master.example.com,10.10.80.131' (ECDSA) to the list of known hosts.
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:12
==========================
Copying ambari sudo script...
==========================
Command start time 2019-01-04 12:38:12
scp /var/lib/ambari-server/ambari-sudo.sh
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying common functions script...
==========================
Command start time 2019-01-04 12:38:13
scp /usr/lib/ambari-server/lib/ambari_commons
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying create-python-wrap script...
==========================
Command start time 2019-01-04 12:38:13
scp /var/lib/ambari-server/create-python-wrap.sh
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying OS type check script...
==========================
Command start time 2019-01-04 12:38:13
scp /usr/lib/ambari-server/lib/ambari_server/os_check_type.py
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Running create-python-wrap script...
==========================
Command start time 2019-01-04 12:38:13
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Running OS type check...
==========================
Command start time 2019-01-04 12:38:14
Cluster primary/cluster OS family is ubuntu16 and local/current OS family is ubuntu16
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Checking 'sudo' package on remote host...
==========================
Command start time 2019-01-04 12:38:14
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Copying repo file to 'tmp' folder...
==========================
Command start time 2019-01-04 12:38:14
scp /etc/apt/sources.list.d/ambari.list
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Moving file to repo dir...
==========================
Command start time 2019-01-04 12:38:15
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Changing permissions for ambari.repo...
==========================
Command start time 2019-01-04 12:38:15
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Update apt cache of repository...
==========================
Command start time 2019-01-04 12:38:15
0% [Working]
Get:1 http://10.10.80.131:8000/ambari/ubuntu16/2.6.2.0-155 Ambari InRelease [7,394 B]
0% [1 InRelease 7,394 B/7,394 B 100%]
0% [Working]
Get:2 http://10.10.80.131:8000/HDP/ubuntu16/2.6.5.0-292 HDP InRelease [7,376 B]
Get:3 http://10.10.80.131:8000/HDP-GPL/ubuntu16/2.6.5.0-292 HDP-GPL InRelease [7,395 B]
Get:4 http://10.10.80.131:8000/HDP-UTILS/ubuntu16/1.1.0.22 HDP-UTILS InRelease [7,405 B]
0% [Working]
0% [1 InRelease gpgv 7,394 B]
0% [Working]
0% [2 InRelease gpgv 7,376 B]
0% [Working]
0% [3 InRelease gpgv 7,395 B]
0% [Working]
0% [4 InRelease gpgv 7,405 B]
100% [Working]
Fetched 29.6 kB in 0s (148 kB/s)
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 8%
Reading package lists... 19%
Reading package lists... 19%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... Done
W: http://10.10.80.131:8000/ambari/ubuntu16/2.6.2.0-155/dists/Ambari/InRelease: Signature by key DF52ED4F7A3A5882C0994C66B9733A7A07513CAD uses weak digest algorithm (SHA1)
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:16
==========================
Copying setup script file...
==========================
Command start time 2019-01-04 12:38:16
scp /usr/lib/ambari-server/lib/ambari_server/setupAgent.py
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:16
==========================
Running setup agent script...
==========================
Command start time 2019-01-04 12:38:16
dpkg-query: no packages found matching ambari-agent
('WARNING 2019-01-04 12:38:28,678 AlertSchedulerHandler.py:280 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs.
INFO 2019-01-04 12:38:28,678 AlertSchedulerHandler.py:175 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7f0e12cad490>; currently running: False
INFO 2019-01-04 12:38:30,685 hostname.py:106 - Read public hostname \'master.example.com\' using socket.getfqdn()
INFO 2019-01-04 12:38:30,686 Hardware.py:68 - Initializing host system information.
INFO 2019-01-04 12:38:30,737 Hardware.py:188 - Some mount points were ignored: /run, /dev/shm, /run/lock, /sys/fs/cgroup, /run/user/0
INFO 2019-01-04 12:38:30,748 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname \'master.example.com\' using socket.getfqdn().
INFO 2019-01-04 12:38:30,750 Facter.py:202 - Directory: \'/etc/resource_overrides\' does not exist - it won\'t be used for gathering system resources.
INFO 2019-01-04 12:38:30,752 Hardware.py:73 - Host system information: {\'kernel\': \'Linux\', \'domain\': \'example.com\', \'physicalprocessorcount\': 4, \'kernelrelease\': \'4.4.0-117-generic\', \'uptime_days\': \'0\', \'memorytotal\': 8174812, \'swapfree\': \'0.00 GB\', \'memorysize\': 8174812, \'osfamily\': \'ubuntu\', \'swapsize\': \'0.00 GB\', \'processorcount\': 4, \'netmask\': \'255.255.255.0\', \'timezone\': \'CST\', \'hardwareisa\': \'x86_64\', \'memoryfree\': 241164, \'operatingsystem\': \'ubuntu\', \'kernelmajversion\': \'4.4\', \'kernelversion\': \'4.4.0\', \'macaddress\': \'00:16:3E:10:7E:5C\', \'operatingsystemrelease\': \'16.04\', \'ipaddress\': \'10.10.80.131\', \'hostname\': \'master\', \'uptime_hours\': \'1\', \'fqdn\': \'master.example.com\', \'id\': \'root\', \'architecture\': \'x86_64\', \'selinux\': False, \'mounts\': [{\'available\': \'4068960\', \'used\': \'0\', \'percent\': \'0%\', \'device\': \'udev\', \'mountpoint\': \'/dev\', \'type\': \'devtmpfs\', \'size\': \'4068960\'}, {\'available\': \'24108320\', \'used\': \'14930060\', \'percent\': \'39%\', \'device\': \'/dev/vda1\', \'mountpoint\': \'/\', \'type\': \'ext4\', \'size\': \'41151808\'}, {\'available\': \'88747784\', \'used\': \'9072256\', \'percent\': \'10%\', \'device\': \'/dev/vdb1\', \'mountpoint\': \'/mnt/disk1\', \'type\': \'ext4\', \'size\': \'103079200\'}], \'hardwaremodel\': \'x86_64\', \'uptime_seconds\': \'7017\', \'interfaces\': \'eth0,lo\'}
INFO 2019-01-04 12:38:30,953 Controller.py:170 - Registering with master.example.com (10.10.80.131) (agent=\'{"hardwareProfile": {"kernel": "Linux", "domain": "example.com", "physicalprocessorcount": 4, "kernelrelease": "4.4.0-117-generic", "uptime_days": "0", "memorytotal": 8174812, "swapfree": "0.00 GB", "memorysize": 8174812, "osfamily": "ubuntu", "swapsize": "0.00 GB", "processorcount": 4, "netmask": "255.255.255.0", "timezone": "CST", "hardwareisa": "x86_64", "memoryfree": 241164, "operatingsystem": "ubuntu", "kernelmajversion": "4.4", "kernelversion": "4.4.0", "macaddress": "00:16:3E:10:7E:5C", "operatingsystemrelease": "16.04", "ipaddress": "10.10.80.131", "hostname": "master", "uptime_hours": "1", "fqdn": "master.example.com", "id": "root", "architecture": "x86_64", "selinux": false, "mounts": [{"available": "4068960", "used": "0", "percent": "0%", "device": "udev", "mountpoint": "/dev", "type": "devtmpfs", "size": "4068960"}, {"available": "24108320", "used": "14930060", "percent": "39%", "device": "/dev/vda1", "mountpoint": "/", "type": "ext4", "size": "41151808"}, {"available": "88747784", "used": "9072256", "percent": "10%", "device": "/dev/vdb1", "mountpoint": "/mnt/disk1", "type": "ext4", "size": "103079200"}], "hardwaremodel": "x86_64", "uptime_seconds": "7017", "interfaces": "eth0,lo"}, "currentPingPort": 8670, "prefix": "/var/lib/ambari-agent/data", "agentVersion": "2.6.2.0", "agentEnv": {"transparentHugePage": "madvise", "hostHealth": {"agentTimeStampAtReporting": 1546576710951, "activeJavaProcs": [], "liveServices": [{"status": "Healthy", "name": "ntp or chrony", "desc": ""}]}, "reverseLookup": true, "alternatives": [], "hasUnlimitedJcePolicy": null, "umask": "18", "firewallName": "ufw", "stackFoldersAndFiles": [], "existingUsers": [], "firewallRunning": false}, "timestamp": 1546576710753, "hostname": "master.example.com", "responseId": -1, "publicHostname": "master.example.com"}\')
INFO 2019-01-04 12:38:30,953 NetUtil.py:70 - Connecting to https://master.example.com:8440/connection_info
INFO 2019-01-04 12:38:30,981 security.py:93 - SSL Connect being called.. connecting to the server
ERROR 2019-01-04 12:38:30,984 Controller.py:226 - Unable to connect to: https://master.example.com:8441/agent/v1/register/master.example.com
Traceback (most recent call last):
File "/usr/lib/ambari-agent/lib/ambari_agent/Controller.py", line 175, in registerWithServer
ret = self.sendRequest(self.registerUrl, data)
File "/usr/lib/ambari-agent/lib/ambari_agent/Controller.py", line 549, in sendRequest
raise IOError(\'Request to {0} failed due to {1}\'.format(url, str(exception)))
IOError: Request to https://master.example.com:8441/agent/v1/register/master.example.com failed due to EOF occurred in violation of protocol (_ssl.c:590)
ERROR 2019-01-04 12:38:30,984 Controller.py:227 - Error:Request to https://master.example.com:8441/agent/v1/register/master.example.com failed due to EOF occurred in violation of protocol (_ssl.c:590)
WARNING 2019-01-04 12:38:30,984 Controller.py:228 - Sleeping for 27 seconds and then trying again
', None)
('WARNING 2019-01-04 12:38:28,678 AlertSchedulerHandler.py:280 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs.
INFO 2019-01-04 12:38:28,678 AlertSchedulerHandler.py:175 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7f0e12cad490>; currently running: False
INFO 2019-01-04 12:38:30,685 hostname.py:106 - Read public hostname \'master.example.com\' using socket.getfqdn()
INFO 2019-01-04 12:38:30,686 Hardware.py:68 - Initializing host system information.
INFO 2019-01-04 12:38:30,737 Hardware.py:188 - Some mount points were ignored: /run, /dev/shm, /run/lock, /sys/fs/cgroup, /run/user/0
INFO 2019-01-04 12:38:30,748 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname \'master.example.com\' using socket.getfqdn().
INFO 2019-01-04 12:38:30,750 Facter.py:202 - Directory: \'/etc/resource_overrides\' does not exist - it won\'t be used for gathering system resources.
INFO 2019-01-04 12:38:30,752 Hardware.py:73 - Host system information: {\'kernel\': \'Linux\', \'domain\': \'example.com\', \'physicalprocessorcount\': 4, \'kernelrelease\': \'4.4.0-117-generic\', \'uptime_days\': \'0\', \'memorytotal\': 8174812, \'swapfree\': \'0.00 GB\', \'memorysize\': 8174812, \'osfamily\': \'ubuntu\', \'swapsize\': \'0.00 GB\', \'processorcount\': 4, \'netmask\': \'255.255.255.0\', \'timezone\': \'CST\', \'hardwareisa\': \'x86_64\', \'memoryfree\': 241164, \'operatingsystem\': \'ubuntu\', \'kernelmajversion\': \'4.4\', \'kernelversion\': \'4.4.0\', \'macaddress\': \'00:16:3E:10:7E:5C\', \'operatingsystemrelease\': \'16.04\', \'ipaddress\': \'10.10.80.131\', \'hostname\': \'master\', \'uptime_hours\': \'1\', \'fqdn\': \'master.example.com\', \'id\': \'root\', \'architecture\': \'x86_64\', \'selinux\': False, \'mounts\': [{\'available\': \'4068960\', \'used\': \'0\', \'percent\': \'0%\', \'device\': \'udev\', \'mountpoint\': \'/dev\', \'type\': \'devtmpfs\', \'size\': \'4068960\'}, {\'available\': \'24108320\', \'used\': \'14930060\', \'percent\': \'39%\', \'device\': \'/dev/vda1\', \'mountpoint\': \'/\', \'type\': \'ext4\', \'size\': \'41151808\'}, {\'available\': \'88747784\', \'used\': \'9072256\', \'percent\': \'10%\', \'device\': \'/dev/vdb1\', \'mountpoint\': \'/mnt/disk1\', \'type\': \'ext4\', \'size\': \'103079200\'}], \'hardwaremodel\': \'x86_64\', \'uptime_seconds\': \'7017\', \'interfaces\': \'eth0,lo\'}
INFO 2019-01-04 12:38:30,953 Controller.py:170 - Registering with master.example.com (10.10.80.131) (agent=\'{"hardwareProfile": {"kernel": "Linux", "domain": "example.com", "physicalprocessorcount": 4, "kernelrelease": "4.4.0-117-generic", "uptime_days": "0", "memorytotal": 8174812, "swapfree": "0.00 GB", "memorysize": 8174812, "osfamily": "ubuntu", "swapsize": "0.00 GB", "processorcount": 4, "netmask": "255.255.255.0", "timezone": "CST", "hardwareisa": "x86_64", "memoryfree": 241164, "operatingsystem": "ubuntu", "kernelmajversion": "4.4", "kernelversion": "4.4.0", "macaddress": "00:16:3E:10:7E:5C", "operatingsystemrelease": "16.04", "ipaddress": "10.10.80.131", "hostname": "master", "uptime_hours": "1", "fqdn": "master.example.com", "id": "root", "architecture": "x86_64", "selinux": false, "mounts": [{"available": "4068960", "used": "0", "percent": "0%", "device": "udev", "mountpoint": "/dev", "type": "devtmpfs", "size": "4068960"}, {"available": "24108320", "used": "14930060", "percent": "39%", "device": "/dev/vda1", "mountpoint": "/", "type": "ext4", "size": "41151808"}, {"available": "88747784", "used": "9072256", "percent": "10%", "device": "/dev/vdb1", "mountpoint": "/mnt/disk1", "type": "ext4", "size": "103079200"}], "hardwaremodel": "x86_64", "uptime_seconds": "7017", "interfaces": "eth0,lo"}, "currentPingPort": 8670, "prefix": "/var/lib/ambari-agent/data", "agentVersion": "2.6.2.0", "agentEnv": {"transparentHugePage": "madvise", "hostHealth": {"agentTimeStampAtReporting": 1546576710951, "activeJavaProcs": [], "liveServices": [{"status": "Healthy", "name": "ntp or chrony", "desc": ""}]}, "reverseLookup": true, "alternatives": [], "hasUnlimitedJcePolicy": null, "umask": "18", "firewallName": "ufw", "stackFoldersAndFiles": [], "existingUsers": [], "firewallRunning": false}, "timestamp": 1546576710753, "hostname": "master.example.com", "responseId": -1, "publicHostname": "master.example.com"}\')
INFO 2019-01-04 12:38:30,953 NetUtil.py:70 - Connecting to https://master.example.com:8440/connection_info
INFO 2019-01-04 12:38:30,981 security.py:93 - SSL Connect being called.. connecting to the server
ERROR 2019-01-04 12:38:30,984 Controller.py:226 - Unable to connect to: https://master.example.com:8441/agent/v1/register/master.example.com
Traceback (most recent call last):
File "/usr/lib/ambari-agent/lib/ambari_agent/Controller.py", line 175, in registerWithServer
ret = self.sendRequest(self.registerUrl, data)
File "/usr/lib/ambari-agent/lib/ambari_agent/Controller.py", line 549, in sendRequest
raise IOError(\'Request to {0} failed due to {1}\'.format(url, str(exception)))
IOError: Request to https://master.example.com:8441/agent/v1/register/master.example.com failed due to EOF occurred in violation of protocol (_ssl.c:590)
ERROR 2019-01-04 12:38:30,984 Controller.py:227 - Error:Request to https://master.example.com:8441/agent/v1/register/master.example.com failed due to EOF occurred in violation of protocol (_ssl.c:590)
WARNING 2019-01-04 12:38:30,984 Controller.py:228 - Sleeping for 27 seconds and then trying again
', None)
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:31
Registering with the server...
Registration with the server failed.
==========================
Creating target directory...
==========================
Command start time 2019-01-04 12:38:12
chmod: cannot access '/var/lib/ambari-agent/data': No such file or directory
Warning: Permanently added 'node1.example.com,10.10.80.130' (ECDSA) to the list of known hosts.
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:12
==========================
Copying ambari sudo script...
==========================
Command start time 2019-01-04 12:38:12
scp /var/lib/ambari-server/ambari-sudo.sh
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying common functions script...
==========================
Command start time 2019-01-04 12:38:13
scp /usr/lib/ambari-server/lib/ambari_commons
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying create-python-wrap script...
==========================
Command start time 2019-01-04 12:38:13
scp /var/lib/ambari-server/create-python-wrap.sh
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying OS type check script...
==========================
Command start time 2019-01-04 12:38:13
scp /usr/lib/ambari-server/lib/ambari_server/os_check_type.py
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Running create-python-wrap script...
==========================
Command start time 2019-01-04 12:38:14
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Running OS type check...
==========================
Command start time 2019-01-04 12:38:14
Cluster primary/cluster OS family is ubuntu16 and local/current OS family is ubuntu16
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Checking 'sudo' package on remote host...
==========================
Command start time 2019-01-04 12:38:14
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Copying repo file to 'tmp' folder...
==========================
Command start time 2019-01-04 12:38:14
scp /etc/apt/sources.list.d/ambari.list
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Moving file to repo dir...
==========================
Command start time 2019-01-04 12:38:15
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Changing permissions for ambari.repo...
==========================
Command start time 2019-01-04 12:38:15
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Update apt cache of repository...
==========================
Command start time 2019-01-04 12:38:15
0% [Working]
Get:1 http://10.10.80.131:8000/ambari/ubuntu16/2.6.2.0-155 Ambari InRelease [7,394 B]
0% [1 InRelease 7,394 B/7,394 B 100%]
0% [Working]
Get:2 http://10.10.80.131:8000/HDP/ubuntu16/2.6.5.0-292 HDP InRelease [7,376 B]
Get:3 http://10.10.80.131:8000/HDP-GPL/ubuntu16/2.6.5.0-292 HDP-GPL InRelease [7,395 B]
Get:4 http://10.10.80.131:8000/HDP-UTILS/ubuntu16/1.1.0.22 HDP-UTILS InRelease [7,405 B]
0% [Working]
0% [1 InRelease gpgv 7,394 B]
0% [Working]
0% [2 InRelease gpgv 7,376 B]
0% [Working]
0% [3 InRelease gpgv 7,395 B]
0% [Working]
0% [4 InRelease gpgv 7,405 B]
100% [Working]
Fetched 29.6 kB in 0s (158 kB/s)
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 8%
Reading package lists... 19%
Reading package lists... 19%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... Done
W: http://10.10.80.131:8000/ambari/ubuntu16/2.6.2.0-155/dists/Ambari/InRelease: Signature by key DF52ED4F7A3A5882C0994C66B9733A7A07513CAD uses weak digest algorithm (SHA1)
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:16
==========================
Copying setup script file...
==========================
Command start time 2019-01-04 12:38:16
scp /usr/lib/ambari-server/lib/ambari_server/setupAgent.py
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:16
==========================
Running setup agent script...
==========================
Command start time 2019-01-04 12:38:16
dpkg-query: no packages found matching ambari-agent
("INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,808 DataCleaner.py:39 - Data cleanup thread started
INFO 2019-01-04 12:38:25,809 DataCleaner.py:120 - Data cleanup started
INFO 2019-01-04 12:38:25,809 DataCleaner.py:122 - Data cleanup finished
INFO 2019-01-04 12:38:25,809 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname 'node1.example.com' using socket.getfqdn().
INFO 2019-01-04 12:38:25,834 PingPortListener.py:50 - Ping port listener started on port: 8670
INFO 2019-01-04 12:38:25,834 main.py:439 - Connecting to Ambari server at https://master.example.com:8440 (10.10.80.131)
INFO 2019-01-04 12:38:25,834 NetUtil.py:70 - Connecting to https://master.example.com:8440/ca
INFO 2019-01-04 12:38:26,256 main.py:449 - Connected to Ambari server master.example.com
INFO 2019-01-04 12:38:26,257 threadpool.py:58 - Started thread pool with 3 core threads and 20 maximum threads
WARNING 2019-01-04 12:38:26,257 AlertSchedulerHandler.py:280 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs.
INFO 2019-01-04 12:38:26,257 AlertSchedulerHandler.py:175 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7ff4bfa91490>; currently running: False
INFO 2019-01-04 12:38:28,265 hostname.py:106 - Read public hostname 'node1.example.com' using socket.getfqdn()
INFO 2019-01-04 12:38:28,266 Hardware.py:68 - Initializing host system information.
INFO 2019-01-04 12:38:28,310 Hardware.py:188 - Some mount points were ignored: /run, /dev/shm, /run/lock, /sys/fs/cgroup, /run/user/0
INFO 2019-01-04 12:38:28,319 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname 'node1.example.com' using socket.getfqdn().
INFO 2019-01-04 12:38:28,321 Facter.py:202 - Directory: '/etc/resource_overrides' does not exist - it won't be used for gathering system resources.
INFO 2019-01-04 12:38:28,323 Hardware.py:73 - Host system information: {'kernel': 'Linux', 'domain': 'example.com', 'physicalprocessorcount': 4, 'kernelrelease': '4.4.0-117-generic', 'uptime_days': '0', 'memorytotal': 4046044, 'swapfree': '0.00 GB', 'memorysize': 4046044, 'osfamily': 'ubuntu', 'swapsize': '0.00 GB', 'processorcount': 4, 'netmask': '255.255.255.0', 'timezone': 'CST', 'hardwareisa': 'x86_64', 'memoryfree': 1011008, 'operatingsystem': 'ubuntu', 'kernelmajversion': '4.4', 'kernelversion': '4.4.0', 'macaddress': '00:16:3E:0E:F0:53', 'operatingsystemrelease': '16.04', 'ipaddress': '10.10.80.130', 'hostname': 'node1', 'uptime_hours': '1', 'fqdn': 'node1.example.com', 'id': 'root', 'architecture': 'x86_64', 'selinux': False, 'mounts': [{'available': '2004576', 'used': '0', 'percent': '0%', 'device': 'udev', 'mountpoint': '/dev', 'type': 'devtmpfs', 'size': '2004576'}, {'available': '35189292', 'used': '3849088', 'percent': '10%', 'device': '/dev/vda1', 'mountpoint': '/', 'type': 'ext4', 'size': '41151808'}, {'available': '97758996', 'used': '61044', 'percent': '1%', 'device': '/dev/vdb1', 'mountpoint': '/mnt/disk1', 'type': 'ext4', 'size': '103079200'}], 'hardwaremodel': 'x86_64', 'uptime_seconds': '7180', 'interfaces': 'eth0,lo'}
", None)
("INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,808 DataCleaner.py:39 - Data cleanup thread started
INFO 2019-01-04 12:38:25,809 DataCleaner.py:120 - Data cleanup started
INFO 2019-01-04 12:38:25,809 DataCleaner.py:122 - Data cleanup finished
INFO 2019-01-04 12:38:25,809 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname 'node1.example.com' using socket.getfqdn().
INFO 2019-01-04 12:38:25,834 PingPortListener.py:50 - Ping port listener started on port: 8670
INFO 2019-01-04 12:38:25,834 main.py:439 - Connecting to Ambari server at https://master.example.com:8440 (10.10.80.131)
INFO 2019-01-04 12:38:25,834 NetUtil.py:70 - Connecting to https://master.example.com:8440/ca
INFO 2019-01-04 12:38:26,256 main.py:449 - Connected to Ambari server master.example.com
INFO 2019-01-04 12:38:26,257 threadpool.py:58 - Started thread pool with 3 core threads and 20 maximum threads
WARNING 2019-01-04 12:38:26,257 AlertSchedulerHandler.py:280 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs.
INFO 2019-01-04 12:38:26,257 AlertSchedulerHandler.py:175 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7ff4bfa91490>; currently running: False
INFO 2019-01-04 12:38:28,265 hostname.py:106 - Read public hostname 'node1.example.com' using socket.getfqdn()
INFO 2019-01-04 12:38:28,266 Hardware.py:68 - Initializing host system information.
INFO 2019-01-04 12:38:28,310 Hardware.py:188 - Some mount points were ignored: /run, /dev/shm, /run/lock, /sys/fs/cgroup, /run/user/0
INFO 2019-01-04 12:38:28,319 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname 'node1.example.com' using socket.getfqdn().
INFO 2019-01-04 12:38:28,321 Facter.py:202 - Directory: '/etc/resource_overrides' does not exist - it won't be used for gathering system resources.
INFO 2019-01-04 12:38:28,323 Hardware.py:73 - Host system information: {'kernel': 'Linux', 'domain': 'example.com', 'physicalprocessorcount': 4, 'kernelrelease': '4.4.0-117-generic', 'uptime_days': '0', 'memorytotal': 4046044, 'swapfree': '0.00 GB', 'memorysize': 4046044, 'osfamily': 'ubuntu', 'swapsize': '0.00 GB', 'processorcount': 4, 'netmask': '255.255.255.0', 'timezone': 'CST', 'hardwareisa': 'x86_64', 'memoryfree': 1011008, 'operatingsystem': 'ubuntu', 'kernelmajversion': '4.4', 'kernelversion': '4.4.0', 'macaddress': '00:16:3E:0E:F0:53', 'operatingsystemrelease': '16.04', 'ipaddress': '10.10.80.130', 'hostname': 'node1', 'uptime_hours': '1', 'fqdn': 'node1.example.com', 'id': 'root', 'architecture': 'x86_64', 'selinux': False, 'mounts': [{'available': '2004576', 'used': '0', 'percent': '0%', 'device': 'udev', 'mountpoint': '/dev', 'type': 'devtmpfs', 'size': '2004576'}, {'available': '35189292', 'used': '3849088', 'percent': '10%', 'device': '/dev/vda1', 'mountpoint': '/', 'type': 'ext4', 'size': '41151808'}, {'available': '97758996', 'used': '61044', 'percent': '1%', 'device': '/dev/vdb1', 'mountpoint': '/mnt/disk1', 'type': 'ext4', 'size': '103079200'}], 'hardwaremodel': 'x86_64', 'uptime_seconds': '7180', 'interfaces': 'eth0,lo'}
", None)
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:28
Registering with the server...
Registration with the server failed.
[/code]
解决方案(谷歌比百度好用)
https://community.hortonworks.com/questions/97831/registration-to-ambari-server-via-the-ambari-agent.html
/etc/ambari-agent/conf修改ambari.ini文件
We have over come the problem by adding
following option to security section in ambari-agent.ini in all the hosts in the cluster:
[security] force_https_protocol=PROTOCOL_TLSv1_2
Ubuntu16.04安装Ambari 2.7.3的更多相关文章
- ubuntu16.04安装jdk，tomcat
ubuntu16.04安装jdk,tomcat 最近装了一下tomcat,网上的教程很多,我也试了很多次,但是有一些教程关于tomcat配置是错误的,让我走上了歧途.差点重装系统,还好王总及时出手帮助 ...
- Ubuntu16.04 安装openjdk-7-jdk
Ubuntu16.04 安装openjdk-7-jdk sudo apt-get install openjdk-7-jre 或者sudo apt-get install openjdk-7-jdk ...
- Ubuntu16.04安装GTK3主题:OSX-Arc
Ubuntu16.04安装GTK3主题:OSX-Arc GTK3主题:OSX-Arc描述: 前几个月,Gnome3.20升3.22的时候,出现了大量主题崩溃的现象,其中包括Arc.Flatabulou ...
- Ubuntu16.04安装opencv for python/c++
Ubuntu16.04安装opencv for python/c++ 网上关于opencv的安装已经有了不少资料,但是没有一篇资料能让我一次性安装成功,因此花费了大量时间去解决各种意外,希望这篇能给一 ...
- ubuntu16.04安装不上有道词典的解决办法
转自:http://www.linuxdiyf.com/linux/21143.html ubuntu16.04安装不上有道词典,提示: le@hu-pc:~/下载$ sudo dpkg -i you ...
- Ubuntu16.04安装mongodb
Ubuntu16.04安装mongodb copy from: http://blog.csdn.net/zhushh/article/details/52451441 1.导入软件源的公钥 sudo ...
- 【Tools】ubuntu16.04安装搜狗输入法
Ubuntu16,04 安装搜狗输入法 1.下载搜狗输入法的安装包 下载地址为:http://pinyin.sogou.com/linux/ 2.按键Ctr+Alt+T打开终端,输入以下命令切换到下载 ...
- Ubuntu16.04安装cuda9.0+cudnn7.0
Ubuntu16.04安装cuda9.0+cudnn7.0 这篇记录拖了好久,估计是去年6月份就已经安装过几遍,然后一方面因为俺比较懒,一方面后面没有经常在自己电脑上跑算法,比较少装cuda和cudn ...
- Ubuntu16.04安装TensorFlow及Mnist训练
版权声明:本文为博主原创文章,欢迎转载,并请注明出处.联系方式:460356155@qq.com TensorFlow是Google开发的开源的深度学习框架,也是当前使用最广泛的深度学习框架. 一.安 ...
随机推荐
- echarts 设置数据加载遮罩层
//显示加载数据的loading chart.showLoading({ text: "图表数据正在努力加载...", x ...
- ubuntu16中部署web项目到tomcat，xft和securecrt连接到ubuntu16(待续。。。)
xftp xftp中新建连接 ubuntu中安装和启动ssh服务 xftp连接到ubuntu 安装JDK,Tomcat sudo tar -zvxf jdk.tar.gz ...
- 用spring的@Scheduled实现定时任务
先在spring的配置文件中添加扫描 在applicationContext.xml中添加 <task:annotation-driven/>,我用的是idea有提示功能 选择第一个后会 ...
- 安装 mongo 4.0
Centos 使用yum安装MongoDB 4.0 1.配置MongoDB的yum源 创建yum源文件: #touch /etc/yum.repos.d/mongodb-org-4.0.repo 添加 ...
- docker挂载目录的深入研究
转载一篇很好的文章: https://www.cnblogs.com/ivictor/p/4834864.html
- python之全局变量与局部变量
全局变量: - 在书写中顶格开始: - 一旦定义完毕在整个文件生效: - 在函数内如果定义了同名全局变量名,会“覆盖”掉全局变量: - 在函数中同名的变量,当在函数退出后消失,全局的同 ...
- Vue父子组件生命过程
加载渲染过程 父beforeCreate->父created->父beforeMount->子beforeCreate->子created->子beforeMount-& ...
- Mac终端中输入ps aux显示全部进程
ps命令是Process Status的缩写. ps aux命令用来列出系统中当前运行的那些进程. ps aux | grep chrome 表示查询关于chrome的所有程序(grep可作为文件内的 ...
- python字符串处理内置方法一览表
python字符串处理内置方法一览表 序号 方法及描述 1 capitalize()将字符串的第一个字符转换为大写 2 center(width, fillchar) 返回一个指定的宽度 widt ...
- Spring 事物Transaction
日常开发中Spring 为我们提供了两种事物的定义方式 XML 配置 方式 :这种方式配置起来比较麻烦,但后期比较好进行维护 注解方式:配置起来比较方便,也是日常开发常用的: 我们这里进行第二种注解的 ...