CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager
CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.搭建CM私有仓库
详情请参考我的笔记： https://www.cnblogs.com/yinzhengjie/p/10412455.html。
1>.安装web服务器并启动
[root@node105 ~]# yum -y install httpd
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* extras: mirrors.tuna.tsinghua.edu.cn
* updates: mirrors.tuna.tsinghua.edu.cn
base | 3.6 kB ::
extras | 3.4 kB ::
updates | 3.4 kB ::
(/): extras//x86_64/primary_db | kB ::
(/): updates//x86_64/primary_db | 2.4 MB ::
Resolving Dependencies
--> Running transaction check
---> Package httpd.x86_64 :2.4.-.el7.centos will be installed
--> Processing Dependency: httpd-tools = 2.4.-.el7.centos for package: httpd-2.4.-.el7.centos.x86_64
--> Processing Dependency: /etc/mime.types for package: httpd-2.4.-.el7.centos.x86_64
--> Processing Dependency: libaprutil-.so.()(64bit) for package: httpd-2.4.-.el7.centos.x86_64
--> Processing Dependency: libapr-.so.()(64bit) for package: httpd-2.4.-.el7.centos.x86_64
--> Running transaction check
---> Package apr.x86_64 :1.4.-.el7_4. will be installed
---> Package apr-util.x86_64 :1.5.-.el7 will be installed
---> Package httpd-tools.x86_64 :2.4.-.el7.centos will be installed
---> Package mailcap.noarch :2.1.-.el7 will be installed
--> Finished Dependency Resolution Dependencies Resolved ========================================================================================================================================================================
Package Arch Version Repository Size
========================================================================================================================================================================
Installing:
httpd x86_64 2.4.-.el7.centos base 2.7 M
Installing for dependencies:
apr x86_64 1.4.-.el7_4. base k
apr-util x86_64 1.5.-.el7 base k
httpd-tools x86_64 2.4.-.el7.centos base k
mailcap noarch 2.1.-.el7 base k Transaction Summary
========================================================================================================================================================================
Install Package (+ Dependent packages) Total download size: 3.0 M
Installed size: M
Downloading packages:
(/): mailcap-2.1.-.el7.noarch.rpm | kB ::
(/): apr-1.4.-.el7_4..x86_64.rpm | kB ::
(/): apr-util-1.5.-.el7.x86_64.rpm | kB ::
(/): httpd-tools-2.4.-.el7.centos.x86_64.rpm | kB ::
(/): httpd-2.4.-.el7.centos.x86_64.rpm | 2.7 MB ::
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total kB/s | 3.0 MB ::
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : apr-1.4.-.el7_4..x86_64 /
Installing : apr-util-1.5.-.el7.x86_64 /
Installing : httpd-tools-2.4.-.el7.centos.x86_64 /
Installing : mailcap-2.1.-.el7.noarch /
Installing : httpd-2.4.-.el7.centos.x86_64 /
Verifying : httpd-tools-2.4.-.el7.centos.x86_64 /
Verifying : apr-1.4.-.el7_4..x86_64 /
Verifying : mailcap-2.1.-.el7.noarch /
Verifying : httpd-2.4.-.el7.centos.x86_64 /
Verifying : apr-util-1.5.-.el7.x86_64 / Installed:
httpd.x86_64 :2.4.-.el7.centos Dependency Installed:
apr.x86_64 :1.4.-.el7_4. apr-util.x86_64 :1.5.-.el7 httpd-tools.x86_64 :2.4.-.el7.centos mailcap.noarch :2.1.-.el7 Complete!
[root@node105 ~]#
[root@node105 ~]# yum -y install httpd #安装httpd服务
[root@node105 ~]# cp /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf.`date +%F`
[root@node105 ~]#
[root@node105 ~]# grep ".tgz" /etc/httpd/conf/httpd.conf | grep -v ' #'
AddType application/x-gzip .gz .tgz
[root@node105 ~]#
[root@node105 ~]# sed -i s'#.tgz#.tgz .parcel#' /etc/httpd/conf/httpd.conf
[root@node105 ~]#
[root@node105 ~]# grep ".tgz" /etc/httpd/conf/httpd.conf | grep -v ' #'
AddType application/x-gzip .gz .tgz .parcel #其实我就是在.tgz后面加了一个“.parcel”的后缀，这是CM官方强调要做的一个操作！
[root@node105 ~]#
[root@node105 ~]#
[root@node105 ~]# systemctl enable httpd
Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.
[root@node105 ~]#
[root@node105 ~]# systemctl start httpd
[root@node105 ~]#
[root@node105 ~]#
[root@node105 ~]# systemctl status httpd
● httpd.service - The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
Active: active (running) since Fri -- :: CST; 3s ago
Docs: man:httpd()
man:apachectl()
Main PID: (httpd)
Status: "Processing requests..."
CGroup: /system.slice/httpd.service
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
└─ /usr/sbin/httpd -DFOREGROUND Feb :: node105.yinzhengjie.org.cn systemd[]: Starting The Apache HTTP Server...
Feb :: node105.yinzhengjie.org.cn systemd[]: Started The Apache HTTP Server.
[root@node105 ~]#
[root@node105 ~]#
[root@node105 ~]# systemctl start httpd #启动httpd服务
2>.下载CM对应到版本到对应到自建到web路径中（https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.16.1/RPMS/x86_64/）
[root@node105 ~]# mkdir /var/www/html/cm-5.16. #创建CM安装包存放目录
[root@node105 ~]#
[root@node105 ~]# ll /var/www/html/cm-5.16./ #去官方网站把你需要的rpm包都下载到你到安装目录即可！
total
-rwxr-xr-x root root Feb : cloudera-manager-agent-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-server-db--5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : jdk-8u201-linux-x64.rpm #这个软件是我从Oracle官方上download下来的，CM默认的JDK是6或者7，而Oracle官方已经不支持使用JDK7啦！
-rw-r--r-- 1 root root 1690 Feb 21 2018 RPM-GPG-KEY-cloudera
[root@node105 ~]#
[root@node105 ~]#
3>.制作本地镜像
[root@node105 ~]# yum -y install yum-utils createrepo yum-plugin-priorities
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* extras: mirrors.tuna.tsinghua.edu.cn
* updates: mirrors.tuna.tsinghua.edu.cn
Resolving Dependencies
--> Running transaction check
---> Package createrepo.noarch :0.9.-.el7 will be installed
--> Processing Dependency: python-deltarpm for package: createrepo-0.9.-.el7.noarch
--> Processing Dependency: libxml2-python for package: createrepo-0.9.-.el7.noarch
--> Processing Dependency: deltarpm for package: createrepo-0.9.-.el7.noarch
---> Package yum-plugin-priorities.noarch :1.1.-.el7 will be installed
---> Package yum-utils.noarch :1.1.-.el7 will be installed
--> Processing Dependency: python-kitchen for package: yum-utils-1.1.-.el7.noarch
--> Running transaction check
---> Package deltarpm.x86_64 :3.6-.el7 will be installed
---> Package libxml2-python.x86_64 :2.9.-.el7_2. will be installed
---> Package python-deltarpm.x86_64 :3.6-.el7 will be installed
---> Package python-kitchen.noarch :1.1.-.el7 will be installed
--> Processing Dependency: python-chardet for package: python-kitchen-1.1.-.el7.noarch
--> Running transaction check
---> Package python-chardet.noarch :2.2.-.el7_1 will be installed
--> Finished Dependency Resolution Dependencies Resolved ========================================================================================================================================================================
Package Arch Version Repository Size
========================================================================================================================================================================
Installing:
createrepo noarch 0.9.-.el7 base k
yum-plugin-priorities noarch 1.1.-.el7 base k
yum-utils noarch 1.1.-.el7 base k
Installing for dependencies:
deltarpm x86_64 3.6-.el7 base k
libxml2-python x86_64 2.9.-.el7_2. base k
python-chardet noarch 2.2.-.el7_1 base k
python-deltarpm x86_64 3.6-.el7 base k
python-kitchen noarch 1.1.-.el7 base k Transaction Summary
========================================================================================================================================================================
Install Packages (+ Dependent packages) Total download size: 1.1 M
Installed size: 4.8 M
Downloading packages:
(/): libxml2-python-2.9.-.el7_2..x86_64.rpm | kB ::
(/): createrepo-0.9.-.el7.noarch.rpm | kB ::
(/): python-deltarpm-3.6-.el7.x86_64.rpm | kB ::
(/): deltarpm-3.6-.el7.x86_64.rpm | kB ::
(/): yum-utils-1.1.-.el7.noarch.rpm | kB ::
(/): python-chardet-2.2.-.el7_1.noarch.rpm | kB ::
(/): yum-plugin-priorities-1.1.-.el7.noarch.rpm | kB ::
(/): python-kitchen-1.1.-.el7.noarch.rpm | kB ::
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total kB/s | 1.1 MB ::
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : deltarpm-3.6-.el7.x86_64 /
Installing : libxml2-python-2.9.-.el7_2..x86_64 /
Installing : python-deltarpm-3.6-.el7.x86_64 /
Installing : python-chardet-2.2.-.el7_1.noarch /
Installing : python-kitchen-1.1.-.el7.noarch /
Installing : yum-utils-1.1.-.el7.noarch /
Installing : createrepo-0.9.-.el7.noarch /
Installing : yum-plugin-priorities-1.1.-.el7.noarch /
Verifying : yum-utils-1.1.-.el7.noarch /
Verifying : python-deltarpm-3.6-.el7.x86_64 /
Verifying : yum-plugin-priorities-1.1.-.el7.noarch /
Verifying : libxml2-python-2.9.-.el7_2..x86_64 /
Verifying : deltarpm-3.6-.el7.x86_64 /
Verifying : createrepo-0.9.-.el7.noarch /
Verifying : python-chardet-2.2.-.el7_1.noarch /
Verifying : python-kitchen-1.1.-.el7.noarch / Installed:
createrepo.noarch :0.9.-.el7 yum-plugin-priorities.noarch :1.1.-.el7 yum-utils.noarch :1.1.-.el7 Dependency Installed:
deltarpm.x86_64 :3.6-.el7 libxml2-python.x86_64 :2.9.-.el7_2. python-chardet.noarch :2.2.-.el7_1 python-deltarpm.x86_64 :3.6-.el7
python-kitchen.noarch :1.1.-.el7 Complete!
[root@node105 ~]#
[root@node105 ~]# yum -y install yum-utils createrepo yum-plugin-priorities #安装本地源制作相关工具
[root@node105 ~]# cd /var/www/html/cm-5.16./
[root@node105 cm-5.16.]#
[root@node105 cm-5.16.]# ll
total
-rwxr-xr-x root root Feb : cloudera-manager-agent-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-server-db--5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : jdk-8u201-linux-x64.rpm
-rw-r--r-- root root Feb RPM-GPG-KEY-cloudera
[root@node105 cm-5.16.]#
[root@node105 cm-5.16.]#
[root@node105 cm-5.16.]# createrepo . #创建yum源的元数据文件！
Spawning worker with pkgs
Spawning worker with pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
[root@node105 cm-5.16.]#
[root@node105 cm-5.16.]# ll
total
-rwxr-xr-x root root Feb : cloudera-manager-agent-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-server-db--5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : jdk-8u201-linux-x64.rpm
drwxr-xr-x root root Feb : repodata #这个目录就是我们执行上面的命令后生成的目录,该文件不可被删除！否则你配置的yum的使用者依旧无法使用yum命令进行安装
-rw-r--r-- root root Feb RPM-GPG-KEY-cloudera [root@node105 cm-5.16.]#
[root@node105 cm-5.16.]#
4>.下载CDH的版本存放到指定路径（https://archive.cloudera.com/cdh5/parcels/5.15.1/）
[root@node105 ~]# mkdir /var/www/html/cdh-5.15.
[root@node105 ~]#
[root@node105 ~]# cd /var/www/html/cdh-5.15./
[root@node105 cdh-5.15.]#
[root@node105 cdh-5.15.]# ll
total
-rw-r--r-- root root Feb : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel
-rw-r--r-- root root Feb : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha
-rw-r--r-- root root Feb : manifest.json
[root@node105 cdh-5.15.]#
5>.各个主机配置CM自建的yum仓库
[root@node101 ~]# cat /etc/yum.repos.d/cloudera-manager.repo
[cloudera-manager]
# Packages for Cloudera Manager, Version , on RedHat or CentOS x86_64
name=Cloudera Manager
baseurl=http://node105.yinzhengjie.org.cn/cm-5.16.1/
gpgkey =http://node105.yinzhengjie.org.cn/cm-5.16.1//RPM-GPG-KEY-cloudera
enabled =
gpgcheck =
[root@node101 ~]#
二.安装和配置元数据库
详情请参考我的笔记：安装和配置元数据库。
三.Cloudera Manager 部署
安装服务前请确保你磁盘的可用空间大于50G！
1>.安装Cloudera Manager Server端（由于我们使用自建的镜像地址安装，而且也配置好了yum源，我们这里直接安装即可！）
[root@node101 ~]# yum -y install cloudera-manager-daemons cloudera-manager-server
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors..com
* extras: mirror01.idc.hinet.net
* updates: centos.cs.nctu.edu.tw
Resolving Dependencies
--> Running transaction check
---> Package cloudera-manager-daemons.x86_64 :5.16.-.cm5161.p0..el7 will be installed
---> Package cloudera-manager-server.x86_64 :5.16.-.cm5161.p0..el7 will be installed
--> Finished Dependency Resolution Dependencies Resolved ========================================================================================================================================================================
Package Arch Version Repository Size
========================================================================================================================================================================
Installing:
cloudera-manager-daemons x86_64 5.16.-.cm5161.p0..el7 cloudera-manager M
cloudera-manager-server x86_64 5.16.-.cm5161.p0..el7 cloudera-manager 8.5 k Transaction Summary
========================================================================================================================================================================
Install Packages Total download size: M
Installed size: M
Downloading packages:
(/): cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64.rpm | 8.5 kB ::
(/): cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64.rpm | MB ::
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total MB/s | MB ::
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64 /
Installing : cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64 /
Verifying : cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64 /
Verifying : cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64 / Installed:
cloudera-manager-daemons.x86_64 :5.16.-.cm5161.p0..el7 cloudera-manager-server.x86_64 :5.16.-.cm5161.p0..el7 Complete!
[root@node101 ~]#
[root@node101 ~]# yum -y install cloudera-manager-daemons cloudera-manager-server
2>.配置Cloudera Manager Server的默认堆内存大小
[root@node101 ~]# grep CMF_JAVA_OPTS /etc/default/cloudera-scm-server #Cloudera Manager默认分配的JVM堆内存是2G，某些情况下会出现OOM的情况，我们可以适当将其调大。
export CMF_JAVA_OPTS="-Xmx2G -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp"
[root@node101 ~]#
[root@node101 ~]# sed -i s'#Xmx2G#Xmx4G#' /etc/default/cloudera-scm-server #我们将Cloudera Manager的堆内存调大到4G。
[root@node101 ~]#
[root@node101 ~]# grep CMF_JAVA_OPTS /etc/default/cloudera-scm-server
export CMF_JAVA_OPTS="-Xmx4G -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp"
[root@node101 ~]#
[root@node101 ~]#
3>.初始化数据库配置
Cloudera官网已经提供了一个sh脚本用户配置Cloudera Manager连接元数据库，官方文档参考：
我们的安装规划下，元数据库和Cloudera Manager运行于同一个节点，且已经预先创建了scm库和用户，因此，命令很简单，具体操作如下：
[root@node101 ~]# /usr/share/cmf/schema/scm_prepare_database.sh mysql scm scm yinzhengjie
JAVA_HOME=/usr/java/jdk1.
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing: /usr/java/jdk1./bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
[ main] DbCommandExecutor INFO Successfully connected to database.
All done, your SCM database is configured correctly! #注意，如果你看到了这一行提示，说明配置完成，否则需要根据提示解决相应的错误！
[root@node101 ~]#
4>.启动Cloudera Manager Server服务
[root@node101 ~]# systemctl status cloudera-scm-server
● cloudera-scm-server.service - LSB: Cloudera SCM Server
Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled)
Active: inactive (dead)
Docs: man:systemd-sysv-generator() Feb :: node101.yinzhengjie.org.cn systemd[]: [/run/systemd/generator.late/cloudera-scm-server.service:] Failed to add dependency on +cloudera-scm-server-db.service, ignoring: Invalid argument
Feb :: node101.yinzhengjie.org.cn systemd[]: [/run/systemd/generator.late/cloudera-scm-server.service:] Failed to add dependency on +cloudera-scm-server-db.service, ignoring: Invalid argument
[root@node101 ~]#
[root@node101 ~]# systemctl start cloudera-scm-server #启动Cloudera Manager，所需要的时间可能在5～10秒左右！
[root@node101 ~]#
[root@node101 ~]# systemctl status cloudera-scm-server
● cloudera-scm-server.service - LSB: Cloudera SCM Server
Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled)
Active: active (exited) since Thu -- :: CST; 2min 38s ago
Docs: man:systemd-sysv-generator()
Process: ExecStart=/etc/rc.d/init.d/cloudera-scm-server start (code=exited, status=/SUCCESS) Feb :: node101.yinzhengjie.org.cn systemd[]: Starting LSB: Cloudera SCM Server...
Feb :: node101.yinzhengjie.org.cn su[]: PAM unable to dlopen(/usr/lib64/security/pam_fprintd.so): /usr/lib64/security/pam_fprintd.so: cannot open shared object file: No such file or directory
Feb :: node101.yinzhengjie.org.cn su[]: PAM adding faulty module: /usr/lib64/security/pam_fprintd.so
Feb :: node101.yinzhengjie.org.cn su[]: (to cloudera-scm) root on none
Feb :: node101.yinzhengjie.org.cn cloudera-scm-server[]: Starting cloudera-scm-server: [ OK ]
Feb :: node101.yinzhengjie.org.cn systemd[]: Started LSB: Cloudera SCM Server.
[root@node101 ~]#
[root@node101 ~]# systemctl start cloudera-scm-server #启动Cloudera Manager，所需要的时间可能在5～10秒左右！
5>.查看Cloudera Manager Server的日志并访问WebUI
[root@node101 ~]# tail -10f /var/log/cloudera-scm-server/cloudera-scm-server.log #查看Cloudera Manager的服务的日志
-- ::, INFO SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Initializing SearchTemplateManager:--21T04::.540Z
-- ::, INFO SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Generating entities:--21T04::.673Z
-- ::, INFO SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Num entities:
-- ::, INFO SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Generating documents:--21T04::.721Z
-- ::, INFO SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Num docs:
-- ::, INFO SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Constructing repo:--21T04::.823Z
-- ::, INFO WebServerImpl:org.mortbay.log: jetty-6.1..cloudera.
-- ::, INFO WebServerImpl:org.mortbay.log: Started SelectChannelConnector@0.0.0.0: #注意，如果出现这一行提示，说明你的服务启动成功啦！默认端口为7180，下面有WebServerImpl：Started Jetty server提示！
-- ::, INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
-- ::, INFO SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Finished constructing repo:--21T04::.827Z
[root@node101 ~]# tail -10f /var/log/cloudera-scm-server/cloudera-scm-server.log #查看Cloudera Manager的服务的日志
打开浏览器，访问：http://node101.yinzhengjie.org.cn:7180/cmf/login，如下图所示：
四.安装CDH5.15.1
1>.登陆Cloudera Manager Server 的WebUI
2>.接受协议，点击继续
3>.选择企业版（该版本的CDH是免费的）
4>.选择CDH
5>.为 CDH 群集安装指定主机。
6>.点击更多选项
7>.选择自建的CDH Parcels包镜像地址
8>.配置Cloudera Manager Agent的地址
9>.不要选中安装oracle Java，因为我们已经安装过JDK8，选中的话会装会JDK7，直接点击继续即可！
10>.不启用单用户模式安装，直接点击继续即可
11>.提供SSH登陆凭据
12>.等待Cloudera Manager Agent 安装完成
13>.查看安装过程的日志详情
14>.安装 Cloudera Manager Agent成功（根据集群性而定，我使用的是MAC，该过程大概5～10分钟左右！）
15>.下载CDH（从我们自建的镜像地址下载）
16>.CDH安装完成，点击继续
17>.等待主机检查
18>.点击完成
19>.完成上述步骤就已经完成了CDH的安装，还有一种方法安装CDH，就是下载CDH并存放到指定目录（了解即可！）
第一步：下载CDH 5.15.1的parcels包（https://archive.cloudera.com/cdh5/parcels/5.15.1/）
第二步：将下载的rpm包存放到CM默认到目录中（/opt/cloudera/parcel-repo）
[root@node101 ~]# ll /opt/cloudera/parcel-repo/
total
-rw-r--r-- root root Feb : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel
-rw-r--r-- root root Feb : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha1
-rw-r--r-- root root Feb : manifest.json
[root@node101 ~]#
[root@node101 ~]# mv /opt/cloudera/parcel-repo/CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha1 /opt/cloudera/parcel-repo/CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha #我们这里需要做一个重命名的操作！
[root@node101 ~]#
[root@node101 ~]# ll /opt/cloudera/parcel-repo/
total
-rw-r--r-- root root Feb : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel
-rw-r--r-- root root Feb : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha
-rw-r--r-- root root Feb : manifest.json
[root@node101 ~]#
五.安装Hadoop初始化的服务
1>.选择自定义服务
2>.选择需要安装的服务，点击继续（为了减少虚拟机的开销和安装时间，我这里只演示安装zookeeper集群和hdfs集群即可！其他服务的安装方法类似）
3>.自定义角色分配
4>.审核更改（这里咱们可以修改一些默认的配置，比如hdfs的块大小，数据存储的路径等等，实验环境我们默认即可，生产环境需要根据实际情况稍作修改即可！）
5>.安装初始化服务（安装过程时间可能需要3～5分钟）
6>.服务安装完成
7>.查看Cloudera Manager的监控界面
8>.安装服务完成后，我们可以查看到剩余的可用空间
CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager的更多相关文章
- 大数据 -- Cloudera Manager(简称CM)+CDH构建大数据平台
一.Cloudera Manager介绍 Cloudera Manager(简称CM)是Cloudera公司开发的一款大数据集群安装部署利器,这款利器具有集群自动化安装.中心化管理.集群监控.报警等功 ...
- CDH构建大数据平台-配置集群的Kerberos认证安全
CDH构建大数据平台-配置集群的Kerberos认证安全 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 当平台用户使用量少的时候我们可能不会在一集群安全功能的缺失,因为用户少,团 ...
- CDH构建大数据平台-Kerberos高可用部署【完结篇】
CDH构建大数据平台-Kerberos高可用部署[完结篇] 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 一.安装Kerberos相关的软件包并同步配置文件 1>.实验环境 ...
- 大数据平台搭建 - Mysql在linux上的安装
一.简介 MySQL是一个关系型数据库系统,由瑞典MySQL AB 公司开发,目前属于 Oracle 旗下产品.MySQL 是最流行的关系型数据库管理系统之一,在 WEB 应用方面,MySQL是最好的 ...
- 大数据平台搭建 - cdh5.11.1 - hive客户端安装
一.简介 hive是基于hadoop的一种数据仓库工具,可以将结构化的文件映射成为数据库的一张表,并提供简单sql查询功能,底层实现是转化为MapReduce任务计算. 二.安装 (1)下载 从cdh ...
- 基于Ambari构建自己的大数据平台产品
目前市场上常见的企业级大数据平台型的产品主流的有两个,一个是Cloudera公司推出的CDH,一个是Hortonworks公司推出的一套HDP,其中HDP是以开源的Ambari作为一个管理监控工具,C ...
- product of大数据平台搭建------CM 和CDH安装
一.安装说明 CM是由cloudera公司提供的大数据组件自动部署和监控管理工具,相应的和CDH是cloudera公司在开源的hadoop社区版的基础上做了商业化的封装的大数据平台. 采用离线安装模式 ...
- 朝花夕拾之--大数据平台CDH集群离线搭建
body { border: 1px solid #ddd; outline: 1300px solid #fff; margin: 16px auto; } body .markdown-body ...
- CM记录-CDH大数据平台实施经验总结2016（转载）
CDH大数据平台实施经验总结2016(转载) 2016年负责实施了一个生产环境的大数据平台,用的CDH平台+docker容器的方式,过了快半年了,现在把总结发出来. 1. 平台规划注意事项 1.1 业 ...
随机推荐
- Python3 queue队列类
class queue.PriorityQueue(maxsize=0) 优先级队列构造函数. maxsize 是个整数,用于设置可以放入队列中的项目数的上限.当达到这个大小的时候,插入操作将阻塞至队 ...
- springboot 整合Elasticsearch
Elasticsearch Elasticsearch 是一个分布式.可扩展.实时的搜索与数据分析引擎. 它能从项目一开始就赋予你的数据以搜索.分析和探索的能力,可用于实现全文搜索和实时数据统计. 在 ...
- Python - Django - 序列化
app01/__int__.py: import pymysql pymysql.install_as_MySQLdb() app01/models.py: from django.db import ...
- Spring boot后台搭建二集成Shiro权限控制
上一篇文章,实现了用户验证 查看,接下来实现下权限控制 权限控制,是管理资源访问的过程,用于对用户进行的操作授权,证明该用户是否允许进行当前操作,如访问某个链接,某个资源文件等 Apache Shir ...
- 【视频开发】 ffmpeg支持的硬解码接口
To enable DXVA2, use the --enable-dxva2 ffmpeg configure switch. To test decoding, use the following ...
- QT/C++ 类型转换
"轻轻地我走了,正如我轻轻地来,我挥一挥衣袖,不带走一片云彩"------阿魔 1) int转为QString: QString::number(int) 2) std::stri ...
- （idea maven）mybatis-generator步骤
1.新建一个maven项目,选择maven-archetype-webapp 点击next 2.项目名称,点击next 3.选择项目存放路径,然后点击finish 4.在main包下 添加包java和 ...
- java email发送（附件中文的处理）
这里使用的是commons-email-1.3.2.jar进行的开发,自认为这是简单的邮件发送. package com.yt.base.common; import java.io.Unsuppor ...
- SQL——LIKE操作符
一.LIKE操作符的基本用法 LIKE操作符用于在WHERE子句中,搜索相似.类似的数据. LIKE操作符语法: SELECT 列名1,列名2... FROM 表名 WHERE 列名 LIKE xxx ...
- STM32中的几个时钟SysTick、FCLK、SYSCLK、HCLK
STM32参考手册中的时钟树: 关于时钟讲解,在时钟树中都可以看出来:下面是正点原子PPT中的插图,看起来比较清晰. 总结一下: 1. 在STM32中,有五个时钟源,为HSI.HSE.LSI.LSE. ...