yum 安装cm

参考官网

主机名

/etc/hosts

hostnamectl  set-hostname  foo-1.data.com
hostnamectl set-hostname foo-2.data.com
hostnamectl set-hostname foo-2.data.com hostnamectl set-hostname cdh-1.data.com
hostnamectl set-hostname cdh-2.data.com
cat >> /etc/hosts <<EOF
192.168.2.150 cdh-1.xxxx.com cdh-1
192.168.2.117 cdh-2.xxxx.com cdh-2
EOF 192.168.1.142 foo-1.xxxx.com foo-1
192.168.1.143 foo-2.xxxx.com foo-2 swappiness /proc/sys/vm/swappiness 设置为 10 echo never > /sys/kernel/mm/transparent_hugepage/defrag

selinux

vi /etc/selinux/config
sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/confi
getenforce

防火墙

systemctl stop firewalld
systemctl disable firewalld
systemctl status firewalld

ssh免密

ssh-keygen -t rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
在一台机器上自己ssh 自己,成功后将.ssh目录拷贝到其他机器

repo本地源

cat >   /etc/yum.repos.d/local.repo  <<EOF
[localrepo]
name=localrepo
baseurl=file:///cm-5.13.3/
#gpgkey=http://192.168.2.113/cm-5.13.3/RPM-GPG-KEY-cloudera
enabled = 1
gpgcheck = 0
EOF

远程源

rm -rf  /etc/yum.repos.d/*
cat > /etc/yum.repos.d/cloudera-manager.repo <<EOF
[clouderamanager]
name=Cloudera Manager
baseurl=http://192.168.2.113/cm-5.13.3/
gpgkey=http://192.168.2.113/cm-5.13.3/RPM-GPG-KEY-cloudera
enabled = 1
gpgcheck = 0
EOF yum clean all && yum makecache

ntp

主节点
yum -y install ntp grep -v "^#" /etc/ntp.conf | grep -v "^$" cat > /etc/ntp.conf <<EOF
driftfile /var/lib/ntp/drift
restrict default nomodify notrap nopeer noquery
restrict 127.0.0.1
restrict ::1
restrict 192.168.2.0 mask 255.255.255.0 nomodify notrap
server 127.127.1.0
Fudge 127.127.1.0 stratum 1
includefile /etc/ntp/crypto/pw
keys /etc/ntp/keys
disable monitor
EOF 客户端
grep -v "^#" /etc/ntp.conf | grep -v "^$"
cat > /etc/ntp.conf <<EOF
driftfile /var/lib/ntp/drift
restrict default nomodify notrap nopeer noquery
####restrict 192.168.1.135 nomodify notrap noquery 这个不知道要不要
restrict 127.0.0.1
restrict ::1
restrict 192.168.2.0 mask 255.255.255.0 nomodify notrap
server 192.168.2.150 profer
Fudge 192.168.2.150 stratum 10
includefile /etc/ntp/crypto/pw
keys /etc/ntp/keys
disable monitor
EOF
开机自动启动
systemctl enable ntpd.service
systemctl start ntpd.service
手动同步
ntpdate -u cdh-1
ntpq -p 查看ntp运行状态 ntpstat

jdk(所有节点)

卸载自带的jdk-open
rpm -qa | grep java
rpm -qa | grep jdk - yum -y remove xxjdk #删除所有的jdk
yum remove -y *jdk*
yum remove -y *java* rpm -ivh jdk-8u202-linux-x64.rpm yum install java -y

数据库安装

卸载原有的mariadb
rpm -qa | grep mariadb #查看安装信息
rpm --nodeps -e `rpm -qa | grep mariadb`
安装数据库顺序 最好解压安装
tar -xf mysql-5.7.21-1.el7.x86_64.rpm-bundle.tar
首先安装(其他节点也需要安装,同时也安装 mysql-community-libs-compat-5.7.21-1.el7.x86_64.rpm ) yum install net-tools -y #安装net-tools包
yum install perl -y #安装perl包 安装顺序
rpm -ivh mysql-community-common-5.7.21-1.el7.x86_64.rpm
rpm -ivh mysql-community-libs-5.7.21-1.el7.x86_64.rpm
rpm -ivh mysql-community-client-5.7.21-1.el7.x86_64.rpm
rpm -ivh mysql-community-server-5.7.21-1.el7.x86_64.rpm
rpm -ivh mysql-community-libs-compat-5.7.21-1.el7.x86_64.rpm(compat解决hue数据库连接不上) 比较重要(其他节点都要安装)
rpm -ivh mysql-community-devel-5.7.21-1.el7.x86_64.rpm #安装mysqlclient需要用到 yum install mysql-community-common -y
yum install mysql-community-libs -y
yum install mysql-community-client -y
yum install mysql-community-libs-compat -y
yum install mysql-community-server -y
yum install mysql-community-devel -y 数据库默认配置
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
transaction-isolation = READ-COMMITTED
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
symbolic-links = 0 key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1 max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M #log_bin should be on a disk with enough free space.
#Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your
#system and chown the specified folder to the mysql user.
log_bin=/var/lib/mysql/mysql_binary_log #In later versions of MySQL, if you enable the binary log and do not set
#a server_id, MySQL will not start. The server_id must be unique within
#the replicating group.
server_id=1 binlog_format = mixed read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M # InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M [mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid sql_mode=STRICT_ALL_TABLES 启动mysql
systemctl start mysqld
systemctl enable mysqld
systemctl daemon-reload 查看root初始密码
cat /var/log/mysqld.log |grep password #vi /etc/my.cnf
在[mysqld]的段中加上一句:skip-grant-tables
例如:
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
skip-grant-tables USE mysql 修改密码;
ALTER USER 'root'@'localhost' IDENTIFIED BY 'xx.com';
FLUSH PRIVILEGES; mysql -uroot -pxx.com 登录mysql数据库
mysql -uroot –p密码 # 密码为上一步看到的
首先,修改validate_password_policy参数的值 set global validate_password_policy=0;
set global validate_password_length=2;
set global validate_password_special_char_count=0;
set global validate_password_mixed_case_count=0;
set global validate_password_number_count=0;
set password = password('daxxx.com'); 查看密码策略:
SHOW VARIABLES LIKE 'validate_password%'; 授权用户远程连接
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'xxx.com' WITH GRANT OPTION;
GRANT ALL PRIVILEGES ON *.* TO 'cdh'@'%' IDENTIFIED BY 'xxx.com' WITH GRANT OPTION; set global validate_password_policy=0;
set global validate_password_length=2;
set global validate_password_special_char_count=0;
set global validate_password_mixed_case_count=0;
set global validate_password_number_count=0; grant all privileges on *.* to 'scm'@'localhost' identified by 'scm' with grant option;
grant all privileges on *.* to 'scm'@'127.0.0.1' identified by 'scm' with grant option;
grant all privileges on *.* to 'scm'@'%' identified by 'scm' with grant option; grant all privileges on *.* to 'root'@'localhost' identified by 'xxxx.com' with grant option;
grant all privileges on *.* to 'root'@'127.0.0.1' identified by 'dxx.com' with grant option;
grant all privileges on *.* to 'root'@'%' identified by 'xxa.com' with grant option; grant all privileges on *.* to 'cdh'@'localhost' identified by 'dxx.com' with grant option;
grant all privileges on *.* to 'cdh'@'127.0.0.1' identified by 'xx.com' with grant option;
grant all privileges on *.* to 'cdh'@'%' identified by 'xxxa.com' with grant option; flush privileges; 创建需要的数据库:
create database metastore default character set utf8 DEFAULT COLLATE utf8_general_ci;
create database scm default character set utf8 DEFAULT COLLATE utf8_general_ci;
create database amon default character set utf8 DEFAULT COLLATE utf8_general_ci;
create database rmon default character set utf8 DEFAULT COLLATE utf8_general_ci;
create database hue default character set utf8 DEFAULT COLLATE utf8_general_ci;
create database sentry default character set utf8 DEFAULT COLLATE utf8_general_ci;
create database nav default character set utf8 DEFAULT COLLATE utf8_general_ci;
create database navms default character set utf8 DEFAULT COLLATE utf8_general_ci;
create database oozie default character set utf8 DEFAULT COLLATE utf8_general_ci;
create database hive default character set utf8 DEFAULT COLLATE utf8_general_ci;

CM安装

[root@foo-1 yum.repos.d]# cat  cloudera-manager.repo
[clouderamanager]
name=Cloudera Manager
baseurl=http://192.168.2.113/cm-5.13.3/
gpgkey=http://192.168.2.113/cm-5.13.3/RPM-GPG-KEY-cloudera
enabled = 1
gpgcheck = 0
install the Cloudera Manager packages.
在管理节点安装cloudera-manager-daemons cloudera-manager-server
#RHEL, if you have a yum repo configured
sudo yum install cloudera-manager-daemons cloudera-manager-server
sudo yum install cloudera-manager-agent #On all hosts, run the following command to install the Cloudera Manager agent:
在子节点,安装管理客户端
sudo yum install cloudera-manager-daemons cloudera-manager-agent *** 在所有节点修改管理代理连接管理服务器的主机名
On all hosts, configure the Cloudera Manager Agent to point to the Cloudera Manager Server by setting the following properties in the /etc/cloudera-scm-agent/config.ini configuration file:
server_host Name of the host where Cloudera Manager Server is running.
server_port Port on the host where Cloudera Manager Server is running. sed -i 's/server_host=localhost/server_host=cdh-1/' /etc/cloudera-scm-agent/config.ini 数据库连接程序 初始化 yum安装的scm_prepare_database.sh 位置
mkdir /usr/share/java/ && cd /usr/share/java/ curl -O http://192.168.2.113/mysql-connector-java/mysql-connector-java-5.1.46.tar.gz mkdir /usr/share/java/
cp mysql-connector-java-5.1.46-bin.jar mysql-connector-java.jar
chmod 755 mysql-connector-java-5.1.46-bin.jar mysql-connector-java.jar cp mysql-connector-java-5.1.46-bin.jar mysql-connector-java.jar /usr/share/cmf/lib/
mkdir /usr/share/java/ 初始化数据库:
/usr/share/cmf/schema/scm_prepare_database.sh mysql cm -hlocalhost -uroot -pxxxx.com --scm-host cdh-1 scm scm scm 所有节点运行管理代理
管理节点
systemctl start cloudera-scm-server
systemctl start cloudera-scm-agent 判断是否启动成功
find / -name cloudera-scm-server.log
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
grep "Started Jetty server" cloudera-scm-server.log 子节点
systemctl start cloudera-scm-agent 安装后节点也需要安装
cd /usr/share/cmf/lib/
mv /usr/share/java/mysql-connector-java-5.1.46-bin.jar . cdh 也需要驱动
scp mysql-connector-java-5.1.46-bin.jar foo-2:/opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p0.2/lib hive 数据库也需要
scp mysql-connector-java-5.1.46-bin.jar foo-2:/opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p0.2/lib/hive/lib
scp mysql-connector-java-5.1.46-bin.jar foo-2:/opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p0.2/lib
scp mysql-connector-java-5.1.46-bin.jar foo-2:/opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p0.2/lib/hive/lib
scp mysql-connector-java-5.1.46-bin.jar foo-2:/opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p0.2/lib/oozie/lib
scp mysql-connector-java-5.1.46-bin.jar foo-2:/opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p0.2/lib/hue/lib
scp mysql-connector-java-5.1.46-bin.jar foo-2:/var/lib/oozie
scp mysql-connector-java-5.1.46-bin.jar foo-2:/var/lib/hue

http://192.168.2.150:7180/cmf/clusters/1/express-add-services/index#step=commandDetailsStep

cdh本地源安装-自用的更多相关文章

  1. Debian8 从本地源安装软件

    假如你从Debian官网下载了三个DVD镜像,那么就可以设置从本地源来安装软件,例如这里我从官网下载了 debian-8.6.0-amd64-DVD-1.iso,debian-8.6.0-amd64- ...

  2. 【从零开始搭建K8S】【第一篇】CentOS7.6离线安装Docker(手动安装以及基于yum本地源安装)

    下载CentOS7.6以及最小化安装CentOS7.6版本.由于CentOS属于开源软件,在国内也有很多的mirror站点可供下载,我选择的是华为站点进行下载:http://mirrors.huawe ...

  3. 使用本地光盘安装Microsoft .NET Framework 3.5 for Win8/WinServer2012

    如何使用本地源安装Microsoft .NET Framework 3.5 作为SQL Server 2012的 必要组件,校验组件过程有个小BUG,即使没有安装也通过,但会卡在安装环节(enabli ...

  4. linux替换yum源及配置本地源

    linux系统安装后自带的bash源由于在国外,安装软件包的时候会非常慢,最好替换一下yum源. ​关于yum源的简单介绍 ​           yum的主要功能是更方便地添加,删除和更新rpmba ...

  5. 创建本地yum软件源,为本地Package安装Cloudera Manager、Cloudera Hadoop及Impala做准备

    一.包管理工具及CentOS的yum 1.包管理工具如何发现可以用的包 包管理工具依赖一系列软件源,工具下载源的信息存储在配置文件中,其位置随某包管理工具不同而变化 使用yum的RedHat/Cent ...

  6. 解决安装rpm包依赖关系的烦恼 - yum工具介绍及本地源配置方法

    版权声明:本文发布于http://www.cnblogs.com/yumiko/,版权由Yumiko_sunny所有,欢迎转载.转载时,请在文章明显位置注明原文链接.若在未经作者同意的情况下,将本文内 ...

  7. 本地yum源安装GCC

    Linux环境下yum源安装GCC 前提条件是有Linux环境的安装盘ISO文件 在Linux系统中创建两个目录,一个是用来存放ISO文件,一个是用来挂载该ISO文件,如下: $mkdir /root ...

  8. 远程安装实施时,如何配置远程服务器的本地 yum 安装源

    配置本地 yum 安装源 overview 1.使用 ftp 将OracleLinux-R5-U8-Server-x86_64-dvd.iso 上传到 /home 目录下.再使用 mount 命令挂载 ...

  9. 通过本地yum源安装软件报错[Errno 14] PYCURL ERROR 56 - "Failure when receiving data from the peer"

    通过本地yum源安装软件报错 http://192.168.3.85/centos/6/os/x86_64/Packages/php-pdo-5.3.3-47.el6.x86_64.rpm: [Err ...

随机推荐

  1. nginx 反向代理是url带后缀

    1.后端:http://10.253.149.2/tdsqlpcloud/index.php 2.配置文件 danjan01deiMac:nginx danjan01$ cat /usr/local/ ...

  2. synchronized(三) 锁的膨胀过程(锁的升级过程)深入剖析

    警告⚠️:本文耗时很长,先做好心理准备................哈哈哈 本篇我们讲通过大量实例代码及hotspot源码分析偏向锁(批量重偏向.批量撤销).轻量级锁.重量级锁及锁的膨胀过程(也就是 ...

  3. Longest Palindromic Subsequence

    Given a string s, find the longest palindromic subsequence's length in s. You may assume that the ma ...

  4. Oracle集群检测命令

    select inst_id, count(inst_id) from gv$session group by inst_id order by inst_id; srvctl stop databa ...

  5. 认识并学会springCloud的使用

    SpringCloud将现在一些流行的技术整合到一起,实现如:配置管理,服务发现,智能路由,负载均衡,熔断器,控制总线,集群状态等等功能.主要涉及的组件有 netflix Eureka:注册中心 Zu ...

  6. POJ - 1149 PIGS (建图思维+最大流)

    (点击查看原题) 题目分析 (以下均为 Edelweiss 大佬的思路,博主承认自己写不了这么好,但是学习的心促使我记录下这个好题的写法,所以代码是我写的) [题目大意] 有 M 个猪圈,每个猪圈里初 ...

  7. 06: zabbix常见面试题

    1.1 zabbix架构 1.zabbix理论 1)Zabbix是一个企业级的.开源的.分布式的监控套件,Zabbix可以监控网络和服务的监控状况. 2)Zabbix利用灵活的告警机制,允许用户对事件 ...

  8. mongodb 数据操作(1)

    切换/创建数据库 use test 添加数据db.student.save({name:"J33ack",age:25}) 查看数据库show dbs 删除当前数据库 db.dro ...

  9. spark教程(10)-sparkSQL

    sparkSQL 的由来 我们知道最初的计算框架叫 mapreduce,他的缺点是计算速度慢,还有一个就是代码比较麻烦,所以有了 hive: hive 是把类 sql 的语句转换成 mapreduce ...

  10. C# 面向对象1(类和对象的概念)

    1.面向过程-->面向对象 面向过程:面向的是完成这件事儿的过程,强调的是完成这件事儿的动作. 2.面向过程的思想: 以上的,需求一更改,会导致不同的方法,一一去更改. 3.面向对象的思想:找个 ...