【测试】RAC搭建（裸设备）

环境描述：

	节点一	节点二
主机名	rac1	rac2
IP	192.168.10.11	192.168.10.12
IP-VIP	192.168.10.111	192.168.10.112
IP-priv	10.10.10.11	10.10.10.12
rac-scan	192.168.10.200

1.安装操作系统

1.1安装操作系统

---同样操作创建rac2

---设置网卡，挂载光盘,打开虚拟机安装系统：

---密码默认oracle

---同样操作安装rac2的操作系统

1.2关闭无用的服务：（所有节点）

service NetworkManager  stop service acpid           stop service anacron         stop service apmd            stop service atd             stop service auditd          stop service autofs          stop service avahi-daemon    stop service avahi-dnsconfd  stop service bluetooth       stop service conman          stop service cpuspeed        stop service cups            stop service dnsmasq         stop service dund            stop service firstboot       stop service gpm             stop service haldaemon       stop service hidd            stop service ip6tables       stop service ipmi            stop service ipmievd         stop service iptables        stop service irda            stop service irqbalance      stop service kdump           stop service kudzu           stop service mcstrans        stop service mdmpd           stop service microcode_ctl   stop service netconsole      stop service netfs           stop service netplugd        stop service nfs             stop service nfslock         stop service nscd            stop service ntpd            stop service pand            stop service pcscd           stop service portmap         stop service psacct          stop service rawdevices      stop service rdisc           stop service readahead_early stop service readahead_later stop service restorecond     stop service rhnsd           stop service rpcgssd         stop service rpcidmapd       stop service rpcsvcgssd      stop service saslauthd       stop service sendmail        stop service setroubleshoot  stop service smartd          stop service vncserver       stop service wdaemon         stop service winbind         stop service wpa_supplicant  stop service xfs             stop service xinetd          stop service ypbind          stop service yum-updatesd    stop

chkconfig NetworkManager  off chkconfig acpid           off chkconfig anacron         off chkconfig apmd            off chkconfig atd             off chkconfig auditd          off chkconfig autofs          off chkconfig avahi-daemon    off chkconfig avahi-dnsconfd  off chkconfig bluetooth       off chkconfig conman          off chkconfig cpuspeed        off chkconfig cups            off chkconfig dnsmasq         off chkconfig dund            off chkconfig firstboot       off chkconfig gpm             off chkconfig haldaemon       off chkconfig hidd            off chkconfig ip6tables       off chkconfig ipmi            off chkconfig ipmievd         off chkconfig iptables        off chkconfig irda            off chkconfig isdn            off chkconfig irqbalance      off chkconfig kdump           off chkconfig kudzu           off chkconfig mcstrans        off chkconfig mdmpd           off chkconfig microcode_ctl   off chkconfig netconsole      off chkconfig netfs           off chkconfig netplugd        off chkconfig nfs             off chkconfig nfslock         off chkconfig nscd            off chkconfig ntpd            off chkconfig pand            off chkconfig pcscd           off chkconfig portmap         off chkconfig psacct          off chkconfig rawdevices      off chkconfig rdisc           off chkconfig readahead_early off chkconfig readahead_later off chkconfig restorecond     off chkconfig rhnsd           off chkconfig rpcgssd         off chkconfig rpcidmapd       off chkconfig rpcsvcgssd      off chkconfig saslauthd       off chkconfig sendmail        off chkconfig setroubleshoot  off chkconfig smartd          off chkconfig vncserver       off chkconfig wdaemon         off chkconfig winbind         off chkconfig wpa_supplicant  off chkconfig xfs             off chkconfig xinetd          off chkconfig ypbind          off chkconfig yum-updatesd    off

1.3关闭系统ntp服务，采用oracle自带的时间同步服务：

[root@rac1 ~]# chkconfig ntpd off
[root@rac1 ~]# mv /etc/ntp.conf /etc/ntp.conf.bak

2.环境配置（所有节点）：

2.1hosts文件设置：

[root@rac1 ~]# vi /etc/hosts

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1       localhost

192.168.10.11 rac1
192.168.10.12 rac2

192.168.10.111 rac1-vip
192.168.10.112 rac2-vip

10.10.10.11 rac1-priv
10.10.10.12 rac2-priv

192.168.10.200 rac-scan

2.2创建用户和组：

---创建用户和组：

groupadd  -g 200 oinstall
groupadd  -g 201 dba
groupadd  -g 202 oper
groupadd  -g 203 asmadmin
groupadd  -g 204 asmoper
groupadd  -g 205 asmdba
useradd -u 200 -g oinstall -G dba,asmdba,oper oracle
useradd -u 201 -g oinstall -G asmadmin,asmdba,asmoper,oper,dba grid

---创建相关的目录：

mkdir -p /u01/app/oraInventory
chown -R grid:oinstall /u01/app/oraInventory/
chmod -R 775 /u01/app/oraInventory/
mkdir -p /u01/11.2.0/grid
chown -R grid:oinstall /u01/11.2.0/grid/
chmod -R 775 /u01/11.2.0/grid/
mkdir -p /u01/app/oracle
mkdir -p /u01/app/oracle/cfgtoollogs
mkdir -p /u01/app/oracle/product/11.2.0/db_1
chown -R oracle:oinstall /u01/app/oracle
chmod -R 775 /u01/app/oracle

---密码设置：

[root@rac1 ~]# passwd oracle
Changing password for user oracle.
New UNIX password:
BAD PASSWORD: it is based on a dictionary word
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
[root@rac1 ~]# passwd grid
Changing password for user grid.
New UNIX password:
BAD PASSWORD: it is based on a dictionary word
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
[root@rac1 ~]# 

2.3修改内核参数：

--修改的时候，直接写入文件最下方即可。

--也可以不提前修改，在安装的时候会有修改提示。

[root@rac1 ~]# vi /etc/sysctl.conf
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 536870912
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586

<tc/sysctl.conf" 50L, 1390C written
[root@rac1 ~]# 

——参数生效

[root@rac1 ~]# sysctl -p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 536870912
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
[root@rac1 ~]# 

---安全限制修改：

[root@rac1 ~]# vi /etc/security/limits.conf

oracle              soft    nproc   2047
oracle              hard    nproc   16384
oracle              soft    nofile  1024
oracle              hard    nofile  65536
oracle              soft    stack   10240
grid              soft    nproc   2047
grid              hard    nproc   16384
grid              soft    nofile  1024
grid              hard    nofile  65536
grid              soft    stack   10240

~
<ty/limits.conf" 62L, 2210C written
[root@rac1 ~]# 

---修改其他参数：

[root@rac1 ~]# vi /etc/pam.d/login
session required /lib/security/pam_limits.so

[root@rac1 ~]# vi /etc/profile :
if [ $USER = "oracle" ]||[ $USER = "grid" ]; then
    if [ $SHELL = "/bin/ksh" ]; then
                ulimit -p 16384
                ulimit -n 65536
        else
                ulimit -u 16384 -n 65536
        fi
fi

2.4修改oracle用户和grid用户的环境变量：

——rac1  ORACLE_SID=PROD1  ORACLE_SID=+ASM1

——rac2  ORACLE_SID=PROD2  ORACLE_SID=+ASM2

——环境变量必须大写，dbca建库，实例名默认大写

——Oracle用户

——Oracle用户

[root@rac1 ~]# su - oracle
[oracle@rac1 ~]$ vi .bash_profile
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin

export EDITOR=vi
export ORACLE_SID=PROD1
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export PATH=$ORACLE_HOME/bin:/bin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/X11R6/bin
umask 022

<9L, 426C written
[oracle@rac1 ~]$ 

——grid用户

[oracle@rac1 ~]$ su - grid
Password:
[grid@rac1 ~]$ vi .bash_profile

# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin

export EDITOR=vi
export ORACLE_SID=+ASM1
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/11.2.0/grid
export GRID_HOME=/u01/11.2.0/grid
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export THREADS_FLAG=native
export PATH=$ORACLE_HOME/bin:/bin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/X11R6/bin
umask 022

~
~
~
~
~
~
<3L, 473C written
[grid@rac1 ~]$ 

3.rpm包安装：

——需要先加载光盘
[grid@rac1 ~]$ su - root
Password:
[root@rac1 ~]# mount /dev/cdrom /mnt
mount: block device /dev/cdrom is write-protected, mounting read-only
[root@rac1 ~]# vi /etc/yum.repos.d/public-yum-el5.repo

[oel5]
name = Enterprise Linux 5.6 DVD
baseurl=file:///mnt/Server/
gpgcheck=0
enabled=1

<um-el5.repo" [New] 7L, 90C written
[root@rac1 ~]# 

[root@rac1 ~]# yum install oracle-validated

4.时间同步

——查看时间是否同步，时间差不要超过20秒

[root@rac1 ~]# date
Sat Oct 29 10:05:19 CST 2016
[root@rac1 ~]# 

[root@rac2 ~]# date
Sat Oct 29 10:05:19 CST 2016
[root@rac2 ~]# 

5.存储

5.1配置共享存储

——关闭虚拟机，注意取消挂载的光盘，避免进入重新安装系统界面

——新创建的4块虚拟磁盘都要设置

—然后在节点2（rac2）添加虚拟磁盘

——注意，添加的顺序要和节点一一致

——打开虚拟机，查看磁盘信息
——节点一
[root@rac1 ~]# fdisk -l

Disk /dev/sda: 51.5 GB, 51539607552 bytes
255 heads, 63 sectors/track, 6266 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14         535     4192965   82  Linux swap / Solaris
/dev/sda3             536        6266    46034257+  83  Linux

Disk /dev/sdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdb doesn't contain a valid partition table

Disk /dev/sdc: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdc doesn't contain a valid partition table

Disk /dev/sdd: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdd doesn't contain a valid partition table

Disk /dev/sde: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sde doesn't contain a valid partition table
[root@rac1 ~]# 

——节点二
[root@rac2 ~]# fdisk -l

Disk /dev/sda: 51.5 GB, 51539607552 bytes
255 heads, 63 sectors/track, 6266 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14         535     4192965   82  Linux swap / Solaris
/dev/sda3             536        6266    46034257+  83  Linux

Disk /dev/sdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdb doesn't contain a valid partition table

Disk /dev/sdc: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdc doesn't contain a valid partition table

Disk /dev/sdd: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdd doesn't contain a valid partition table

Disk /dev/sde: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sde doesn't contain a valid partition table
[root@rac2 ~]# 

5.2配置裸设备：

——在节点一上临时设置，主要为了查看major
[root@rac1 ~]# raw /dev/raw/raw1 /dev/sdb
/dev/raw/raw1:  bound to major 8, minor 16
[root@rac1 ~]# raw /dev/raw/raw2 /dev/sdc
/dev/raw/raw2:  bound to major 8, minor 32
[root@rac1 ~]# raw /dev/raw/raw3 /dev/sdd
/dev/raw/raw3:  bound to major 8, minor 48
[root@rac1 ~]# raw /dev/raw/raw4 /dev/sde
/dev/raw/raw4:  bound to major 8, minor 64
[root@rac1 ~]# 

——设置参数文件（所有节点）
[root@rac1 ~]# vi /etc/udev/rules.d/60-raw.rules
# Enter raw device bindings here.
#
# An example would be:
#   ACTION=="add", KERNEL=="sda", RUN+="/bin/raw /dev/raw/raw1 %N"
# to bind /dev/raw/raw1 to /dev/sda, or
#   ACTION=="add", ENV{MAJOR}=="8", ENV{MINOR}=="1", RUN+="/bin/raw /dev/raw/raw2 %M %m"
# to bind /dev/raw/raw2 to the device with major 8, minor 1.

ACTION=="add", KERNEL=="/dev/sdb",RUN+="/bin/raw /dev/raw/raw1 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="16",RUN+="/bin/raw /dev/raw/raw1 %M %m"
ACTION=="add", KERNEL=="/dev/sdc",RUN+="/bin/raw /dev/raw/raw2 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="32",RUN+="/bin/raw /dev/raw/raw2 %M %m"
ACTION=="add", KERNEL=="/dev/sdd",RUN+="/bin/raw /dev/raw/raw3 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="48",RUN+="/bin/raw /dev/raw/raw3 %M %m"
ACTION=="add", KERNEL=="/dev/sde",RUN+="/bin/raw /dev/raw/raw4 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="64",RUN+="/bin/raw /dev/raw/raw4 %M %m"
KERNEL=="raw[1-4]", OWNER="grid", GROUP="asmadmin", MODE="660"

——启动服务或者重启节点（所有节点）
[root@rac1 ~]# start_udev
Starting udev: [  OK  ]
[root@rac1 ~]# 

——查看
——节点一
[root@rac1 ~]# ls -lrt /dev/raw/raw*
crw-rw---- 1 grid asmadmin 162, 3 Oct 29 11:15 /dev/raw/raw3
crw-rw---- 1 grid asmadmin 162, 2 Oct 29 11:15 /dev/raw/raw2
crw-rw---- 1 grid asmadmin 162, 4 Oct 29 11:15 /dev/raw/raw4
crw-rw---- 1 grid asmadmin 162, 1 Oct 29 11:15 /dev/raw/raw1
[root@rac1 ~]# 

——节点二
[root@rac2 ~]# ls -lrt /dev/raw/raw*
crw-rw---- 1 grid asmadmin 162, 4 Oct 29 11:17 /dev/raw/raw4
crw-rw---- 1 grid asmadmin 162, 3 Oct 29 11:17 /dev/raw/raw3
crw-rw---- 1 grid asmadmin 162, 2 Oct 29 11:17 /dev/raw/raw2
crw-rw---- 1 grid asmadmin 162, 1 Oct 29 11:17 /dev/raw/raw1
[root@rac2 ~]# 

6.安装GI（单一节点操作）

6.1上传安装介质

——在grid用户下创建soft文件夹，安装介质上传到soft文件夹后解压缩

6.2安装前检测：

[grid@rac1 grid]$ ./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -verbose

6.3安装GI

——添加节点一和节点二的grid用户互信

——查看IP地址段是否正确

——DNS错误，忽略就可以了

——优先在节点一执行脚本，执行完一个脚本后，在执行一个

——可以忽略报错

6.4验证安装是否成功：

[grid@rac1 ~]$ crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora....ER.lsnr ora....er.type ONLINE    ONLINE    rac1
ora....N1.lsnr ora....er.type ONLINE    ONLINE    rac1
ora.OCR.dg     ora....up.type ONLINE    ONLINE    rac1
ora.asm        ora.asm.type   ONLINE    ONLINE    rac1
ora.cvu        ora.cvu.type   ONLINE    ONLINE    rac1
ora.gsd        ora.gsd.type   OFFLINE   OFFLINE
ora....network ora....rk.type ONLINE    ONLINE    rac1
ora.oc4j       ora.oc4j.type  ONLINE    ONLINE    rac1
ora.ons        ora.ons.type   ONLINE    ONLINE    rac1
ora....SM1.asm application    ONLINE    ONLINE    rac1
ora....C1.lsnr application    ONLINE    ONLINE    rac1
ora.rac1.gsd   application    OFFLINE   OFFLINE
ora.rac1.ons   application    ONLINE    ONLINE    rac1
ora.rac1.vip   ora....t1.type ONLINE    ONLINE    rac1
ora....SM2.asm application    ONLINE    ONLINE    rac2
ora....C2.lsnr application    ONLINE    ONLINE    rac2
ora.rac2.gsd   application    OFFLINE   OFFLINE
ora.rac2.ons   application    ONLINE    ONLINE    rac2
ora.rac2.vip   ora....t1.type ONLINE    ONLINE    rac2
ora.scan1.vip  ora....ip.type ONLINE    ONLINE    rac1
[grid@rac1 ~]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[grid@rac1 ~]$ 

 7.安装oracle软件（单一节点操作）：

7.1上传安装介质：

——在oracle用户中创建soft文件夹，上传介质到soft文件夹中，解压缩

7.2安装oracle：

——创建两个节点的oracle用户互信

——在所有节点的root用户下依次执行脚本

 8.创建asm磁盘组：

——在节点一的grid用户下，使用asmca打开图形化进行创建

——需要注意是否所有节点都mount上了

 9.DBCA建库

——在oracle用户下，使用dbca图形化建库

——通过rman恢复的方式建库，快一些

---至此，rac以全部搭建完成，过程很多很杂。