VMware搭建Oracle 11g RAC测试环境 For Linux

环境如下：

Linux操作系统：Centos 6.5 64bit （这个版本的redhat 6内核等OS在安装grid最后执行root.sh时会出现crs-4124，是oracle11.2.0.1的bug）

VMware version：Workstation 8.0.3 build-703057

Oracle版本：11G R2 grid/oracle 64位 11.2.0.1.0 for linux

步骤规划：

1，配置虚拟机、安装Linux操作系统

2，配置OS

3，配置ASM

4，安装Oracle grid

5，安装Oracle database

6，创建ASM磁盘组

7，建立数据库实例

1，配置虚拟机、安装Linux操作系统

1.1  新建虚拟机：RAC1和RAC2，并安装OS

下载Centos的iso后新建虚拟机后安装即可，这里略。注意内存分配，每个虚拟机至少1.5G内存比较好，我的机器8G内存，给每个虚拟机分了2G，给每个虚拟机分配50GB硬盘容量，配置2个桥接网卡。

1.2   配置Share Disks

新建共享硬盘，在DOS VM安装目录下执行：

cd \D F:/Software/vm8/

vmware-vdiskmanager.exe -c -s 1000Mb -a lsilogic -t 2 K:\VMDisk\Sharedisk\ocr.vmdk
vmware-vdiskmanager.exe -c -s 1000Mb -a lsilogic -t 2 K:\VMDisk\Sharedisk\ocr2.vmdk
vmware-vdiskmanager.exe -c -s 1000Mb -a lsilogic -t 2 K:\VMDisk\Sharedisk\votingdisk.vmdk
vmware-vdiskmanager.exe -c -s 20000Mb -a lsilogic -t 2 K:\VMDisk\Sharedisk\data.vmdk
vmware-vdiskmanager.exe -c -s 10000Mb -a lsilogic -t 2 K:\VMDisk\Sharedisk\backup.vmdk

配置虚拟机RAC1 RAC2目录下的vmx虚拟机配置文件，在行最后添加如下：

scsi1.present = "TRUE"
scsi1.virtualDev = "lsilogic"
scsi1.sharedBus = "virtual"

scsi1:1.present = "TRUE"
scsi1:1.mode = "independent-persistent"
scsi1:1.filename = "K:\VMDisk\Sharedisk\ocr.vmdk"
scsi1:1.deviceType = "plainDisk"

scsi1:2.present = "TRUE"
scsi1:2.mode = "independent-persistent"
scsi1:2.filename = "K:\VMDisk\Sharedisk\votingdisk.vmdk"
scsi1:2.deviceType = "plainDisk"

scsi1:3.present = "TRUE"
scsi1:3.mode = "independent-persistent"
scsi1:3.filename = "K:\VMDisk\Sharedisk\data.vmdk"
scsi1:3.deviceType = "plainDisk"

scsi1:4.present = "TRUE"
scsi1:4.mode = "independent-persistent"
scsi1:4.filename = "K:\VMDisk\Sharedisk\backup.vmdk"
scsi1:4.deviceType = "plainDisk"

scsi1:5.present = "TRUE"
scsi1:5.mode = "independent-persistent"
scsi1:5.filename = "K:\VMDisk\Sharedisk\ocr2.vmdk"
scsi1:5.deviceType = "plainDisk"

disk.locking = "false"
diskLib.dataCacheMaxSize = "0"
diskLib.dataCacheMaxReadAheadSize = "0"
diskLib.DataCacheMinReadAheadSize = "0"
diskLib.dataCachePageSize = "4096"
diskLib.maxUnsyncedWrites = "0"

保存退出。

最后虚拟机图大致如下：

再次提醒注意内存、硬盘和网卡配置。

配置虚拟机、安装Linux操作系统完成。

2，配置OS

以下没有特别说明都需要同时在两个节点执行。

2.1 关闭防火墙、selinux(考虑到后面安装时会碰到一些情况，所以需要执行这步骤）

关闭selinux
setenforce 0   
[root@rac1 ~]# vim /etc/selinux/config
SELINUX=disabled

关闭防火墙
service iptables stop

chkconfig iptables off

2.2 配置网络

修改节点主机名为rac1，rac2

hostname rac1

hostname rac2

或修改/etc/sysconfig/network

RAC1

eth0: 192.168.1.130

eth1: 10.10.17.221

RAC2

eth0: 192.168.1.140

eth1: 10.10.17.222

host文件，两个节点都配置

Vim /etc/hosts

192.168.1.130 rac1
192.168.1.140 rac2

192.168.1.131 rac1-vip
192.168.1.141 rac2-vip

10.10.17.221 rac1-priv
10.10.17.222 rac2-priv

192.168.1.200 scan-ip

保存退出。

配置完之后互相ping看看是否连通。

2.3 配置CTSS服务、节点SSH等效

CTSS

需要把ntp服务停止和删除配置文件及pid文件

service ntpd stop

chkconfig ntpd off

rm -rf /etc/ntpd.conf

rm -rf /var/run/ntpd.pid

查看ctss服务资源

crsctl check ctss

SSH

在grid安装时会提供配置步骤，所以不需要手工配置。

ssh具体配置

rac1 rac2
mkdir ~/.ssh
chmod 700 ~/.ssh
ssh-keygen -t rsa
ssh-keygen -t dsa

rac1 rac2
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

测试SSH

ssh rac2

ssh rac1

2.4 配置Oracle软件安装用户、组、环境变量、安装目录

#组
 groupadd -g 1300 dba
 groupadd -g 1301 oper
 groupadd -g 1000 oinstall
 groupadd -g 1200 asmadmin
 groupadd -g 1201 asmdba
 groupadd -g 1202 asmoper

#用户
useradd -m -u 1100 -g oinstall -G asmadmin,asmdba,asmoper -s /bin/bash grid

useradd -m -u 1101 -g oinstall -G dba,oper,asmdba -s /bin/bash oracle
#用户环境变量修改home下的 .bash_profile

ORACLE

export TMP=/tmp
 export TMPDIR=$TMP
 export ORACLE_SID=RAC1  # RAC1
 export ORACLE_SID=RAC2  # RAC2
 export ORACLE_BASE=/u01/app/oracle
 export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
 export TNS_ADMIN=$ORACLE_HOME/network/admin
 export PATH=/usr/sbin:$PATH
 export PATH=$ORACLE_HOME/bin:$PATH
 export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
 export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
 umask 022

GRID
 export TMP=/tmp
 export TMPDIR=$TMP
 export ORACLE_SID=+ASM1  # RAC1

export ORACLE_SID=+ASM2  # RAC2

export ORACLE_BASE=/u01/app/grid
 export ORACLE_HOME=/u01/app/11.2.0/grid
 export PATH=/usr/sbin:$PATH
 export PATH=$ORACLE_HOME/bin:$PATH
 export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
 export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
 umask 022

配置目录
 mkdir -p  /u01/app/11.2.0/grid
 mkdir -p /u01/app/grid
 mkdir -p /u01/app/oracle
 chown grid:oinstall /u01/app/11.2.0/grid
 chown grid:oinstall /u01/app/grid
 chown oracle:oinstall /u01/app/oracle
 chmod -R 775 /u01/
 chown -R grid:oinstall /u01

2.5 配置系统文件

配置oracle、grid用户的shell限制

vim /etc/security/limits.conf

添加

grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536

修改/etc/pam.d/login配置文件

Vim  /etc/pam.d/login
添加 session    required     pam_limits.so

修改内核配置文件/etc/sysctl.conf

fs.aio-max-nr = 1048576
 fs.file-max = 6815744
 kernel.shmall = 2097152
 kernel.shmmax = 1054472192
 kernel.shmmni = 4096
 kernel.sem = 250 32000 100 128
 net.ipv4.ip_local_port_range = 9000 65500
 net.core.rmem_default = 262144
 net.core.rmem_max = 4194304
 net.core.wmem_default = 262144
 net.core.wmem_max = 1048586
 net.ipv4.tcp_wmem = 262144 262144 262144
 net.ipv4.tcp_rmem = 4194304 4194304 4194304
保存退出后执行 sysctl -p 立即生效。

2.6 配置共享磁盘 （在一个节点执行即可）

查看分区

ls /dev/sd*

使用fdisk进行分区，其他区按照自己的规划划分就可以了，例：

fdisk /dev/sdb

[root@rac1 ~]# fdisk /dev/sdb

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1000, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1000, default 1000): +500M

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (502-1000, default 502):
Using default value 502
Last cylinder, +cylinders or +size{K,M,G} (502-1000, default 1000):
Using default value 1000

Command (m for help): w

The partition table has been altered!

最后配置完后查看分区

fidks -l

[root@rac1 ~]# ls /dev/sd*
/dev/sda  /dev/sda1  /dev/sda2  /dev/sdb  /dev/sdb1  /dev/sdc 
/dev/sdc1  /dev/sdd  /dev/sdd1  /dev/sdd2  /dev/sde  /dev/sde1 
/dev/sde2  /dev/sdf  /dev/sdf1

2.7 安装Oracle grid等需要的包

binutils-2.20.51.0.2-5.11.el6 (x86_64)
compat-libcap1-1.10-1 (x86_64)
compat-libstdc++-33-3.2.3-69.el6 (x86_64)
compat-libstdc++-33-3.2.3-69.el6.i686
gcc-4.4.4-13.el6 (x86_64)
gcc-c++-4.4.4-13.el6 (x86_64)
glibc-2.12-1.7.el6 (i686)
glibc-2.12-1.7.el6 (x86_64)
glibc-devel-2.12-1.7.el6 (x86_64)
glibc-devel-2.12-1.7.el6.i686
ksh
libgcc-4.4.4-13.el6 (i686)
libgcc-4.4.4-13.el6 (x86_64)
libstdc++-4.4.4-13.el6 (x86_64)
libstdc++-4.4.4-13.el6.i686
libstdc++-devel-4.4.4-13.el6 (x86_64)
libstdc++-devel-4.4.4-13.el6.i686
libaio-0.3.107-10.el6 (x86_64)
libaio-0.3.107-10.el6.i686
libaio-devel-0.3.107-10.el6 (x86_64)
libaio-devel-0.3.107-10.el6.i686
make-3.81-19.el6
sysstat-9.0.4-11.el6 (x86_64)

3，配置ASM

3.1 查看操作系统内核版本对应下载

uname -a

下载oracleasm http://www.oracle.com/technetwork/topics/linux/asmlib/index-101839.html

安装

yum -y install kmod-oracleasm.x86_64

rpm -ivh oracleasmlib-2.0.4-1.el6.x86_64.rpm
rpm -ivh oracleasm-support-2.1.8-1.el6.x86_64.rpm

3.2 配置ASM磁盘

在两个节点上执行

[root@rac1 oracle]# /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: grid
Default group to own the driver interface []: asmdba
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]:
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver:                     [  OK  ]
Scanning the system for Oracle ASMLib disks:               [  OK  ]
[root@rac1 oracle]# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks:               [  OK  ]

[root@rac1 ~]# /etc/init.d/oracleasm listdisks
VOL1
VOL2
VOL3
VOL4
VOL5
VOL6
VOL7

注：如果没有查看到asm磁盘组需要查看之前的selinux是否有禁用。

4，安装Oracle grid

安装前可以执行Oracle提供的检查环境工具

./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -fixup -verbose

4.1 在安装目录下执行./runinstaller，如果出现安装grid提示must be configured to display at least 256 colors Failed，就执行如下
 root 下先执行#xhost +
 su  -  grid
 export DISPLAY=:0.0

注意在此建立SSH等效。执行setup后点test测试。如果不成功就手工配置SSH再进行测试。

这个地方忘记截图了，就是类似这个图。定义磁盘组名字，这里我选择normal冗余，需要选择3个VOL。

这里是把OCR和votingdisk安装在所选磁盘。后面还需要新建asm磁盘组来安装其他数据文件。

这里的software location应该是 /u01/app/11.2.0/grid

通过Oracle的系统环境检查，安装缺失的包和修改系统环境参数等。

安装65%的时候可能会出现卡顿的情况，这个时候需要检查防火墙和selinux是否关闭，详细情况可以点击detail查看安装进度，如出现拷贝文件到RAC2节点的信息则正常。

根据提示执行orainstRoot.sh和root.sh脚本，先执行rac1后再执行rac2。

在执行root .sh时可能在的时候会提示出现crs-4124，这个是oracle的bug，当在redhat 6等OS下安装11.2.0.1版本Oracle的时候会出现此错误。

解决如下：

在执行root.sh时，在出现Adding daemon to inittab时，需要不停地执行命令：

/bin/dd if=/var/tmp/.oracle/npohasd of=/dev/null bs=1024 count=1

[root@rac1 grid]# ./root.sh
Running Oracle 11g root.sh script...

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2015-01-06 23:46:35: Parsing the host name
2015-01-06 23:46:35: Checking for super user privileges
2015-01-06 23:46:35: User has super user privileges
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
Adding daemon to inittab

CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
ADVM/ACFS is not supported on centos-release-6-6.el6.centos.12.2.x86_64

CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded

ASM created and started successfully.

DiskGroup DATA created successfully.

clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk 60b5c5168a574f3fbfabf7d93a17c18f.
Successful addition of voting disk a59fab87e8634f00bf5b44656d6caaf1.
Successful addition of voting disk 228f0986e7c84f20bf66ed6905b69e43.
Successfully replaced voting disk group with +DATA.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   60b5c5168a574f3fbfabf7d93a17c18f (ORCL:VOL1) [DATA]
 2. ONLINE   a59fab87e8634f00bf5b44656d6caaf1 (ORCL:VOL2) [DATA]
 3. ONLINE   228f0986e7c84f20bf66ed6905b69e43 (ORCL:VOL4) [DATA]
Located 3 voting disk(s).
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'rac1'
CRS-2676: Start of 'ora.evmd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.DATA.dg' on 'rac1'
CRS-2676: Start of 'ora.DATA.dg' on 'rac1' succeeded

rac1     2015/01/06 23:52:15     /u01/app/11.2.0/grid/cdata/rac1/backup_20150106_235215.olr
Preparing packages for installation...
cvuqdisk-1.0.7-1
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Updating inventory properties for clusterware
Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 3906 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /u01/app/oraInventory
'UpdateNodeList' was successful.

root.sh执行完毕。在rac2节点执行也需要此步骤。

检查grid组件启动情况
[grid@rac1 ~]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[grid@rac1 ~]$ crs_stat -v -t
Name           Type           R/RA   F/FT   Target    State     Host
----------------------------------------------------------------------
ora.DATA.dg    ora....up.type 0/5    0/     ONLINE    ONLINE    rac1
ora....ER.lsnr ora....er.type 0/5    0/     ONLINE    ONLINE    rac1
ora....N1.lsnr ora....er.type 0/5    0/0    ONLINE    ONLINE    rac1
ora.asm        ora.asm.type   0/5    0/     ONLINE    ONLINE    rac1
ora.eons       ora.eons.type  0/3    0/     ONLINE    ONLINE    rac1
ora.gsd        ora.gsd.type   0/5    0/     OFFLINE   OFFLINE
ora....network ora....rk.type 0/5    0/     ONLINE    ONLINE    rac1
ora.oc4j       ora.oc4j.type  0/5    0/0    OFFLINE   OFFLINE
ora.ons        ora.ons.type   0/3    0/     ONLINE    ONLINE    rac1
ora....SM1.asm application    0/5    0/0    ONLINE    ONLINE    rac1
ora....C1.lsnr application    0/5    0/0    ONLINE    ONLINE    rac1
ora.rac1.gsd   application    0/5    0/0    OFFLINE   OFFLINE
ora.rac1.ons   application    0/3    0/0    ONLINE    ONLINE    rac1
ora.rac1.vip   ora....t1.type 0/0    0/0    ONLINE    ONLINE    rac1
ora....SM2.asm application    0/5    0/0    ONLINE    ONLINE    rac2
ora....C2.lsnr application    0/5    0/0    ONLINE    ONLINE    rac2
ora.rac2.gsd   application    0/5    0/0    OFFLINE   OFFLINE
ora.rac2.ons   application    0/3    0/0    ONLINE    ONLINE    rac2
ora.rac2.vip   ora....t1.type 0/0    0/0    ONLINE    ONLINE    rac2
ora.scan1.vip  ora....ip.type 0/0    0/0    ONLINE    ONLINE    rac1

备注：

在重启节点后出现Oracle has不能通信的情况

1，[root@rac1 ~]# crsctl check crs
CRS-4639: Could not contact Oracle High Availability Services

需要在启动时执行 /bin/dd if=/var/tmp/.oracle/npohasd of=/dev/null bs=1024 count=1。

然后再执行 crsctl check crs查看服务启动情况。

2，如果出现以下情况，需要查看日志提示，有一次是因为内核参数配置值的问题导致了这个错误。

[root@rac1 bin]# ./crsctl check crs
 CRS-4638: Oracle High Availability Services is online
 CRS-4535: Cannot communicate with Cluster Ready Services
 CRS-4529: Cluster Synchronization Services is online
 CRS-4533: Event Manager is online

5，安装Oracle database软件

需要用oracle用户安装

su - oracle

./runInstaller

仅安装数据库软件即可，这里就根据OUI的提示安装就可以了，最后按照提示执行相关脚本。

找时间补上截图

6，创建ASM磁盘组

需要新建其他磁盘组来安装数据文件等

su - grid

asmca

根据界面提示创建磁盘组。

找时间补上截图

7，建立数据库实例

su - oracle

./runInstaller

根据OUI步骤提示选择rac选项新建数据库实例即可。

完成。

小结：通过实验了解RAC环境的搭建和工作条件，但是这里面还涉及到许多集群的理论知识，需要不断地区学习和巩固，然后手动去实践。

http://blog.csdn.net/seulk_Q/article/details/42612393