在NAS设备上用NFS服务为RAC数据库和集群件存储oracle文件时的mount选项

今天在家折腾自己的小实验室，把自己NAS上的一个目录用NFS挂载到一套11g RAC的实验环境中。

当我在备份数据库到NAS上时，发现一个奇怪的问题，同样的目录下，默认backup 备份集的情况，备份是成功的，但如果使用backup as copy备份则会报错，现象如下：

RMAN> backup datafile 6 format '/public/backup/oradb23/dbs_d_jingyu.bak';

Starting backup at 25-MAR-17
using channel ORA_DISK_1
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00006 name=+DATA/jyzhao/datafile/dbs_d_jingyu.268.937515173
channel ORA_DISK_1: starting piece 1 at 25-MAR-17
channel ORA_DISK_1: finished piece 1 at 25-MAR-17
piece handle=/public/backup/oradb23/dbs_d_jingyu.bak tag=TAG20170325T210550 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:07
Finished backup at 25-MAR-17

Starting Control File and SPFILE Autobackup at 25-MAR-17
piece handle=/public/backup/oradb23/c-2520969382-20170325-01 comment=NONE
Finished Control File and SPFILE Autobackup at 25-MAR-17

RMAN> backup as copy datafile 6 format '/public/backup/oradb23/%u.dbf';

Starting backup at 25-MAR-17
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile copy
input datafile file number=00006 name=+DATA/jyzhao/datafile/dbs_d_jingyu.268.937515173
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of backup command on ORA_DISK_1 channel at 03/25/2017 21:07:30
ORA-19504: failed to create file "/public/backup/oradb23/0vs0200h.dbf"
ORA-27054: NFS file system where the file is created or resides is not mounted with correct options
Additional information: 3
Additional information: 12

RMAN>

看backup as copy的报错明显是告诉我们无法在/public/backup/oradb23/目录下创建文件，但目录肯定有写权限，不然备份集备份也一样不会成功。

使用oerr简单先查看该错误号的说明：

[oracle@oradb23 ~]$ oerr ora 27504
27504, 00000, "IPC error creating OSD context"
// *Cause:  This is an operating system/cluster interconnect error.
// *Action: Check the value of errno and contact Oracle Support Services.

也没有太多有价值的内容。

进而查看MOS，最终找到下面这篇文档，

Mount Options for Oracle files for RAC databases and Clusterware when used with NFS on NAS devices (文档 ID 359515.1)

RAC including RACone and single instance RAC

In the table below

Binaries is the shared mount points where the Oracle Home and CRS_HOME is installed.

Datafiles includes Online Logs, Controlfile and Datafiles

nfsvers and vers are identical on those OS platforms that has nfsvers. The ver option is an alternative to the nfsvers option. It is included for compatibility with other operating systems

Please note that the mount options on each of the following cells are applicable only to those type of files listed in the column heading.

For RMAN backup sets, image copies, and Data Pump dump files, the "NOAC" mount option should not be specified - that is because RMAN and Data Pump do not check this option and specifying this can adversely affect performance.

Due to Unpublished bug 5856342, it is necessary to use the following init.ora parameter when using NAS with all versions of RAC on Linux (x86 & X86-64 platforms) until 10.2.0.4. This bug is fixed and included in 10.2.0.4 patchset.

filesystemio_options = DIRECTIO

大概意思是存储数据文件的话，在mount时，还需要指定一些特定的选项：

--vi /etc/fstab
#192.168.1.196:/mnt/HD/HD_a2/Public     /public nfs rw
192.168.1.196:/mnt/HD/HD_a2/Public     /public nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,actimeo=0,vers=3,timeo=600

第一行注释的是之前我配置的NFS挂载，而实际如果要存放数据文件，则需要按照上面提到的MOS文档进行配置，也就是第二行内容。

修改完重新挂载/public, 再次测试backup as copy，已经可以正常备份了。

RMAN> backup as copy datafile 6 format '/public/backup/oradb23/%u.dbf';

Starting backup at 25-MAR-17
using channel ORA_DISK_1
channel ORA_DISK_1: starting datafile copy
input datafile file number=00006 name=+DATA/jyzhao/datafile/dbs_d_jingyu.268.937515173
output file name=/public/backup/oradb23/17s022f8.dbf tag=TAG20170325T214928 RECID=16 STAMP=939592197
channel ORA_DISK_1: datafile copy complete, elapsed time: 00:00:35
Finished backup at 25-MAR-17

Starting Control File and SPFILE Autobackup at 25-MAR-17
piece handle=/public/backup/oradb23/c-2520969382-20170325-03 comment=NONE
Finished Control File and SPFILE Autobackup at 25-MAR-17

至此，就可以正常backup as copy到我的NAS设备中了。