为Linux版本Oracle 11gR2配置HugePage

HugePage是Oracle在Linux版本下一种性能优化手段。对于共享内存区域（SGA）的数据库系统，Oracle通常都推荐在操作系统层面配置上HugePage，为Oracle实例准备更大的可用共享内存和整块的内存。

本文主要介绍在Oracle 11gR2版本上，进行HugePage配置全过程。记录下来，共需要的朋友待查。

1、环境介绍

笔者使用Oracle 11gR2系统，具体版本为11.2.0.4。

SQL> select * from v$version;

BANNER

--------------------------------------------------------------------------------

Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production

PL/SQL Release 11.2.0.4.0 - Production

CORE     11.2.0.4.0     Production

TNS for Linux: Version 11.2.0.4.0 - Production

NLSRTL Version 11.2.0.4.0 – Production

当前数据库配置内存策略为ASMM，也就是分别设置PGA和SGA的大小范围。Oracle根据负载情况动态自己调节内部各个子池的空间。注意：在当前版本中，AMM（Automatic Memory Management）和HugePage之间还是不能兼容。所以，如果要求配置HugePage，就只能退而求其次选择ASMM。

数据库设置的SGA空间如下：

SQL> show parameter sga;

NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

lock_sga                             boolean     FALSE

pre_page_sga                         boolean     FALSE

sga_max_size                         big integer 12G

sga_target                           big integer 12G

12G的SGA，Oracle在启动过程中，针对大内存情况给出了提示在alert log中。

ue Jan 12 11:31:33 2016

Starting ORACLE instance (normal)

************************ Large Pages Information *******************

Per process system memlock (soft) limit = 64 KB

Total Shared Global Region in Large Pages = 0 KB (0%)

Large Pages used by this instance: 0 (0 KB)

Large Pages unused system wide = 0 (0 KB)

Large Pages configured system wide = 0 (0 KB)

Large Page size = 2048 KB

RECOMMENDATION:

Total System Global Area size is 12 GB. For optimal performance,

prior to the next instance restart:

1. Increase the number of unused large pages by

at least 6145 (page size 2048 KB, total size 12 GB) system wide to

get 100% of the System Global Area allocated with large pages

2. Large pages are automatically locked into physical memory.

Increase the per process memlock (soft) limit to at least 12 GB to lock

100% System Global Area's large pages into physical memory

首先Oracle发现当前系统并没有设置HugePage，只有默认的2M大小的Page可选。于是，从建议的角度，给出了提示意见，建议开启HugePage选项在操作系统层面。此时，操作系统层面也没有专门的设置。

[oracle@PROD_PC-DB trace]$ grep Huge /proc/meminfo

AnonHugePages:     36864 kB

HugePages_Total:       0

HugePages_Free:        0

HugePages_Rsvd:        0

HugePages_Surp:        0

Hugepagesize:       2048 kB

2、配置HugePage

在11.2的小版本中，HugePage处理的行为其实是在变化的。在11.2.0.2中，Oracle推出了一个参数use_large_page，作为从实例层面控制Oracle使用Huge Page的开关。

SQL> show parameter large_pages

NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

use_large_pages                      string      TRUE

该参数默认取值为true，是一种向上最优策略。如果设置为true，Oracle实例会去检查是否操作系统开启了HugePage，如果有就采用，如果没有就不用。另外两个取值分别为false和only，如果选择为false，表示Oracle实例不管操作系统设置，都不会使用HugePage。如果选择为only，就表示Oracle实例只会选择HugePage上运行。

此外，在行为上也有一些变化。在11.2.0.2版本中，如果操作系统没有足够的HugePage使用，SGA内存就只有使用系统剩下的小页（Small Pages）。如果余下的小页small page不够用，或者内核使用的比较多，系统会报错ora-4030错误。

这个情况在11.2.0.3版本中有一些变化。Oracle首先会使用可用的HugePage，如果使用尽了，同样会采用剩下的small pages作为SGA使用。变化在于对使用HugePage的方式上，根据官方的说法：

With this new behavior additional shared memory segments are an expected side effect. Part of the change is to ensure that each shared memory segment making up the SGA only contains sub-areas with an identical alignment requirement - hence the SGA will spread over more separate SHM segments. In this supported mixed page mode the database will exhaust the available hugepages, before switching to regular sized pages.

新版本的操作方式可以更好的使用可用的大页资源。当前操作系统版本为Linux 2.6内核版本。

[oracle@PROD_PC-DB trace]$ uname -r

2.6.32-431.el6.x86_64

这个过程中涉及到不同操作系统环境、不同Oracle配置下的推荐HugePage数量问题。Oracle在MOS上发布出一个Shell脚本hugepages_settings.sh，可以动态计算出推荐的HugePage数量。

具体脚本内容可以从MOS Doc ID 401749.1上进行下载，在笔者之前关于HugePage的文章中也有所涉及。执行脚本后结果如下：

[oracle@PROD_PC-DB trace]$ ls -l | grep hugepag

-rw-r--r--. 1 oracle oinstall    3129 Jan 12 13:32 hugepages_settings.sh

[oracle@PROD_PC-DB trace]$ chmod +x hugepages_settings.sh

[oracle@PROD_PC-DB trace]$ ls -l | grep hugepag

-rwxr-xr-x. 1 oracle oinstall    3129 Jan 12 13:32 hugepages_settings.sh

[oracle@PROD_PC-DB trace]$ uname -r

2.6.32-431.el6.x86_64

[oracle@PROD_PC-DB trace]$ ./hugepages_settings.sh

This script is provided by Doc ID 401749.1 from My Oracle Support

(http://support.oracle.com) where it is intended to compute values for

the recommended HugePages/HugeTLB configuration for the current shared

memory segments on Oracle Linux. Before proceeding with the execution please note following:

* For ASM instance, it needs to configure ASMM instead of AMM.

* The 'pga_aggregate_target' is outside the SGA and

you should accommodate this while calculating SGA size.

* In case you changes the DB SGA size,

as the new SGA will not fit in the previous HugePages configuration,

it had better disable the whole HugePages,

start the DB with new SGA size and run the script again.

And make sure that:

* Oracle Database instance(s) are up and running

* Oracle Database 11g Automatic Memory Management (AMM) is not setup

(See Doc ID 749851.1)

* The shared memory segments can be listed by command:

# ipcs -m

Press Enter to proceed...

Recommended setting: vm.nr_hugepages = 6148

操作系统层面配置HugePage，需要在两个层面进行配置。事先是limits.conf文件，开启对用户的内存限制。

[root@PROD_PC-DB security]# vi limits.conf

# /etc/security/limits.conf

#

#Each line describes a limit for a user in the form:

#

（篇幅原因，有省略……）

oracle soft nproc 2047

oracle hard nproc 16384

oracle soft nofile 1024

oracle hard nofile 65536

oracle soft memlock 32879476  --对应内存总量

oracle hard memlock 32879476

确认修改。

[oracle@PROD_PC-DB ~]$ ulimit -l

32879476

将脚本计算出的结果，修改入/etc/sysctl.conf文件。

net.ipv4.ip_local_port_range = 9000 65500

net.core.rmem_default = 262144

net.core.rmem_max = 4194304

net.core.wmem_default = 262144

net.core.wmem_max = 1048576

fs.aio-max-nr = 1048576

fs.file-max = 6815744

vm.nr_hugepages = 6148

"/etc/sysctl.conf" 67L, 1631C written

执行sysctl –p落实修改。

[root@PROD_PC-DB security]# sysctl -p

net.ipv4.ip_forward = 0

net.ipv4.conf.default.rp_filter = 1

（篇幅原因，有省略……）

fs.file-max = 6815744

vm.nr_hugepages = 6148

重新启动实例数据库。

SQL> conn / as sysdba

Connected.

SQL> shutdown immediate;

Database closed.

Database dismounted.

ORACLE instance shut down.

SQL> startup

ORACLE instance started.

Total System Global Area 1.2827E+10 bytes

Fixed Size                  2265224 bytes

Variable Size            2214596472 bytes

Database Buffers         1.0603E+10 bytes

Redo Buffers                7307264 bytes

Database mounted.

Database opened.

SQL>

在操作系统层面上，可以看到HugePage使用情况。

[oracle@PROD_PC-DB ~]$ ipcs -m

------ Shared Memory Segments --------

key        shmid      owner      perms      bytes      nattch     status

0x00000000 786435     oracle     640        67108864   22

0x00000000 819204     oracle     640        12817793024 22

0x617ab988 851973     oracle     640        2097152    22

[oracle@PROD_PC-DB ~]$ grep Huge /proc/meminfo

AnonHugePages:     20480 kB

HugePages_Total:    6148

HugePages_Free:     5531

HugePages_Rsvd:     5528

HugePages_Surp:        0

Hugepagesize:       2048 kB

同时，启动过程中的推荐提示信息消失。

Tue Jan 12 14:26:30 2016

Starting ORACLE instance (normal)

************************ Large Pages Information *******************

Per process system memlock (soft) limit = 31 GB

Total Shared Global Region in Large Pages = 12 GB (100%)

Large Pages used by this instance: 6145 (12 GB)

Large Pages unused system wide = 3 (6144 KB)

Large Pages configured system wide = 6148 (12 GB)

Large Page size = 2048 KB

********************************************************************

LICENSE_MAX_SESSION = 0

LICENSE_SESSIONS_WARNING = 0

Initial number of CPU is 8

Number of processor cores in the system is 8

Number of processor sockets in the system is 1

Picked latch-free SCN scheme 3

Using LOG_ARCHIVE_DEST_1 parameter default value as USE_DB_RECOVERY_FILE_DEST

Autotune of undo retention is turned on.

IMODE=BR

3、结论

对于投产在Linux版本下的Oracle共享内存系统而言，配置HugePage还是有现实意义的。

转：http://blog.itpub.net/17203031/viewspace-1975653/