环境:Linux + Oracle 11.2.0.1 ADG

现象:发现备库没有应用日志

1. 数据库查询备库目前状态

发现备库目前没有应用日志,apply lag已经显示备库有3天21小时多没有应用日志。

SQL> set linesize 1200
SQL> SELECT OPEN_MODE, DATABASE_ROLE, SWITCHOVER_STATUS, FORCE_LOGGING, DATAGUARD_BROKER, GUARD_STATUS FROM V$DATABASE; OPEN_MODE DATABASE_ROLE SWITCHOVER_STATUS FOR DATAGUAR GUARD_S
-------------------- ---------------- -------------------- --- -------- -------
READ ONLY PHYSICAL STANDBY NOT ALLOWED YES DISABLED NONE SQL> select * from v$dataguard_stats; NAME VALUE UNIT TIME_COMPUTED DATUM_TIME
-------------------------------- ---------------------------------------------------------------- ------------------------------ ------------------------------ ------------------------------
transport lag +00 00:00:00 day(2) to second(0) interval 01/17/2017 16:07:12 01/17/2017 16:07:12
apply lag +03 21:34:49 day(2) to second(0) interval 01/17/2017 16:07:12 01/17/2017 16:07:12
apply finish time +00 03:10:34.000 day(2) to second(3) interval 01/17/2017 16:07:12
estimated startup time 15 second 01/17/2017 16:07:12

2. 查询alert告警日志

从alert告警日志中定位到ADG出现问题的时刻,有600报错信息,进而导致MRP进程终止,详细日志如下:

Fri Jan 13 18:32:25 2017
Errors in file /home/oracle/opt/oracle/diag/rdbms/orcl/orcl/trace/orcl_pr03_22555.trc (incident=67480):
ORA-00600: internal error code, arguments: [kcbr_apply_change_11], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /home/oracle/opt/oracle/diag/rdbms/orcl/orcl/incident/incdir_67480/orcl_pr03_22555_i67480.trc
Slave exiting with ORA-600 exception
Errors in file /home/oracle/opt/oracle/diag/rdbms/orcl/orcl/trace/orcl_pr03_22555.trc:
ORA-00600: internal error code, arguments: [kcbr_apply_change_11], [], [], [], [], [], [], [], [], [], [], []
Fri Jan 13 18:32:26 2017
Errors in file /home/oracle/opt/oracle/diag/rdbms/orcl/orcl/trace/orcl_mrp0_22547.trc (incident=67448):
ORA-00600: internal error code, arguments: [kcbr_apply_change_11], [], [], [], [], [], [], [], [], [], [], []
Incident details in: /home/oracle/opt/oracle/diag/rdbms/orcl/orcl/incident/incdir_67448/orcl_mrp0_22547_i67448.trc
Fri Jan 13 18:32:26 2017
Trace dumping is performing id=[cdmp_20170113183226]
Recovery Slave PR03 previously exited with exception 600
Fri Jan 13 18:32:27 2017
MRP0: Background Media Recovery terminated with error 448
Errors in file /home/oracle/opt/oracle/diag/rdbms/orcl/orcl/trace/orcl_pr00_22549.trc:
ORA-00448: normal completion of background process
Managed Standby Recovery not using Real Time Apply
Recovery interrupted!
Fri Jan 13 18:32:27 2017
Sweep [inc][67480]: completed
Sweep [inc][67480]: completed
Recovered data files to a consistent state at change 2010287982
Errors in file /home/oracle/opt/oracle/diag/rdbms/orcl/orcl/trace/orcl_pr00_22549.trc:
ORA-00448: normal completion of background process
Errors in file /home/oracle/opt/oracle/diag/rdbms/orcl/orcl/trace/orcl_mrp0_22547.trc:
ORA-00600: internal error code, arguments: [kcbr_apply_change_11], [], [], [], [], [], [], [], [], [], [], []
MRP0: Background Media Recovery process shutdown (orcl)
Sweep [inc][67448]: completed
Sweep [inc2][67480]: completed
Sweep [inc2][67448]: completed
Trace dumping is performing id=[cdmp_20170113183227]
Fri Jan 13 18:33:04 2017
Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST

3. 尝试手工启动备库MRP恢复进程

发现手工启动备库MRP恢复进程,告警日志中依然会报出相同ORA-600 [kcbr_apply_change_11]错误。

4. 尝试mount状态启动MRP恢复进程

发现在mount状态下,可以正常启动MRP恢复进程,等恢复完成后,重新开启ADG实时应用,一切正常。

shutdown immediate
startup mount
alter database recover managed standby database disconnect from session;
此时等待恢复完成...
alter database recover managed standby database cancel;
alter database open;
alter database recover managed standby database using current logfile disconnect from session;

查询备库状态确认一切正常:

SQL> SELECT OPEN_MODE, DATABASE_ROLE, SWITCHOVER_STATUS, FORCE_LOGGING, DATAGUARD_BROKER, GUARD_STATUS FROM V$DATABASE; 

OPEN_MODE            DATABASE_ROLE    SWITCHOVER_STATUS    FOR DATAGUAR GUARD_S
-------------------- ---------------- -------------------- --- -------- -------
READ ONLY WITH APPLY PHYSICAL STANDBY NOT ALLOWED YES DISABLED NONE SQL> select * from v$dataguard_stats; NAME VALUE UNIT TIME_COMPUTED DATUM_TIME
-------------------------------- ---------------------------------------------------------------- ------------------------------ ------------------------------ ------------------------------
transport lag +00 00:00:00 day(2) to second(0) interval 01/17/2017 17:42:26 01/17/2017 17:42:26
apply lag +00 00:00:00 day(2) to second(0) interval 01/17/2017 17:42:26 01/17/2017 17:42:26
apply finish time +00 00:00:00.000 day(2) to second(3) interval 01/17/2017 17:42:26
estimated startup time 18 second 01/17/2017 17:42:26

5. 查询MOS,定位根本原因

查询MOS发现该现象与bug 10419984 相匹配

Bug 10419984 : ACTIVE DATA GUARD STANDBY GIVES ORA-600 [KCBR_APPLY_CHANGE_11]

建议应用该补丁防止该问题再次被触发。

Oracle 11.2.0.1 ADG环境MRP进程遭遇ORA-600异常终止的更多相关文章

  1. Oracle 11.2.0.1 ADG环境MRP进程遭遇ORA

    环境:Linux + Oracle 11.2.0.1 ADG现象:发现备库没有应用日志 1. 数据库查询备库目前状态发现备库目前没有应用日志,apply lag已经显示备库有3天21小时多没有应用日志 ...

  2. Oracle 11.2.0.4 DataGuard 环境打PSU,OJVM PSU补丁快速参考

    环境:RHEL6.5 + Oracle 11.2.0.4 DataGuard physical standby 主库和备库都是单节点. 需求:主备库同时应用160719的PSU和OJVM PSU补丁. ...

  3. 【Oracle】11G 11.2.0.4 RAC环境打补丁

    一.准备工作 1,数据库环境 操作系统版本  : RedHat 7.2 x64   数据库版本    : Oracle 11.2.0.4 x64 RAC    Grid          : 11.2 ...

  4. Oracle 11.2.0.4单实例打PSU,OJVM PSU补丁快速参考

    写在前面: 1.Oracel打每个补丁的操作有时存在差异,所以不管多熟悉,都应该在打任何补丁之前阅读新补丁中附带的readme. 2.Oracle每季度都会更新一个最新的PSU,本文最新指的是当前最新 ...

  5. Oracle 11.2.0.4 RAC安装最新PSU补丁

    环境:两节点RAC(RHEL 6.4 + GI 11.2.0.4 + Oracle 11.2.0.4) 需求:安装最新PSU补丁11.2.0.4.7 1.下载补丁和最新OPatch 2.检查数据库当前 ...

  6. Linux 6.5(oracle 11.2.0.4)单实例ASM安装

    Linux 6.5(oracle 11.2.0.4) 1.解析主机.配置网络等 /etc/hosts /etc/sysconfig/network /etc/init.d/NetworkManager ...

  7. ORACLE LINUX 6.3 + ORACLE 11.2.0.3 RAC + VBOX安装文档

    ORACLE LINUX 6.3 + ORACLE 11.2.0.3 RAC + VBOX安装文档 2015-10-21 12:51 525人阅读 评论(0) 收藏 举报  分类: Oracle RA ...

  8. oracle 11.2.0.4单实例文件系统安装与补丁

    [TOC] 一,预安装处理 1.版本准备 操作系统:RHEL 6.5 数据库版本:Oracle 11.2.0.4 相关包:p13390677_112040_Linux-x86-64_1of7.zip  ...

  9. centos6.5 x86_64安装oracle 11.2.0.3grid

     centos6.5 x86_64安装oracle 11.2.0.3grid 1.安装前的准备 工作 1.1.配置node1 1.1.1.配置虚拟机并安装centos 安装node1----- 1 ...

随机推荐

  1. [leetcode-592-Fraction Addition and Subtraction]

    Given a string representing an expression of fraction addition and subtraction, you need to return t ...

  2. android app调试没问题,但打包签名的apk,运行时出现闪退怎么办?

    在用Eclipse编写Android app时,有时调试时没有问题,但一经打包签名,运行就出现闪退,还报错说找不到某某类.一开始以为是混淆导致的,后来我没有混淆竟然也还是这个问题.无奈只得网上寻找解决 ...

  3. Spring Boot1.5.4 AOP实例

    原文:https://github.com/x113773/testall/issues/12 1. 还是首先添加依赖(使用当前springboot的默认版本)```<dependency> ...

  4. C++ 宏定义#define 中##的使用

    在C++的宏定义中,符号##一般是用于连接,包括参数的连接,参数与标识符的连接等,然后形成一个新的标识符. 下面举几个例子来进行说明. eg1: #define ADD(a,b) a##b #defi ...

  5. *bzoj1083题解

    题目: 城市C是一个非常繁忙的大都市,城市中的道路十分的拥挤,于是市长决定对其中的道路进行改造.城市C的道路是这样分布的:城市中有n个交叉路口,有些交叉路口之间有道路相连,两个交叉路口之间最多有一条道 ...

  6. java 重载与重写 【转】

    首先我们来讲讲:重载(Overloading) (1) 方法重载是让类以统一的方式处理不同类型数据的一种手段.多个同名函数同时存在,具有不同的参数个数/类型. 重载Overloading是一个类中多态 ...

  7. 使用ant自动构建apk

    最近因渠道过多,需要单独接入渠道支付sdk的渠道也很多,而首发在即.人手不足,所以着手了部分相关的工作,看了下目前的操作流程..无奈人比较懒,所以决定进行一波简化的技术,先考虑到了两种方案: 1)脚本 ...

  8. str-字符串功能介绍

    叨逼叨:字符串的各个功能修改不是本身,本身不变,会产生新的值,需要赋值给新的变量来接收 以下 "举例" 是解释每个功能的实例   "举例"下一行是pycharm ...

  9. ActiveMQ结合WebScoket应用例子以及介绍

    一.ActiveMQ的介绍? 1.JMS基础概念 JMS(java Message Service) 即使java消息服务,它提供标准的产生.发送.接收的接口简化企业应用开发,它支持两种消息通信模型: ...

  10. FreeRTOS——资源管理

    1. 多任务系统存在一个潜在的风险:资源管理. 2. 基本临界区:taskENTER_CRITICAL() 与 taskEXIT_CRITICAL() 或 taskENTER_CRITICAL_FRO ...