Oracle CDC简介及异步在线日志CDC部署示例
摘要
1. Oracle CDC 简介
1.1 Synchronous Change Data Capture Configuration(同步复制)
1.2 Asynchronous HotLog Configuration(异步在线日志CDC)
1.3 Asynchronous Distributed HotLog Configuration(异步分布式CDC)
1.4 Asynchronous Autolog Online Change Data Capture Configuration(异步在线日志复制CDC)
1.5 Asynchronous AutoLog Archive Change Data Capture Configuration(归档日志CDC)
2. 异步在线日志CDC环境部署
2.1 环境配置准备
SQL> select * from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
PL/SQL Release 11.2.0.1.0 - Production
CORE 11.2.0.1.0 Production
TNS for Linux: Version 11.2.0.1.0 - Production
NLSRTL Version 11.2.0.1.0 - Production
SQL> alter system set streams_pool_size=50m;
System altered.
SQL> alter system set java_pool_size=50m;
System altered.
SQL> alter system set undo_retention=3600;
System altered.
SQL> show parameter streams_pool
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
streams_pool_size big integer 52M
SQL> show parameter java_pool
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
java_pool_size big integer 52M
SQL> show parameter undo_re
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
undo_retention integer 3600
SQL> archive log list
Database log mode Archive Mode
Automatic archival Enabled
Archive destination USE_DB_RECOVERY_FILE_DEST
Oldest online log sequence 487
Next log sequence to archive 489
Current log sequence 489
SQL> alter database force logging;
Database altered.
SQL> alter database add supplemental log data;
Database altered.
SQL> select LOG_MODE,FORCE_LOGGING,SUPPLEMENTAL_LOG_DATA_MIN from v$database;
LOG_MODE FOR SUPPLEME
------------ --- --------
ARCHIVELOG YES YES
SQL> create table employee_info(n number,name varchar(20),address varchar(150),department varchar(120),organization varchar(150)) tablespace datafile1;
Table created.
SQL> insert into employee_info values(1, 'bendsha', 'lianhang road, shanghai, China', 'AnyBackup', 'EISOO');
1 row created.
SQL> insert into employee_info values(2, 'bendsha', 'lianhang road, shanghai, China', 'AnyBackup', 'EISOO');
1 row created.
2.2 创建发布者和订阅者
SQL> create tablespace cdc_datafile datafile '/u01/app/oracle/orcl/cdc_datafile.dbf' size 1G;
Tablespace created.
SQL> create user cdc_publisher identified by cdc_publisher default tablespace cdc_datafile temporary tablespace temp;
User created.
SQL> grant create session TO cdc_publisher;
Grant succeeded.
SQL> grant create table TO cdc_publisher;
Grant succeeded.
SQL> grant create sequence TO cdc_publisher;
Grant succeeded.
SQL> grant create procedure TO cdc_publisher;
Grant succeeded.
SQL> grant create any job TO cdc_publisher;
Grant succeeded.
SQL> grant execute_catalog_role TO cdc_publisher;
Grant succeeded.
SQL> grant select_catalog_role TO cdc_publisher;
Grant succeeded.
SQL> grant execute ON dbms_cdc_publish TO cdc_publisher;
Grant succeeded.
SQL> grant execute ON dbms_lock TO cdc_publisher;
Grant succeeded.
SQL> grant unlimited tablespace TO cdc_publisher;
Grant succeeded.
SQL> execute dbms_streams_auth.grant_admin_privilege('CDC_PUBLISHER');
PL/SQL procedure successfully completed.
SQL> grant all on backupuser.employee_info to cdc_publisher;
Grant succeeded.
SQL> create user cdc_subscriber identified by cdc_subscriber default tablespace cdc_datafile temporary tablespace temp;
User created.
SQL> grant create session TO cdc_subscriber;
Grant succeeded.
2.3 发布/订阅具体数据
SQL>
conn cdc_publisher/cdc_publisher
BEGIN
DBMS_CAPTURE_ADM.PREPARE_TABLE_INSTANTIATION(TABLE_NAME => 'backupuser.employee_info');
END;
Connected.
SQL> 2 3 4 /
PL/SQL procedure successfully completed.
SQL>
conn cdc_publisher/cdc_publisher
BEGIN
DBMS_CDC_PUBLISH.CREATE_CHANGE_SET(
change_set_name => 'cdc_employee_info_cs',
description => 'Change set for backupuser.employee_info info',
change_source_name => 'HOTLOG_SOURCE',
stop_on_ddl => 'y'
);
END;
Connected.
SQL> 2 3 4 5 6 7 8 9 /
PL/SQL procedure successfully completed.
SQL>
conn cdc_publisher/cdc_publisher
BEGIN
DBMS_CDC_PUBLISH.CREATE_CHANGE_TABLE(
owner => 'cdc_publisher',
change_table_name => 'employee_info_ct',
change_set_name => 'cdc_employee_info_cs',
source_schema => 'backupuser',
source_table => 'employee_info',
column_type_list =>'n number,name varchar(20),address varchar(150)',
capture_values => 'both',
rs_id => 'y',
row_id => 'n',
user_id => 'n',
timestamp => 'n',
object_id => 'n',
source_colmap => 'n',
target_colmap => 'y',
options_string =>'');
END;
19 /
PL/SQL procedure successfully completed.
SQL>
conn cdc_publisher/cdc_publisher
BEGIN
DBMS_CDC_PUBLISH.ALTER_CHANGE_SET(
change_set_name => 'cdc_employee_info_cs',
enable_capture => 'y');
END;
Connected.
SQL> 2 3 4 5 6 /
PL/SQL procedure successfully completed.
SQL>
conn cdc_publisher/cdc_publisher
GRANT SELECT ON cdc_publisher.employee_info_ct TO cdc_subscriber;
conn / as sysdba
GRANT CREATE TABLE TO cdc_subscriber;
GRANT CREATE SESSION TO cdc_subscriber;
GRANT CREATE VIEW TO cdc_subscriber;
- GRANT UNLIMITED TABLESPACE TO cdc_subscriber;
Grant succeeded.
SQL>
conn cdc_subscriber/cdc_subscriber
BEGIN
DBMS_CDC_SUBSCRIBE.CREATE_SUBSCRIPTION(
change_set_name => 'cdc_employee_info_cs',
description => 'Change data for employee_info',
subscription_name => 'employee_info_sub');
END;
Connected.
SQL> 2 3 4 5 6 7 /
PL/SQL procedure successfully completed.
SQL>
conn cdc_subscriber/cdc_subscriber
BEGIN
DBMS_CDC_SUBSCRIBE.SUBSCRIBE(
subscription_name => 'employee_info_sub',
source_schema => 'backupuser',
source_table => 'employee_info',
column_list => 'n,name,address',
subscriber_view => 'employee_info_view');
END;
Connected.
SQL> 2 3 4 5 6 7 8 9 /
PL/SQL procedure successfully completed.
SQL>
conn cdc_subscriber/cdc_subscriber
BEGIN
DBMS_CDC_SUBSCRIBE.ACTIVATE_SUBSCRIPTION(
subscription_name => 'employee_info_sub');
END;
Connected.
SQL> 2 3 4 5 /
PL/SQL procedure successfully completed.
SQL>
conn cdc_subscriber/cdc_subscriber
BEGIN
DBMS_CDC_SUBSCRIBE.EXTEND_WINDOW(
subscription_name => 'employee_info_sub');
END;
Connected.
SQL> 2 3 4 5 /
PL/SQL procedure successfully completed.
SQL>
conn cdc_subscriber/cdc_subscriber
Connected.
SQL> select * from employee_info_view;
no rows selected
2.4 测试发布/订阅
SQL>
conn backupuser/backupuser123
insert into employee_info values(1, 'bendsha', 'lianhang road, shanghai, China', 'SmartData', 'SmartDB');
insert into employee_info values(2, 'bendsha', 'lianhang road, shanghai, China', 'SmartData', 'SmartDB');
insert into employee_info values(3, 'bendsha', 'lianhang road, shanghai, China', 'SmartData', 'SmartDB');
update employee_info set name = 'zhuzi' where n = 2;
delete from employee_info where n = 1;
Connected.
SQL>
1 row created.
SQL>
1 row created.
SQL>
1 row created.
SQL>
1 row updated.
SQL>
1 row deleted.
SQL> commit;
Commit complete.
SQL>
conn cdc_publisher/cdc_publisher
Connected.
SQL> select OPERATION$,n,name,address from employee_info_ct;
OP N NAME
-- ---------- --------------------
ADDRESS
--------------------------------------------------------------------------------
I 1 bendsha
lianhang road, shanghai, China
I 2 bendsha
lianhang road, shanghai, China
I 3 bendsha
lianhang road, shanghai, China
OP N NAME
-- ---------- --------------------
ADDRESS
--------------------------------------------------------------------------------
UO 2 bendsha
lianhang road, shanghai, China
UN 2 zhuzi
lianhang road, shanghai, China
D 1 bendsha
lianhang road, shanghai, China
6 rows selected.
SQL>
conn cdc_subscriber/cdc_subscriber
BEGIN
DBMS_CDC_SUBSCRIBE.EXTEND_WINDOW(
subscription_name => 'employee_info_sub');
END;
Connected.
SQL> 2 3 4 5 /
PL/SQL procedure successfully completed.
SQL> select OPERATION$,n,name,address from employee_info_view;
OP N NAME
-- ---------- --------------------
ADDRESS
--------------------------------------------------------------------------------
I 1 bendsha
lianhang road, shanghai, China
I 2 bendsha
lianhang road, shanghai, China
I 3 bendsha
lianhang road, shanghai, China
OP N NAME
-- ---------- --------------------
ADDRESS
--------------------------------------------------------------------------------
UO 2 bendsha
lianhang road, shanghai, China
UN 2 zhuzi
lianhang road, shanghai, China
D 1 bendsha
lianhang road, shanghai, China
6 rows selected.
SQL>
conn cdc_subscriber/cdc_subscriber
BEGIN
DBMS_CDC_SUBSCRIBE.PURGE_WINDOW(
subscription_name => 'employee_info_sub');
END;
Connected.
SQL> 2 3 4 5 /
PL/SQL procedure successfully completed.
SQL> select OPERATION$,n,name,address from employee_info_view;
no rows selected
SQL> conn cdc_publisher/cdc_publisher
Connected.
SQL> truncate table employee_info_ct;
Table truncated.
SQL> select OPERATION$,n,name,address from employee_info_ct;
no rows selected
3. 常见问题解决方法
3.1 ORA-31466: 未找到发布内容
SQL> conn cdc_publisher/cdc_publisher
Connected.
SQL> grant select on cdc_employee_info to cdc_subscriber;
Grant succeeded.
3.2 激活订阅之后,对源表进行操作,捕获不到数据
4. 参考文档
Oracle CDC简介及异步在线日志CDC部署示例的更多相关文章
- Oracle_CDC异步Autolog online redo部署示例
一.CDC简介 Oracle CDC (Change Data Capture)变化数据捕获,是一种数据增量处理技术.CDC特性是在Oracle9i数据库中引入的.CDC能够帮助你识别从上次提取之后发 ...
- Oracle 物理结构(五) 文件-在线日志文件
一.什么是在线日志文件 默认情况下redo的块大小是磁盘的扇区大小,通常是512字节,但是现在很多磁盘开始支持4k的扇区,oracle能自动识别并使用4k的大小,但是使用4k的redo block会有 ...
- ORACLE DG在线日志修改
ORACLE DG在线日志修改 SQL>select SEQUENCE#,first_time,next_time,APPLIED, THREAD# from v$archived_log or ...
- Oracle RACDB 增加、删除 在线重做日志组
Oracle RACDB 增加.删除 在线重做日志组 select * from v$log;select * from v$logfile ; ----删除日志组:alter database dr ...
- PowerJob 在线日志饱受好评的秘诀:小但实用的分布式日志系统
本文适合有 Java 基础知识的人群 作者:HelloGitHub-Salieri HelloGitHub 推出的<讲解开源项目>系列. 项目地址: https://github.com/ ...
- Oracle数据库简介
Oracle数据库简介 一.介绍 Oracle数据库系统是美国Oracle(甲骨文)公司提供的以分布式数据库为核心的一组软件产品,是目前最流行的客户/服务器(Client/Server,C/S)或浏览 ...
- oracle体系结构简介
oracle体系结构简介 一.物理存储结构 1.数据文件 存放数据库数据,以dbf为扩展名.将数据放在多个数据文件中, 再将数据文件分放在不同的硬盘中,可以提高存取速度. ...
- 1.Oracle数据库简介
Oracle数据库简介 Oracle Database,又名Oracle RDBMS,或简称Oracle.是甲骨文公司的一款关系数据库管理系统.它是在数据库领域一直处于领先地位的产品.可以说Oracl ...
- 重复造轮子,编写一个轻量级的异步写日志的实用工具类(LogAsyncWriter)
一说到写日志,大家可能推荐一堆的开源日志框架,如:Log4Net.NLog,这些日志框架确实也不错,比较强大也比较灵活,但也正因为又强大又灵活,导致我们使用他们时需要引用一些DLL,同时还要学习各种用 ...
随机推荐
- 简聊iOS支付集成(支付宝和微信支付)
一.支付集成是什么 1.现在大部分app都有快捷支付功能,支付集成将第三方支付平台集成到自己的项目中,能够完成自己项目中的支付功能, 二.支付集成的使用 <1>.支付宝: 下载SDK和De ...
- 【redis专题(8)】命令语法介绍之通用KEY
select num 数据库选择 默认有16[0到15]个数据库,默认自动选择0号数据库 move key num 移动key到num服务器 del key [key ...] 删除给定的一个或多个 ...
- Hadoop之RPC简单使用(远程过程调用协议)
一.RPC概述 RPC是指远程过程调用,也就是说两台不同的服务器(不受操作系统限制),一个应用部署在Linux-A上,一个应用部署在Windows-B或Linux-B上,若A想要调用B上的某个方法me ...
- 【shell编程基础0】bash shell编程的基本配置
前面一篇“shell编程之变量篇”主要讲述下shell编程的变量的基本知识:设置变量的方式,自定义变量和环境变量的差别,变量的替换.删除.测试等. 这一篇主要是讲述在bash shell下的一些基本配 ...
- js 不要使用new
(1)不要使用new Array(),new Number, new String, or new Boolean. 等等 如果要新建数组,没有必要使用new Array(),使用[];原因是直观. ...
- vue2.0版cnode社区项目搭建及实战开发
_________________________________________________________________________ 初涉vue就深深的被vue强大的功能,快速的开发能力 ...
- Linq: Aggregate
Aggregate累加器 今天看东西的时候看见这么个扩展方法Aggregate(累加器)很是陌生,于是乎查了查,随手记录一下. 直接看一个最简答的版本,其他版本基本没什么区别,需要的时候可看一下 pu ...
- 商城项目实战 | 1.1 Android 仿京东商城底部布局的选择效果 —— Selector 选择器的实现
前言 本文为菜鸟窝作者刘婷的连载."商城项目实战"系列来聊聊仿"京东淘宝的购物商城"如何实现. 京东商城的底部布局的选择效果看上去很复杂,其实很简单,这主要是要 ...
- java异常处理机制(try-catch-finally)
/* * 异常处理机制 * 1.分类:Error和Exception * Error错误是JVM自动报错的,程序员无法解决例如开数组过大int a[]=new int [1024*1024*1024] ...
- 初识Android触摸事件传递机制
前言 今天总结的一个知识点是Andorid中View事件传递机制,也是核心知识点,相信很多开发者在面对这个问题时候会觉得困惑,另外,View的另外一个难题滑动冲突,比如在ScrollView中嵌套Li ...