利用sqlldr从MySQL导出一张表数据到Oracle

根据业务需求,需要从MySQL库中同步一张表tap_application到Oracle中,下面是记录的导入过程.

1. 查看MySQL表结构

desc tap_application;
+---------------------+---------------+------+-----+---------------------+----------------+
| Field               | Type          | Null | Key | Default             | Extra          |
+---------------------+---------------+------+-----+---------------------+----------------+
| ID                  | int(11)       | NO   | PRI | NULL                | auto_increment |
| VENDORID            | int(11)       | NO   |     | NULL                |                |
| NAME                | varchar(128)  | NO   |     | NULL                |                |
| DESCRIPTION         | varchar(255)  | YES  |     | NULL                |                |
| CREATIONDATE        | datetime      | NO   |     | 0000-00-00 00:00:00 |                |
| MODIFICATIONDATE    | datetime      | NO   |     | 0000-00-00 00:00:00 |                |
| STATUS              | int(11)       | NO   |     | NULL                |                |
| URL                 | varchar(1024) | YES  |     | NULL                |                |
| APPTYPE             | smallint(6)   | YES  |     | NULL                |                |
| TEST_KEY            | varchar(512)  | YES  |     | NULL                |                |
| PRODUCTION_KEY      | varchar(512)  | YES  |     | NULL                |                |
| TEST_APISUSED       | varchar(256)  | YES  |     | NULL                |                |
| PRODUCTION_APISUSED | varchar(256)  | YES  |     | NULL                |                |
+---------------------+---------------+------+-----+---------------------+----------------+
13 rows in set (0.01 sec)

 2. 查看表是否有特殊字符

select * from tap_application where ID=1 or ID=55\G
*************************** 1. row ***************************
                 ID: 1
           VENDORID: 1
               NAME: app name
        DESCRIPTION: NULL
       CREATIONDATE: 2009-03-30 18:40:36
   MODIFICATIONDATE: 2009-03-30 18:40:36
             STATUS: 1
                URL: NULL
            APPTYPE: NULL
           TEST_KEY: AQAAASBaL2Igf/////////8AAAABAAAAAQEAAAAQnPDr8PHxexN/hwVcTYQL9AEAAAAOAwAAAAEAAAABAAAAAQA=
     PRODUCTION_KEY: NULL
      TEST_APISUSED: NULL
PRODUCTION_APISUSED: NULL
*************************** 2. row ***************************
                 ID: 55
           VENDORID: 52
               NAME: Raj Rao's test 1 application.
        DESCRIPTION: <script type='text/javascript'>
      alert("I am an alert box!");
</script>

       CREATIONDATE: 2010-04-09 15:20:53
   MODIFICATIONDATE: 2010-06-04 09:09:01
             STATUS: 1
                URL: http://somewhere.a.b
            APPTYPE: 1
           TEST_KEY: NULL
     PRODUCTION_KEY: NULL
      TEST_APISUSED: NULL
PRODUCTION_APISUSED: NULL

#可以看到"TEST_KEY"字段有特殊字符存在,"DESCRIPTION"字段也有换行符等特殊字符存在,因此在导出时需要特别留意,实际上通过cat -v查看数据时可以看到windows换行符"\r\n"

3. 导出MySQL表数据到文本文件中

select count(*) from tap_application;
+----------+
| count(*) |
+----------+
|      847 |
+----------+
1 row in set (0.01 sec)

select
IFNULL(ID,''),
IFNULL(VENDORID,''),
IFNULL(NAME,''),
REPLACE(IFNULL(DESCRIPTION,''),'\r\n','/$'),
IFNULL(CREATIONDATE,''),
IFNULL(MODIFICATIONDATE,''),
IFNULL(STATUS,''),
IFNULL(URL,''),
IFNULL(APPTYPE,''),
IFNULL(TEST_KEY,''),
IFNULL(PRODUCTION_KEY,''),
IFNULL(TEST_APISUSED,''),
IFNULL(PRODUCTION_APISUSED,'')
into outfile '/usr/local/mysql/dba/exp/tap_application.txt' fields terminated by '||' optionally enclosed by '^' lines terminated by '\r\n' from tap_application;

Query OK, 847 rows affected (0.01 sec)

scp /usr/local/mysql/dba/exp/tap_application.txt oracle@10.189.102.118:/u01/app/oracle/dba/imp/

 4. 在Oracle数据库创建对应表结构

CREATE TABLE tap_application (
  ID                       INT NOT NULL PRIMARY KEY,
  VENDORID                 INT NOT NULL,
  NAME                     VARCHAR2(128) NOT NULL,
  DESCRIPTION              VARCHAR2(255),
  CREATIONDATE             DATE NOT NULL,
  MODIFICATIONDATE         DATE NOT NULL,
  STATUS                   INT NOT NULL,
  URL                      VARCHAR2(1024),
  APPTYPE                  INT,
  TEST_KEY                 VARCHAR2(512),
  PRODUCTION_KEY           VARCHAR2(512),
  PRODUCTION_APISUSED      VARCHAR2(256)
);

 5. 创建sqlldr控制文件

$ cat /u01/app/oracle/dba/imp/load_tap_application.ctl
load DATA
INFILE '/u01/app/oracle/dba/imp/tap_application.txt'
BADFILE '/u01/app/oracle/dba/imp/tap_application.bad'
DISCARDFILE '/u01/app/oracle/dba/imp/tap_application.dsc'
TRUNCATE
INTO TABLE TAP_APPLICATION
fields terminated by '||' optionally enclosed by '^'
TRAILING NULLCOLS
(
ID integer external,
VENDORID integer external,
NAME char(128),
DESCRIPTION char(255) "replace(:DESCRIPTION,'/$','\r\n')",
CREATIONDATE date "YYYY-MM-DD HH24:MI:SS",
MODIFICATIONDATE date "YYYY-MM-DD HH24:MI:SS",
STATUS integer external,
URL char(1024),
APPTYPE integer external,
TEST_KEY char(512),
PRODUCTION_KEY char(512),
PRODUCTION_APISUSED char(256)
)

 6. 导入数据到Oracle

$ $ORACLE_HOME/bin/sqlldr system/888888 control=/u01/app/oracle/dba/imp/load_tap_application.ctl errors=50000 log=/u01/app/oracle/dba/imp/load_tap_application.log

SQL*Loader: Release 11.2.0.4.0 - Production on Fri Jul 7 08:33:39 2017

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

Commit point reached - logical record count 60
Commit point reached - logical record count 120
Commit point reached - logical record count 180
Commit point reached - logical record count 240
Commit point reached - logical record count 300
Commit point reached - logical record count 360
Commit point reached - logical record count 420
Commit point reached - logical record count 480
Commit point reached - logical record count 540
Commit point reached - logical record count 600
Commit point reached - logical record count 660
Commit point reached - logical record count 720
Commit point reached - logical record count 780
Commit point reached - logical record count 840
Commit point reached - logical record count 847

7. 查看导入的log信息

$ cat /u01/app/oracle/dba/imp/load_tap_application.log 

SQL*Loader: Release 11.2.0.4. - Production on Fri Jul  :: 

Copyright (c) , , Oracle and/or its affiliates.  All rights reserved.

Control File:   /u01/app/oracle/dba/imp/load_tap_application.ctl
Data File:      /u01/app/oracle/dba/imp/tap_application.txt
  Bad File:     /u01/app/oracle/dba/imp/tap_application.bad
  Discard File: /u01/app/oracle/dba/imp/tap_application.dsc
 (Allow all discards)

Number to load: ALL
Number to skip:
Errors allowed:
Bind array:      rows, maximum of  bytes
Continuation:    none specified
Path used:      Conventional

Table TAP_APPLICATION, loaded from every logical record.
Insert option in effect for this table: TRUNCATE
TRAILING NULLCOLS option in effect

   Column Name                  Position   Len  Term Encl Datatype
------------------------------ ---------- ----- ---- ---- ---------------------
ID                                  FIRST     *      O(^) CHARACTER
    Terminator string : '||'
VENDORID                             NEXT     *      O(^) CHARACTER
    Terminator string : '||'
NAME                                 NEXT         O(^) CHARACTER
    Terminator string : '||'
DESCRIPTION                          NEXT         O(^) CHARACTER
    Terminator string : '||'
    SQL string for column : "replace(:DESCRIPTION,'/$','
')"
CREATIONDATE                         NEXT     *      O(^) DATE YYYY-MM-DD HH24:MI:SS
    Terminator string : '||'
MODIFICATIONDATE                     NEXT     *      O(^) DATE YYYY-MM-DD HH24:MI:SS
    Terminator string : '||'
STATUS                               NEXT     *      O(^) CHARACTER
    Terminator string : '||'
URL                                  NEXT        O(^) CHARACTER
    Terminator string : '||'
APPTYPE                              NEXT     *      O(^) CHARACTER
    Terminator string : '||'
TEST_KEY                             NEXT         O(^) CHARACTER
    Terminator string : '||'
PRODUCTION_KEY                       NEXT         O(^) CHARACTER
    Terminator string : '||'
PRODUCTION_APISUSED                  NEXT         O(^) CHARACTER
    Terminator string : '||'

value used for ROWS parameter changed from  to 

Table TAP_APPLICATION:
   Rows successfully loaded.
   Rows not loaded due to data errors.
   Rows not loaded because all WHEN clauses were failed.
   Rows not loaded because all fields were null.

Space allocated for bind array:                  bytes( rows)
Read   buffer bytes: 

Total logical records skipped:
Total logical records read:
Total logical records rejected:
Total logical records discarded:        

Run began on Fri Jul  ::
Run ended on Fri Jul  :: 

Elapsed time was:     ::00.44
CPU time was:         ::00.01