MySQL Percona Toolkit--pt-osc重点参数

修改命令参数alter

1、不需要包含alter table关键字，可以包含多个修改操作，使用逗号分开，如"drop clolumn c1, add column c2 int"
2、不支持rename语句来对表进行重命名操作
3、不支持对索引进行重命名操作
4、如果删除外键，需要对外键名加下划线，如删除外键fk_uid, 修改语句为"DROP FOREIGN KEY _fk_uid"
5、重命名字段，不能使用drop add方式，会导致数据丢失，使用“change col1 col1_new type constraint"，保持数据和约束一致。

版本检查参数

在使用PT-OSC对阿里云RDS上表进行操作时，如未指定--noversion-check参数，会报如下错误：
Can't use an undefined value as an ARRAY reference at /usr/bin/pt-online-schema-change line 7547.

版本检查参数：
--[no]version-check
Check for the latest version of Percona Toolkit, MySQL, and other programs (default yes)

外键参数alter-foreign-keys-method

当对“被其他表外键关联的表”做修改时，RENAME操作会导致外键关联失败而不允许RENAME执行。
pt-osc提供--alter-foreign-keys-method选项：rebuild_constraints/drop_swap/auto/none

假设 t1 是要修改的表，t2 有外键依赖于 t1，_t1_new 是pt-osc工具产生的新临时表。

rebuild_constraints实现方式：
1、删除T2上的外键约束
2、对T1和_t1_new进行重命令
3、新增T2上的外键约束

drop_swap实现方式：
1、禁用T2的外键约束检查，FOREIGN_KEY_CHECKS=0，
2、DROP原始表t1，对_t1_new进行重命令
3、开启T2的外键约束检查

none实现方式：
1、禁用T2的外键约束检查，FOREIGN_KEY_CHECKS=0，
2、对表t1和_t1_new进行重命令
3、开启T2的外键约束检查

auto实现方式：
自动决定使用rebuild_constraints或drop_swap

none和drop swap区别在于none方式不删除原表

使用none参数生成的日志：

2019-10-10T07:58:42.430688-05:00     3430 Query    RENAME TABLE `dadaabc_test`.`tb001` TO `dadaabc_test`.`_tb001_old`, `dadaabc_test`.`_tb001_new` TO `dadaabc_test`.`tb001`
2019-10-10T07:58:42.451818-05:00     3430 Query    SET foreign_key_checks=0
2019-10-10T07:58:42.451884-05:00     3430 Query    DROP TABLE IF EXISTS `dadaabc_test`.`_tb001_old`
2019-10-10T07:58:42.740162-05:00     3430 Query    DROP TRIGGER IF EXISTS `dadaabc_test`.`pt_osc_dadaabc_test_tb001_del`
2019-10-10T07:58:42.740738-05:00     3430 Query    DROP TRIGGER IF EXISTS `dadaabc_test`.`pt_osc_dadaabc_test_tb001_upd`
2019-10-10T07:58:42.741216-05:00     3430 Query    DROP TRIGGER IF EXISTS `dadaabc_test`.`pt_osc_dadaabc_test_tb001_ins`
2019-10-10T07:58:42.741832-05:00     3430 Query    SHOW TABLES FROM `dadaabc_test` LIKE '\_tb001\_new'

如果有表tb002依赖于tb001,那么就会导致修改tb001完成后tb002的建表语句变化为：

CREATE TABLE `tb002` (
  `ID` INT(11) NOT NULL AUTO_INCREMENT,
  `C1` INT(11) DEFAULT NULL,
  `C2` INT(11) DEFAULT NULL,
  `C3` INT(11) DEFAULT NULL,
  PRIMARY KEY (`ID`),
  KEY `FK_C1` (`C1`),
  CONSTRAINT `_FK_C1` FOREIGN KEY (`C1`) REFERENCES `_tb001_old` (`ID`)
) ENGINE=INNODB AUTO_INCREMENT=6356899 DEFAULT CHARSET=utf8

正因如此，在使用alter-foreign-keys-method=none时，会有如下警告：

Using alter-foreign-keys-method = "none". This will typically cause foreign key violations!
This method of handling foreign key constraints is only provided so that the database administrator can disable the tool’s built-in functionality if desired.

如果使用alter-foreign-keys-method=drop_swap时，不会对原表执行rename操作，也不会导致依赖该表的外键受影响，其执行日志为：

2019-10-10T07:48:06.695144-05:00     3428 Query    ANALYZE TABLE `dadaabc_test`.`_tb001_new` /* pt-online-schema-change */
2019-10-10T07:48:06.706498-05:00     3428 Query    SET foreign_key_checks=0
2019-10-10T07:48:06.707180-05:00     3428 Query    DROP TABLE IF EXISTS `dadaabc_test`.`tb001`
2019-10-10T07:48:06.731187-05:00     3428 Query    RENAME TABLE `dadaabc_test`.`_tb001_new` TO `dadaabc_test`.`tb001`
2019-10-10T07:48:06.734388-05:00     3428 Query    DROP TRIGGER IF EXISTS `dadaabc_test`.`pt_osc_dadaabc_test_tb001_del`
2019-10-10T07:48:06.736416-05:00     3428 Query    DROP TRIGGER IF EXISTS `dadaabc_test`.`pt_osc_dadaabc_test_tb001_upd`
2019-10-10T07:48:06.736893-05:00     3428 Query    DROP TRIGGER IF EXISTS `dadaabc_test`.`pt_osc_dadaabc_test_tb001_ins`
2019-10-10T07:48:06.737513-05:00     3428 Query    SHOW TABLES FROM `dadaabc_test` LIKE '\_tb001\_new'

主机性能压力检查参数

在每次考完完成一个chunk数据后，会在主库上执行SHOW GLOBAL STATUS来获取当前的运行状态，判断是否继续拷贝数据和是否取消执行。
--critical-load=(type: Array; default: Threads_running=50)
Examine SHOW GLOBAL STATUS after every chunk, and abort if the load is too high (default Threads_running=50)
--max-load=(type: Array; default: Threads_running=25)
Examine SHOW GLOBAL STATUS after every chunk, and pause if any status variables are higher than their thresholds (default Threads_running=25)

参数--max-load不仅可以执行Threads_running状态的阈值，还可以指定其他状态阈值，如果未指定，则会默认使用当前值的120%作为阈值，如假设未指定Threads_connected参数，且当前Threads_connected参数值为100，则当Threads_connected参数值超过120时，会暂停拷贝。

当状态值超过--critical-load阈值后，会暂停拷贝数据并休眠一个复制周期(--chunk-time值)，
当状态值超过--critical-load阈值后，会取消pt-osc工具继续执行并清理中间数据。

从库复制延迟检查参数

--recursion-method
Preferred recursion method for discovering replicas (default processlist,hosts)
选择通过何种方式获取从库信息，可选参数：
processlist  SHOW PROCESSLIST
hosts        SHOW SLAVE HOSTS
dsn=DSN      DSNs from a table
none         Do not find slaves

--max-lag(type: time; default: 1s)
Pause the data copy until all replicas' lag is less than this value (default 1s).
Optional suffix s=seconds, m=minutes, h=hours, d=days; if no suffix, s is used.
默认为1s，每个chunks拷贝完成后，会查看check-slave-lag参数所指定的从库的延迟信息，如果超过max-log的阀值，则暂停复制数据，直到复制延迟小于max-log的阀值。
检查复制延迟信息依赖于SHOW SLAVE STATUS语句中返回的Seconds_Behind_Master列的值。

--check-interval(type: time; default: 1s)
Sleep time between checks for --max-lag (default 1).
Optional suffix s=seconds, m=minutes, h=hours, d=days; if no suffix, s is used.
当出现复制延迟暂停复制数据后，按照check-interval指定的时间进行周期检查复制延迟。

--check-slave-lag=s
Pause the data copy until this replica's lag is less than --max-lag
需要检查复制延迟的从库IP
如果指定check-slave-lag参数，且从库无法正常连接或从库IO线程和SQL线程停止，会认为主从存在延迟，导致复制数据操作一直暂停。
如果未指定check-slave-lag参数，默认会检查从库的延迟，但复制延迟不会导致数据复制暂停。

--skip-check-slave-lag=d
DSN to skip when checking slave lag
不需要检查复制延迟的从库IP

--slave-user=s
Sets the user to be used to connect to the slaves

--slave-password=s
Sets the password to be used to connect to the slaves

如果未指明从库账号信息，则默认使用主库相同的端口/用户/密码等信息。

批量复制数据参数

--chunk-index=s
Prefer this index for chunking tables

--chunk-index-columns=i
Use only this many left-most columns of a --chunk-index

--chunk-size=z
Number of rows to select for each chunk copied (default 1000)

--chunk-size-limit=f
Do not copy chunks this much larger than the desired chunk size (default 4.0)

--chunk-time=f
Adjust the chunk size dynamically so each data-copy query takes this long to execute (default 0.5)

当chunk-size和chunk-time两者都未指定时，chunk-size默认值为1000，chunk-time默认值为0.5S，第一次按照chunk-size来进行数据复制，然后根据第一次复制的时间动态调整chumk-size的大小，以适应服务器的性能变化，如上一次复制1000行消耗0.1S，则下次动态调整chumk-size为5000。

如果明确指定chumk-size的值或将chunk-time指定为0，则每次都按照chunk-size复制数据。

帮助文档（3.0.3）

pt-online-schema-change alters a table's structure without blocking reads or
writes.  Specify the database and table in the DSN.  Do not use this tool before
reading its documentation and checking your backups carefully.  For more
details, please use the --help option, or try 'perldoc
/usr/bin/pt-online-schema-change' for complete documentation.

Usage: pt-online-schema-change [OPTIONS] DSN

Options:

  --alter=s                        The schema modification, without the ALTER
                                   TABLE keywords
  --alter-foreign-keys-method=s    How to modify foreign keys so they reference
                                   the new table
  --[no]analyze-before-swap        Execute ANALYZE TABLE on the new table
                                   before swapping with the old one (default
                                   yes)
  --ask-pass                       Prompt for a password when connecting to
                                   MySQL
  --charset=s                  -A  Default character set
  --[no]check-alter                Parses the --alter specified and tries to
                                   warn of possible unintended behavior (
                                   default yes)
  --check-interval=m               Sleep time between checks for --max-lag (
                                   default 1).  Optional suffix s=seconds, m=
                                   minutes, h=hours, d=days; if no suffix, s is
                                   used.
  --[no]check-plan                 Check query execution plans for safety (
                                   default yes)
  --[no]check-replication-filters  Abort if any replication filter is set on
                                   any server (default yes)
  --check-slave-lag=s              Pause the data copy until this replica's lag
                                   is less than --max-lag
  --chunk-index=s                  Prefer this index for chunking tables
  --chunk-index-columns=i          Use only this many left-most columns of a --
                                   chunk-index
  --chunk-size=z                   Number of rows to select for each chunk
                                   copied (default 1000)
  --chunk-size-limit=f             Do not copy chunks this much larger than the
                                   desired chunk size (default 4.0)
  --chunk-time=f                   Adjust the chunk size dynamically so each
                                   data-copy query takes this long to execute (
                                   default 0.5)
  --config=A                       Read this comma-separated list of config
                                   files; if specified, this must be the first
                                   option on the command line
  --critical-load=A                Examine SHOW GLOBAL STATUS after every
                                   chunk, and abort if the load is too high (
                                   default Threads_running=50)
  --data-dir=s                     Create the new table on a different
                                   partition using the DATA DIRECTORY feature
  --database=s                 -D  Connect to this database
  --default-engine                 Remove ENGINE from the new table
  --defaults-file=s            -F  Only read mysql options from the given file
  --[no]drop-new-table             Drop the new table if copying the original
                                   table fails (default yes)
  --[no]drop-old-table             Drop the original table after renaming it (
                                   default yes)
  --[no]drop-triggers              Drop triggers on the old table. --no-drop-
                                   triggers forces --no-drop-old-table (default
                                   yes)
  --dry-run                        Create and alter the new table, but do not
                                   create triggers, copy data, or replace the
                                   original table
  --execute                        Indicate that you have read the
                                   documentation and want to alter the table
  --force                          This options bypasses confirmation in case
                                   of using alter-foreign-keys-method = none ,
                                   which might break foreign key constraints
  --help                           Show help and exit
  --host=s                     -h  Connect to host
  --max-flow-ctl=f                 Somewhat similar to --max-lag but for PXC
                                   clusters
  --max-lag=m                      Pause the data copy until all replicas' lag
                                   is less than this value (default 1s).
                                   Optional suffix s=seconds, m=minutes, h=
                                   hours, d=days; if no suffix, s is used.
  --max-load=A                     Examine SHOW GLOBAL STATUS after every
                                   chunk, and pause if any status variables are
                                   higher than their thresholds (default
                                   Threads_running=25)
  --new-table-name=s               New table name before it is swapped. %T is
                                   replaced with the original table name (
                                   default %T_new)
  --null-to-not-null               Allows MODIFYing a column that allows NULL
                                   values to one that doesn't allow them
  --password=s                 -p  Password to use when connecting
  --pause-file=s                   Execution will be paused while the file
                                   specified by this param exists
  --pid=s                          Create the given PID file
  --plugin=s                       Perl module file that defines a
                                   pt_online_schema_change_plugin class
  --port=i                     -P  Port number to use for connection
  --print                          Print SQL statements to STDOUT
  --progress=a                     Print progress reports to STDERR while
                                   copying rows (default time,30)
  --quiet                      -q  Do not print messages to STDOUT (disables --
                                   progress)
  --recurse=i                      Number of levels to recurse in the hierarchy
                                   when discovering replicas
  --recursion-method=a             Preferred recursion method for discovering
                                   replicas (default processlist,hosts)
  --remove-data-dir                If the original table was created using the
                                   DATA DIRECTORY feature, remove it and create
                                   the new table in MySQL default directory
                                   without creating a new isl file (default no)
  --set-vars=A                     Set the MySQL variables in this comma-
                                   separated list of variable=value pairs
  --skip-check-slave-lag=d         DSN to skip when checking slave lag
  --slave-password=s               Sets the password to be used to connect to
                                   the slaves
  --slave-user=s                   Sets the user to be used to connect to the
                                   slaves
  --sleep=f                        How long to sleep (in seconds) after copying
                                   each chunk (default 0)
  --socket=s                   -S  Socket file to use for connection
  --statistics                     Print statistics about internal counters
  --[no]swap-tables                Swap the original table and the new, altered
                                   table (default yes)
  --tries=a                        How many times to try critical operations
  --[no]use-insert-ignore          pt-online-schema-change by default use
                                   INSERT LOW_PRIORITY IGNORE statements to
                                   copy rows from the old table to the new one (
                                   default yes)
  --user=s                     -u  User for login if not current user
  --version                        Show version and exit
  --[no]version-check              Check for the latest version of Percona
                                   Toolkit, MySQL, and other programs (default
                                   yes)

Option types: s=string, i=integer, f=float, h/H/a/A=comma-separated list, d=DSN, z=size, m=time

Rules:

  --dry-run and --execute are mutually exclusive.
  This tool accepts additional command-line arguments. Refer to the SYNOPSIS and usage information for details.

DSN syntax is key=value[,key=value...]  Allowable DSN keys:

  KEY  COPY  MEANING
  ===  ====  =============================================
  A    yes   Default character set
  D    yes   Database for the old and new table
  F    yes   Only read default options from the given file
  P    yes   Port number to use for connection
  S    yes   Socket file to use for connection
  h    yes   Connect to host
  p    yes   Password to use when connecting
  t    no    Table to alter
  u    yes   User for login if not current user

  If the DSN is a bareword, the word is treated as the 'h' key.

Options and values after processing arguments:

  --alter                          (No value)
  --alter-foreign-keys-method      (No value)
  --analyze-before-swap            TRUE
  --ask-pass                       FALSE
  --charset                        (No value)
  --check-alter                    TRUE
  --check-interval                 1
  --check-plan                     TRUE
  --check-replication-filters      TRUE
  --check-slave-lag                (No value)
  --chunk-index                    (No value)
  --chunk-index-columns            (No value)
  --chunk-size                     1000
  --chunk-size-limit               4.0
  --chunk-time                     0.5
  --config                         /etc/percona-toolkit/percona-toolkit.conf,/etc/percona-toolkit/pt-online-schema-change.conf,/root/.percona-toolkit.conf,/root/.pt-online-schema-change.conf
  --critical-load                  Threads_running=50
  --data-dir                       (No value)
  --database                       (No value)
  --default-engine                 FALSE
  --defaults-file                  (No value)
  --drop-new-table                 TRUE
  --drop-old-table                 TRUE
  --drop-triggers                  TRUE
  --dry-run                        FALSE
  --execute                        FALSE
  --force                          FALSE
  --help                           TRUE
  --host                           (No value)
  --max-flow-ctl                   (No value)
  --max-lag                        1
  --max-load                       Threads_running=25
  --new-table-name                 %T_new
  --null-to-not-null               FALSE
  --password                       (No value)
  --pause-file                     (No value)
  --pid                            (No value)
  --plugin                         (No value)
  --port                           (No value)
  --print                          FALSE
  --progress                       time,30
  --quiet                          FALSE
  --recurse                        (No value)
  --recursion-method               processlist,hosts
  --remove-data-dir                TRUE
  --set-vars
  --skip-check-slave-lag           (No value)
  --slave-password                 (No value)
  --slave-user                     (No value)
  --sleep                          0
  --socket                         (No value)
  --statistics                     FALSE
  --swap-tables                    TRUE
  --tries                          (No value)
  --use-insert-ignore              TRUE
  --user                           (No value)
  --version                        FALSE
  --version-check                  TRUE

参考资料

https://www.cnblogs.com/xiaoyanger/p/6043986.html