MySQL Online DDL 工具之pt-online-schema-change
MySQL DDL:
DDL是一个令所有MySQL dDBA 诟病的一个功能,因为在MySQL中在对表进行dDDL时,会锁表,当表比较小比如小于1W行时,对前端影响较小,当时遇到千万级别的表,就会影响前端应用对表的写操作!
InnoDB引擎是通过以下步骤来进行DDL的:
1、按照原始表(original_table)的表结构和DDL语句,新建一个不可见的临时表(tmp_table)
2、在原表上加write lock,阻塞所有更新操作(insert、delete、update等)
3、执行insert into tmp_table select * from original_table
4、rename original_table和tmp_table,最后drop original_table
5、释放 write lock。
可以看见在InnoDB执行DDL的时候,原表是只能读不能写的。为此 perconal 推出一个工具 pt-online-schema-change ,其特点是修改过程中不会造成读写阻塞。
工作原理:
如果表有外键,除非使用 --alter-foreign-keys-method 指定特定的值,否则工具不予执行。
1 创建一个和你要执行 alter 操作的表一样的空表结构。
2 执行表结构修改,然后从原表中的数据到copy到 表结构修改后的表,
3 在原表上创建触发器将 copy 数据的过程中,在原表的更新操作 更新到新表.
注意:如果表中已经定义了触发器这个工具就不能工作了。
4 copy 完成以后,用rename table 新表代替原表,默认删除原表。
用法介绍:
pt-online-schema-change [OPTIONS] DSN
options 可以自行查看 help,DNS 为你要操作的数据库和表。这里有两个参数需要介绍一下:
--dry-run
这个参数不建立触发器,不拷贝数据,也不会替换原表。只是创建和更改新表。
--execute
这个参数的作用和前面工作原理的介绍的一样,会建立触发器,来保证最新变更的数据会影响至新表。注意:如果不加这个参数,这个工具会在执行一些检查后退出。
依赖条件:
操作的表必须有主键否则 报如下错误:
[root@hank-yoon ~]# pt-online-schema-change -u root -pyoon -h127.0.0.1 --alter='add column vname varchar(20)' --execute D=yoon,t=yoon
No slaves found. See --recursion-method if host hank-yoon.com has slaves.
Not checking slave lag because no slaves were found and --check-slave-lag was not specified.
Operation, tries, wait:
copy_rows, 10, 0.25
create_triggers, 10, 1
drop_triggers, 10, 1
swap_tables, 10, 1
update_foreign_keys, 10, 1
Altering `yoon`.`yoon`...
Creating new table...
Created new table yoon._yoon_new OK.
Altering new table...
Altered `yoon`.`_yoon_new` OK.
2016-01-08T17:51:43 Dropping new table...
2016-01-08T17:51:43 Dropped new table OK.
`yoon`.`yoon` was not altered.
The new table `yoon`.`_yoon_new` does not have a PRIMARY KEY or a unique index which is required for the DELETE trigger.
字段vname没有添加成功!
mysql> show create table yoon\G
*************************** 1. row ***************************
Table: yoon
Create Table: CREATE TABLE `yoon` (
`actor_id` smallint(8) unsigned NOT NULL DEFAULT '0',
`first_name` varchar(45) NOT NULL,
`last_name` varchar(45) NOT NULL,
`last_update` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
) ENGINE=InnoDB DEFAULT CHARSET=utf8
添加主键:
mysql> alter table yoon modify actor_id smallint(8) unsigned primary key;
Query OK, 0 rows affected (0.06 sec)
Records: 0 Duplicates: 0 Warnings: 0
[root@hank-yoon ~]# pt-online-schema-change -u root -pyoon -h127.0.0.1 --alter='add column vname varchar(20)' --execute D=yoon,t=yoon
No slaves found. See --recursion-method if host hank-yoon.com has slaves.
Not checking slave lag because no slaves were found and --check-slave-lag was not specified.
Operation, tries, wait:
copy_rows, 10, 0.25
create_triggers, 10, 1
drop_triggers, 10, 1
swap_tables, 10, 1
update_foreign_keys, 10, 1
Altering `yoon`.`yoon`...
Creating new table...
Created new table yoon._yoon_new OK.
Altering new table...
Altered `yoon`.`_yoon_new` OK.
2016-01-08T17:57:09 Creating triggers...
2016-01-08T17:57:09 Created triggers OK.
2016-01-08T17:57:09 Copying approximately 200 rows...
2016-01-08T17:57:09 Copied rows OK.
2016-01-08T17:57:09 Swapping tables...
2016-01-08T17:57:09 Swapped original and new tables OK.
2016-01-08T17:57:09 Dropping old table...
2016-01-08T17:57:09 Dropped old table `yoon`.`_yoon_old` OK.
2016-01-08T17:57:09 Dropping triggers...
2016-01-08T17:57:09 Dropped triggers OK.
Successfully altered `yoon`.`yoon`.
mysql> show create table yoon\G
*************************** 1. row ***************************
Table: yoon
Create Table: CREATE TABLE `yoon` (
`actor_id` smallint(8) unsigned NOT NULL,
`first_name` varchar(45) NOT NULL,
`last_name` varchar(45) NOT NULL,
`last_update` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`vname` varchar(20) DEFAULT NULL,
PRIMARY KEY (`actor_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
添加多个字段:
[root@hank-yoon ~]# pt-online-schema-change -u root -pyoon -h127.0.0.1 --alter='add column aname varchar(20),add column bname varchar(30)' --execute D=yoon,t=yoon
No slaves found. See --recursion-method if host hank-yoon.com has slaves.
Not checking slave lag because no slaves were found and --check-slave-lag was not specified.
Operation, tries, wait:
copy_rows, 10, 0.25
create_triggers, 10, 1
drop_triggers, 10, 1
swap_tables, 10, 1
update_foreign_keys, 10, 1
Altering `yoon`.`yoon`...
Creating new table...
Created new table yoon._yoon_new OK.
Altering new table...
Altered `yoon`.`_yoon_new` OK.
2016-01-08T18:04:25 Creating triggers...
2016-01-08T18:04:25 Created triggers OK.
2016-01-08T18:04:25 Copying approximately 200 rows...
2016-01-08T18:04:25 Copied rows OK.
2016-01-08T18:04:25 Swapping tables...
2016-01-08T18:04:26 Swapped original and new tables OK.
2016-01-08T18:04:26 Dropping old table...
2016-01-08T18:04:26 Dropped old table `yoon`.`_yoon_old` OK.
2016-01-08T18:04:26 Dropping triggers...
2016-01-08T18:04:26 Dropped triggers OK.
Successfully altered `yoon`.`yoon`.
mysql> show create table yoon\G
*************************** 1. row ***************************
Table: yoon
Create Table: CREATE TABLE `yoon` (
`actor_id` smallint(8) unsigned NOT NULL,
`first_name` varchar(45) NOT NULL,
`last_name` varchar(45) NOT NULL,
`last_update` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`vname` varchar(20) DEFAULT NULL,
`aname` varchar(20) DEFAULT NULL,
`bname` varchar(30) DEFAULT NULL,
PRIMARY KEY (`actor_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
删除字段:
[root@hank-yoon ~]# pt-online-schema-change -u root -pyoon -h127.0.0.1 --alter='drop column aname,drop column bname' --execute D=yoon,t=yoon
No slaves found. See --recursion-method if host hank-yoon.com has slaves.
Not checking slave lag because no slaves were found and --check-slave-lag was not specified.
Operation, tries, wait:
copy_rows, 10, 0.25
create_triggers, 10, 1
drop_triggers, 10, 1
swap_tables, 10, 1
update_foreign_keys, 10, 1
Altering `yoon`.`yoon`...
Creating new table...
Created new table yoon._yoon_new OK.
Altering new table...
Altered `yoon`.`_yoon_new` OK.
2016-01-08T18:05:45 Creating triggers...
2016-01-08T18:05:45 Created triggers OK.
2016-01-08T18:05:45 Copying approximately 200 rows...
2016-01-08T18:05:45 Copied rows OK.
2016-01-08T18:05:45 Swapping tables...
2016-01-08T18:05:45 Swapped original and new tables OK.
2016-01-08T18:05:45 Dropping old table...
2016-01-08T18:05:45 Dropped old table `yoon`.`_yoon_old` OK.
2016-01-08T18:05:45 Dropping triggers...
2016-01-08T18:05:46 Dropped triggers OK.
Successfully altered `yoon`.`yoon`.
添加索引:
[root@hank-yoon ~]# pt-online-schema-change -u root -pyoon -h127.0.0.1 --alter='add key index_first(first_name)' --execute D=yoon,t=yoon No slaves found. See --recursion-method if host hank-yoon.com has slaves.
Not checking slave lag because no slaves were found and --check-slave-lag was not specified.
Operation, tries, wait:
copy_rows, 10, 0.25
create_triggers, 10, 1
drop_triggers, 10, 1
swap_tables, 10, 1
update_foreign_keys, 10, 1
Altering `yoon`.`yoon`...
Creating new table...
Created new table yoon._yoon_new OK.
Altering new table...
Altered `yoon`.`_yoon_new` OK.
2016-01-08T18:06:38 Creating triggers...
2016-01-08T18:06:38 Created triggers OK.
2016-01-08T18:06:38 Copying approximately 200 rows...
2016-01-08T18:06:38 Copied rows OK.
2016-01-08T18:06:38 Swapping tables...
2016-01-08T18:06:38 Swapped original and new tables OK.
2016-01-08T18:06:38 Dropping old table...
2016-01-08T18:06:38 Dropped old table `yoon`.`_yoon_old` OK.
2016-01-08T18:06:38 Dropping triggers...
2016-01-08T18:06:38 Dropped triggers OK.
Successfully altered `yoon`.`yoon`.
MySQL Online DDL 工具之pt-online-schema-change的更多相关文章
- [资料收集]MySQL在线DDL工具pt-online-schema-change
MySQL在线DDL工具pt-online-schema-change pt-online-schema-change使用说明(未完待续) 官网
- MySQL在线DDL工具 gh-ost
一.简介 gh-ost基于 golang 语言,是 github 开源的一个 DDL 工具,是 GitHub's Online Schema Transmogrifier/Transfigurator ...
- MySQL Online DDL工具
MySQL在线表结构变更工具 MySQL的大表表结构变更常用的解决方案无外乎三种: 一是利用Percona的pt-online-schema-change,Facebook的OSC等三方工具, 二是在 ...
- MySQL在线DDL gh-ost 使用说明
背景: 作为一个DBA,大表的DDL的变更大部分都是使用Percona的pt-online-schema-change,本文说明下另一种工具gh-ost的使用:不依赖于触发器,是因为他是通过模拟从库, ...
- MySQL Binlog 解析工具 Maxwell 详解
maxwell 简介 Maxwell是一个能实时读取MySQL二进制日志binlog,并生成 JSON 格式的消息,作为生产者发送给 Kafka,Kinesis.RabbitMQ.Redis.Goog ...
- mysql online ddl
大家知道,互联网业务是典型的OLTP(online transaction process)应用,这种应用访问数据库的特点是大量的短事务高并发运行.因此任何限制高并发的动作都是不可接受的,甚至 ...
- Online Schema Change for MySQL
It is great to be able to build small utilities on top of an excellent RDBMS. Thank you MySQL. This ...
- MySQL 闪回工具之 binlog2sql
生产上误删数据.误改数据的现象也是时常发生的现象,作为 DBA 这时候就需要出来补锅了,最开始的做法是恢复备份,然后从中找到需要的数据再进行修复,但是这个时间太长了,对于大表少数数据的修复来讲,动作太 ...
- Online DDL工具的安装与使用
最近经常在线上经常遇到有性能问题的SQL,有些表没有合理添加索引,有些表添加的索引不合理,各种各样的问题,导致SQL的执行效率不高.这时DBA们不得不重构SQL,使其达到最好的性能,这时我们往往要在线 ...
随机推荐
- linux ----虚拟机无法与本地机通信
1.以前能正常通信,电脑重启或休眠唤醒后, 虚拟机centos无法与本地主机通信,但能ping通同一局域网的其他主机,也能与外网通信 故障原因: 未找到 解决办法: 1.重启电脑 2.重启xshell ...
- Laravel 清空配置缓存
清空配置缓存 php artisan cache:clear php artisan config:clear
- 统计机器学习(statistical machine learning)
组要组成部分:监督学习(supervised learning),非监督学习(unsupervised learning),半监督学习(semi-supervised learning),强化学习(r ...
- 【CSS3】---练习制作导航菜单
练习题 根据所学知识,使用CSS3实现下图的导航菜单效果 任务 1.制作导航圆角 提示:使用border-radius实现圆角 2.制作导航立体风格 提示:使用box-shadow实现立体风格 3.制 ...
- PHP与MySQL中编码的设置
php代码 header("Content-type:text/html;Charset=utf8"); myql_query("set names utf8" ...
- Part 14 ng hide and ng show in AngularJS
ng-hide and ng-show directives are used to control the visibility of the HTML elements. Let us under ...
- 清除windows的EFS加密
所使用软件为aefsdr_setup_en,搜索名为advanced.efs.data.recovery 1. 创建需要加密的文件 2. 进行加密 ...
- 使用runtime给类动态添加方法并调用 - class_addMethod
上手开发 iOS 一段时间后,我发现并不能只着眼于完成需求,利用闲暇之余多研究其他的开发技巧,才能在有限时间内提升自己水平.当然,“其他开发技巧”这个命题对于任何一个开发领域都感觉不找边际,而对于我来 ...
- 关于IIS中WEB网站访问弹“验证输入框”及“401限制访问”的解决办法
最近在配置IIS网站的过程中,不管是本地还是外部访问配置的网站,出现了需要输入用户名.密码.及域的对话框,解决之后又出现页面401限制访问的错误页面. 就这两项错误,依次做一下解决办法的整理. 解决流 ...
- 对索引像素格式的图片进行Setpixel(具有索引像素格式的图像不支持SetPixel)解决方案
最近编写了一个验证码识别软件.其中对png.jpg图片进行二值化处理时,出现了错误:具有索引像素格式的图像不支持SetPixel解决方案.从字面上来看,这说明我对一个具有索引色的图片进行了直接RGB颜 ...