大家都知道MySQL Binlog 有三种格式，分别是Statement、Row、Mixd。Statement记录了用户执行的原始SQL，而Row则是记录了行的修改情况，在MySQL 5.6以上的版本默认是Mixd格式，但为了保证复制数据的完整性，建议生产环境都使用Row格式，就前面所说的Row记录的是行数据的修改情况，而不是原始SQL。那么线上或者测试环境误操删除或者更新几条数据后，又想恢复，那怎么办呢？下面演示基于Binlog格式为Row的误操后数据恢复，那么怎么把Binlog解析出来生成反向的原始SQL呢？下面我们一起来学习。

下面我们使用 binlog-rollback.pl 对数据进行恢复演示。（这脚本的作者不知道是谁，Github上也没找到这个脚本，所以无法标明出处），脚本是用Perl语言写的，非常好用的一个脚本，当然你也可以用Shell或者Python脚本来实现，下面是脚本的代码：

  1. #!/usr/lib/perl -w
  2.  
  3. use strict;
  4. use warnings;
  5.  
  6. use Class::Struct;
  7. use Getopt::Long qw(:config no_ignore_case);                    # GetOption
  8. # register handler system signals
  9. use sigtrap 'handler', \&sig_int, 'normal-signals';
  10.  
  11. # catch signal
  12. sub sig_int(){
  13.     my ($signals) = @_;
  14.     print STDERR "# Caught SIG$signals.\n";
  15.     exit ;
  16. }
  17.  
  18. my %opt;
  19. my $srcfile;
  20. my $host = '127.0.0.1';
  21. my $port = ;
  22. my ($user,$pwd);
  23. my ($MYSQL, $MYSQLBINLOG, $ROLLBACK_DML);
  24. my $outfile = '/dev/null';
  25. my (%do_dbs,%do_tbs);
  26.  
  27. # tbname=>tbcol, tbcol: @n=>colname,type
  28. my %tbcol_pos;
  29.  
  30. my $SPLITER_COL = ',';
  31. my $SQLTYPE_IST = 'INSERT';
  32. my $SQLTYPE_UPD = 'UPDATE';
  33. my $SQLTYPE_DEL = 'DELETE';
  34. my $SQLAREA_WHERE = 'WHERE';
  35. my $SQLAREA_SET = 'SET';
  36.  
  37. my $PRE_FUNCT = '========================== ';
  38.  
  39. # =========================================================
  40. # 基于row模式的binlog，生成DML(insert/update/delete)的rollback语句
  41. # 通过mysqlbinlog -v 解析binlog生成可读的sql文件
  42. # 提取需要处理的有效sql
  43. #     "### "开头的行.如果输入的start-position位于某个event group中间，则会导致"无法识别event"错误
  44. #
  45. # 将INSERT/UPDATE/DELETE 的sql反转,并且1个完整sql只能占1行
  46. #     INSERT: INSERT INTO => DELETE FROM, SET => WHERE
  47. #     UPDATE: WHERE => SET, SET => WHERE
  48. #     DELETE: DELETE FROM => INSERT INTO, WHERE => SET
  49. # 用列名替换位置@{1,2,3}
  50. #     通过desc table获得列顺序及对应的列名
  51. #     特殊列类型value做特别处理
  52. # 逆序
  53. #
  54. # 注意:
  55. #     表结构与现在的表结构必须相同[谨记]
  56. #     由于row模式是幂等的，并且恢复是一次性，所以只提取sql，不提取BEGIN/COMMIT
  57. #     只能对INSERT/UPDATE/DELETE进行处理
  58. # ========================================================
  59. sub main{
  60.  
  61.     # get input option
  62.     &get_options();
  63.  
  64.     #
  65.     &init_tbcol();
  66.  
  67.     #
  68.     &do_binlog_rollback();
  69. }
  70.  
  71. &main();
  72.  
  73. # ----------------------------------------------------------------------------------------
  74. # Func : get options and set option flag
  75. # ----------------------------------------------------------------------------------------
  76. sub get_options{
  77.     #Get options info
  78.     GetOptions(\%opt,
  79.         'help',                    # OUT : print help info
  80.         'f|srcfile=s',            # IN  : binlog file
  81.         'o|outfile=s',            # out : output sql file
  82.         'h|host=s',                # IN  :  host
  83.         'u|user=s',             # IN  :  user
  84.         'p|password=s',         # IN  :  password
  85.         'P|port=i',                # IN  :  port
  86.         'start-datetime=s',        # IN  :  start datetime
  87.         'stop-datetime=s',        # IN  :  stop datetime
  88.         'start-position=i',        # IN  :  start position
  89.         'stop-position=i',        # IN  :  stop position
  90.         'd|database=s',            # IN  :  database, split comma
  91.         'T|table=s',            # IN  :  table, split comma
  92.         'i|ignore',                # IN  :  ignore binlog check ddl and so on
  93.         'debug',                # IN  :  print debug information
  94.       ) or print_usage();
  95.  
  96.     if (!scalar(%opt)) {
  97.         &print_usage();
  98.     }
  99.  
  100.     # Handle for options
  101.     if ($opt{'f'}){
  102.         $srcfile = $opt{'f'};
  103.     }else{
  104.         &merror("please input binlog file");
  105.     }
  106.  
  107.     $opt{'h'} and $host = $opt{'h'};
  108.     $opt{'u'} and $user = $opt{'u'};
  109.     $opt{'p'} and $pwd = $opt{'p'};
  110.     $opt{'P'} and $port = $opt{'P'};
  111.     if ($opt{'o'}) {
  112.         $outfile = $opt{'o'};
  113.         # 清空 outfile
  114.         `echo '' > $outfile`;
  115.     }
  116.  
  117.     #
  118.     $MYSQL = qq{mysql -h$host -u$user -p'$pwd' -P$port};
  119.     &mdebug("get_options::MYSQL\n\t$MYSQL");
  120.  
  121.     # 提取binlog,不需要显示列定义信息，用-v，而不用-vv
  122.     $MYSQLBINLOG = qq{mysqlbinlog -v};
  123.     $MYSQLBINLOG .= " --start-position=".$opt{'start-position'} if $opt{'start-position'};
  124.     $MYSQLBINLOG .= " --stop-position=".$opt{'stop-position'} if $opt{'stop-postion'};
  125.     $MYSQLBINLOG .= " --start-datetime='".$opt{'start-datetime'}."'" if $opt{'start-datetime'};
  126.     $MYSQLBINLOG .= " --stop-datetime='$opt{'stop-datetime'}'" if $opt{'stop-datetime'};
  127.     $MYSQLBINLOG .= " $srcfile";
  128.     &mdebug("get_options::MYSQLBINLOG\n\t$MYSQLBINLOG");
  129.  
  130.     # 检查binlog中是否含有 ddl sql: CREATE|ALTER|DROP|RENAME
  131.     &check_binlog() unless ($opt{'i'});
  132.  
  133.     # 不使用mysqlbinlog过滤，USE dbname;方式可能会漏掉某些sql，所以不在mysqlbinlog过滤
  134.     # 指定数据库
  135.     if ($opt{'d'}){
  136.         my @dbs = split(/,/,$opt{'d'});
  137.         foreach my $db (@dbs){
  138.             $do_dbs{$db}=;
  139.         }
  140.     }
  141.  
  142.     # 指定表
  143.     if ($opt{'T'}){
  144.         my @tbs = split(/,/,$opt{'T'});
  145.         foreach my $tb (@tbs){
  146.             $do_tbs{$tb}=;
  147.         }
  148.     }
  149.  
  150.     # 提取有效DML SQL
  151.     $ROLLBACK_DML = $MYSQLBINLOG." | grep '^### '";
  152.     # 去掉注释: '### ' -> ''
  153.     # 删除首尾空格
  154.     $ROLLBACK_DML .= " | sed 's/###\\s*//g;s/\\s*\$//g'";
  155.     &mdebug("rollback dml\n\t$ROLLBACK_DML");
  156.  
  157.     # 检查内容是否为空
  158.     my $cmd = "$ROLLBACK_DML | wc -l";
  159.     &mdebug("check contain dml sql\n\t$cmd");
  160.     my $size = `$cmd`;
  161.     chomp($size);
  162.     unless ($size >){
  163.         &merror("binlog DML is empty:$ROLLBACK_DML");
  164.     };
  165.  
  166. }    
  167.  
  168. # ----------------------------------------------------------------------------------------
  169. # Func :  check binlog contain DDL
  170. # ----------------------------------------------------------------------------------------
  171. sub check_binlog{
  172.     &mdebug("$PRE_FUNCT check_binlog");
  173.     my $cmd = "$MYSQLBINLOG ";
  174.     $cmd .= " | grep -E -i '^(CREATE|ALTER|DROP|RENAME)' ";
  175.     &mdebug("check binlog has DDL cmd\n\t$cmd");
  176.     my $ddlcnt = `$cmd`;
  177.     chomp($ddlcnt);
  178.  
  179.     my $ddlnum = `$cmd | wc -l`;
  180.     chomp($ddlnum);
  181.     my $res = ;
  182.     if ($ddlnum>){
  183.         # 在ddl sql前面加上前缀<DDL>
  184.         $ddlcnt = `echo '$ddlcnt' | sed 's/^/<DDL>/g'`;
  185.         &merror("binlog contain $ddlnum DDL:$MYSQLBINLOG. ddl sql:\n$ddlcnt");
  186.     }
  187.  
  188.     return $res;
  189. }
  190.  
  191. # ----------------------------------------------------------------------------------------
  192. # Func : init all table column order
  193. #        if input --database --table params, only get set table column order
  194. # ----------------------------------------------------------------------------------------
  195. sub init_tbcol{
  196.     &mdebug("$PRE_FUNCT init_tbcol");
  197.     # 提取DML语句
  198.     my $cmd .= "$ROLLBACK_DML | grep -E '^(INSERT|UPDATE|DELETE)'";
  199.     # 提取表名，并去重
  200.     #$cmd .= " | awk '{if (\$1 ~ \"^UPDATE\") {print \$2}else {print \$3}}' | uniq ";
  201.     $cmd .= " | awk '{if (\$1 ~ \"^UPDATE\") {print \$2}else {print \$3}}' | sort | uniq ";
  202.     &mdebug("get table name cmd\n\t$cmd");
  203.     open ALLTABLE, "$cmd | " or die "can't open file:$cmd\n";
  204.  
  205.     while (my $tbname = <ALLTABLE>){
  206.         chomp($tbname);
  207.         #if (exists $tbcol_pos{$tbname}){
  208.         #    next;
  209.         #}
  210.         &init_one_tbcol($tbname) unless (&ignore_tb($tbname));
  211.  
  212.     }
  213.     close ALLTABLE or die "can't close file:$cmd\n";
  214.  
  215.     # init tb col
  216.     foreach my $tb (keys %tbcol_pos){
  217.         &mdebug("tbname->$tb");
  218.         my %colpos = %{$tbcol_pos{$tb}};
  219.         foreach my $pos (keys %colpos){
  220.             my $col = $colpos{$pos};
  221.             my ($cname,$ctype) = split(/$SPLITER_COL/, $col);
  222.             &mdebug("\tpos->$pos,cname->$cname,ctype->$ctype");
  223.         }
  224.     }
  225. };
  226.  
  227. # ----------------------------------------------------------------------------------------
  228. # Func : init one table column order
  229. # ----------------------------------------------------------------------------------------
  230. sub init_one_tbcol{
  231.     my $tbname = shift;
  232.     &mdebug("$PRE_FUNCT init_one_tbcol");
  233.     # 获取表结构及列顺序
  234.     my $cmd = $MYSQL." --skip-column-names --silent -e 'desc $tbname'";
  235.     # 提取列名，并拼接
  236.     $cmd .= " | awk -F\'\\t\' \'{print NR\"$SPLITER_COL`\"\$1\"`$SPLITER_COL\"\$2}'";
  237.     &mdebug("get table column infor cmd\n\t$cmd");
  238.     open TBCOL,"$cmd | " or die "can't open desc $tbname;";
  239.  
  240.     my %colpos;
  241.     while (my $line = <TBCOL>){
  242.         chomp($line);
  243.         my ($pos,$col,$coltype) = split(/$SPLITER_COL/,$line);
  244.         &mdebug("linesss=$line\n\t\tpos=$pos\n\t\tcol=$col\n\t\ttype=$coltype");
  245.         $colpos{$pos} = $col.$SPLITER_COL.$coltype;
  246.     }
  247.     close TBCOL or die "can't colse desc $tbname";
  248.  
  249.     $tbcol_pos{$tbname} = \%colpos;
  250. }
  251.  
  252. # ----------------------------------------------------------------------------------------
  253. # Func :  rollback sql:    INSERT/UPDATE/DELETE
  254. # ----------------------------------------------------------------------------------------
  255. sub do_binlog_rollback{
  256.     my $binlogfile = "$ROLLBACK_DML ";
  257.     &mdebug("$PRE_FUNCT do_binlog_rollback");
  258.  
  259.     # INSERT|UPDATE|DELETE
  260.     my $sqltype;
  261.     # WHERE|SET
  262.     my $sqlarea;
  263.  
  264.     my ($tbname, $sqlstr) = ('', '');
  265.     my ($notignore, $isareabegin) = (,);
  266.  
  267.     # output sql file
  268.     open SQLFILE, ">> $outfile" or die "Can't open sql file:$outfile";
  269.  
  270.     # binlog file
  271.     open BINLOG, "$binlogfile |" or die "Can't open file: $binlogfile";
  272.     while (my $line = <BINLOG>){
  273.         chomp($line);
  274.         if ($line =~ /^(INSERT|UPDATE|DELETE)/){
  275.             # export sql
  276.             if ($sqlstr ne ''){
  277.                 $sqlstr .= ";\n";
  278.                 print SQLFILE $sqlstr;
  279.                 &mdebug("export sql\n\t".$sqlstr);
  280.                 $sqlstr = '';
  281.             }
  282.  
  283.             if ($line =~ /^INSERT/){
  284.                 $sqltype = $SQLTYPE_IST;
  285.                 $tbname = `echo '$line' | awk '{print \$3}'`;
  286.                 chomp($tbname);
  287.                 $sqlstr = qq{DELETE FROM $tbname};
  288.             }elsif ($line =~ /^UPDATE/){
  289.                 $sqltype = $SQLTYPE_UPD;
  290.                 $tbname = `echo '$line' | awk '{print \$2}'`;
  291.                 chomp($tbname);
  292.                 $sqlstr = qq{UPDATE $tbname};
  293.             }elsif ($line =~ /^DELETE/){
  294.                 $sqltype = $SQLTYPE_DEL;
  295.                 $tbname = `echo '$line' | awk '{print \$3}'`;
  296.                 chomp($tbname);
  297.                 $sqlstr = qq{INSERT INTO $tbname};
  298.             }
  299.  
  300.             # check ignore table
  301.             if(&ignore_tb($tbname)){
  302.                 $notignore = ;
  303.                 &mdebug("<BINLOG>#IGNORE#:line:".$line);
  304.                 $sqlstr = '';
  305.             }else{
  306.                 $notignore = ;
  307.                 &mdebug("<BINLOG>#DO#:line:".$line);
  308.             }
  309.         }else {
  310.             if($notignore){
  311.                 &merror("can't get tbname") unless (defined($tbname));
  312.                 if ($line =~ /^WHERE/){
  313.                     $sqlarea = $SQLAREA_WHERE;
  314.                     $sqlstr .= qq{ SET};
  315.                     $isareabegin = ;
  316.                 }elsif ($line =~ /^SET/){
  317.                     $sqlarea = $SQLAREA_SET;
  318.                     $sqlstr .= qq{ WHERE};
  319.                     $isareabegin = ;
  320.                 }elsif ($line =~ /^\@/){
  321.                     $sqlstr .= &deal_col_value($tbname, $sqltype, $sqlarea, $isareabegin, $line);
  322.                     $isareabegin = ;
  323.                 }else{
  324.                     &mdebug("::unknown sql:".$line);
  325.                 }
  326.             }
  327.         }
  328.     }
  329.     # export last sql
  330.     if ($sqlstr ne ''){
  331.         $sqlstr .= ";\n";
  332.         print SQLFILE $sqlstr;
  333.         &mdebug("export sql\n\t".$sqlstr);
  334.     }
  335.  
  336.     close BINLOG or die "Can't close binlog file: $binlogfile";
  337.  
  338.     close SQLFILE or die "Can't close out sql file: $outfile";
  339.  
  340.     # 逆序
  341.     # 1!G: 只有第一行不执行G, 将hold space中的内容append回到pattern space
  342.     # h: 将pattern space 拷贝到hold space
  343.     # $!d: 除最后一行都删除
  344.     my $invert = "sed -i '1!G;h;\$!d' $outfile";
  345.     my $res = `$invert`;
  346.     &mdebug("inverter order sqlfile :$invert");
  347. }
  348.  
  349. # ----------------------------------------------------------------------------------------
  350. # Func :  transfer column pos to name
  351. #    deal column value
  352. #
  353. #  &deal_col_value($tbname, $sqltype, $sqlarea, $isareabegin, $line);
  354. # ----------------------------------------------------------------------------------------
  355. sub deal_col_value($$$$$){
  356.     my ($tbname, $sqltype, $sqlarea, $isareabegin, $line) = @_;
  357.     &mdebug("$PRE_FUNCT deal_col_value");
  358.     &mdebug("input:tbname->$tbname,type->$sqltype,area->$sqlarea,areabegin->$isareabegin,line->$line");
  359.     my @vals = split(/=/, $line);
  360.     my $pos = substr($vals[],);
  361.     my $valstartpos = length($pos)+;
  362.     my $val = substr($line,$valstartpos);
  363.     my %tbcol = %{$tbcol_pos{$tbname}};
  364.     my ($cname,$ctype) = split(/$SPLITER_COL/,$tbcol{$pos});
  365.     &merror("can't get $tbname column $cname type") unless (defined($cname) || defined($ctype));
  366.     &mdebug("column infor:cname->$cname,type->$ctype");
  367.  
  368.     # join str
  369.     my $joinstr;
  370.     if ($isareabegin){
  371.         $joinstr = ' ';
  372.     }else{
  373.         # WHERE 被替换为 SET, 使用 ,  连接
  374.         if ($sqlarea eq $SQLAREA_WHERE){
  375.             $joinstr = ', ';
  376.         # SET 被替换为 WHERE 使用 AND 连接
  377.         }elsif ($sqlarea eq $SQLAREA_SET){
  378.             $joinstr = ' AND ';
  379.         }else{
  380.             &merror("!!!!!!The scripts error");
  381.         }
  382.     }
  383.  
  384.     #
  385.     my $newline = $joinstr;
  386.  
  387.     # NULL value
  388.     if (($val eq 'NULL') && ($sqlarea eq $SQLAREA_SET)){
  389.         $newline .= qq{ $cname IS NULL};
  390.     }else{
  391.         # timestamp: record seconds
  392.         if ($ctype eq 'timestamp'){
  393.             $newline .= qq{$cname=from_unixtime($val)};
  394.         # datetime: @n=yyyy-mm-dd hh::ii::ss
  395.         }elsif ($ctype eq 'datetime'){
  396.             $newline .= qq{$cname='$val'};
  397.         }else{
  398.             $newline .= qq{$cname=$val};
  399.         }
  400.     }
  401.     &mdebug("\told>$line\n\tnew>$newline");
  402.  
  403.     return $newline;
  404. }
  405.  
  406. # ----------------------------------------------------------------------------------------
  407. # Func :  check is ignore table
  408. # params: IN table full name #  format:`dbname`.`tbname`
  409. # RETURN：
  410. #        0 not ignore
  411. #        1 ignore
  412. # ----------------------------------------------------------------------------------------
  413. sub ignore_tb($){
  414.     my $fullname = shift;
  415.     # 删除`
  416.     $fullname =~ s/`//g;
  417.     my ($dbname,$tbname) = split(/\./,$fullname);
  418.     my $res = ;
  419.  
  420.     # 指定了数据库
  421.     if ($opt{'d'}){
  422.         # 与指定库相同
  423.         if ($do_dbs{$dbname}){
  424.             # 指定表
  425.             if ($opt{'T'}){
  426.                 # 与指定表不同
  427.                 unless ($do_tbs{$tbname}){
  428.                     $res = ;
  429.                 }
  430.             }
  431.         # 与指定库不同
  432.         }else{
  433.             $res = ;
  434.         }
  435.     }
  436.     #&mdebug("Table check ignore:$fullname->$res");
  437.     return $res;
  438. }
  439.  
  440. # ----------------------------------------------------------------------------------------
  441. # Func :  print debug msg
  442. # ----------------------------------------------------------------------------------------
  443. sub mdebug{
  444.     my (@msg) = @_;
  445.     print "<DEBUG>@msg\n" if ($opt{'debug'});
  446. }
  447.  
  448. # ----------------------------------------------------------------------------------------
  449. # Func :  print error msg and exit
  450. # ----------------------------------------------------------------------------------------
  451. sub merror{
  452.     my (@msg) = @_;
  453.     print "<Error>:@msg\n";
  454.     &print_usage();
  455.     exit();
  456. }
  457.  
  458. # ----------------------------------------------------------------------------------------
  459. # Func :  print usage
  460. # ----------------------------------------------------------------------------------------
  461. sub print_usage{
  462.     print <<EOF;
  463. ==========================================================================================
  464. Command line options :
  465.     --help                # OUT : print help info
  466.     -f, --srcfile            # IN  : binlog file. [required]
  467.     -o, --outfile            # OUT : output sql file. [required]
  468.     -h, --host            # IN  : host. default '127.0.0.1'
  469.     -u, --user            # IN  : user. [required]
  470.     -p, --password            # IN  : password. [required]
  471.     -P, --port            # IN  : port. default '3306'
  472.     --start-datetime        # IN  : start datetime
  473.     --stop-datetime            # IN  : stop datetime
  474.     --start-position        # IN  : start position
  475.     --stop-position            # IN  : stop position
  476.     -d, --database            # IN  : database, split comma
  477.     -T, --table            # IN  : table, split comma. [required] set -d
  478.     -i, --ignore            # IN  : ignore binlog check contain DDL(CREATE|ALTER|DROP|RENAME)
  479.     --debug                # IN  :  print debug information
  480.  
  481. Sample :
  482.    shell> perl binlog-rollback.pl -f 'mysql-bin.000001' -o '/tmp/t.sql' -u 'user' -p 'pwd'
  483.    shell> perl binlog-rollback.pl -f 'mysql-bin.000001' -o '/tmp/t.sql' -u 'user' -p 'pwd' -i
  484.    shell> perl binlog-rollback.pl -f 'mysql-bin.000001' -o '/tmp/t.sql' -u 'user' -p 'pwd' --debug
  485.    shell> perl binlog-rollback.pl -f 'mysql-bin.000001' -o '/tmp/t.sql' -h '192.168.1.2' -u 'user' -p 'pwd' -P
  486.    shell> perl binlog-rollback.pl -f 'mysql-bin.000001' -o '/tmp/t.sql' -u 'user' -p 'pwd' --start-position=
  487.    shell> perl binlog-rollback.pl -f 'mysql-bin.000001' -o '/tmp/t.sql' -u 'user' -p 'pwd' --start-position= --stop-position=
  488.    shell> perl binlog-rollback.pl -f 'mysql-bin.000001' -o '/tmp/t.sql' -u 'user' -p 'pwd' -d 'db1,db2'
  489.    shell> perl binlog-rollback.pl -f 'mysql-bin.0000*' -o '/tmp/t.sql' -u 'user' -p 'pwd' -d 'db1,db2' -T 'tb1,tb2'
  490. ==========================================================================================
  491. EOF
  492.     exit;
  493. }
  494.  
  495. ;

这脚本含有注释以及使用说明，所以使用起来还是比较简单的，如果你会Perl语言，相信也很容易看懂代码。binlog-rollback.pl的使用参数如下：

  1. [root@localhost mysql3306]# perl binlog-rollback.pl
  2. ==========================================================================================
  3. Command line options :
  4.         --help                          # OUT : print help info
  5.         -f, --srcfile                   # IN  : binlog file. [required]
  6.         -o, --outfile                   # OUT : output sql file. [required]
  7.         -h, --host                      # IN  : host. default '127.0.0.1'
  8.         -u, --user                      # IN  : user. [required]
  9.         -p, --password                  # IN  : password. [required]
  10.         -P, --port                      # IN  : port. default ''
  11.         --start-datetime                # IN  : start datetime
  12.         --stop-datetime                 # IN  : stop datetime
  13.         --start-position                # IN  : start position
  14.         --stop-position                 # IN  : stop position
  15.         -d, --database                  # IN  : database, split comma
  16.         -T, --table                     # IN  : table, split comma. [required] set -d
  17.         -i, --ignore                    # IN  : ignore binlog check contain DDL(CREATE|ALTER|DROP|RENAME)
  18.         --debug                         # IN  :  print debug information
  19.  
  20. Sample :
  21.    shell> perl binlog-rollback.pl -f 'mysql-bin.000001' -o '/tmp/t.sql' -u 'user' -p 'pwd'
  22.    shell> perl binlog-rollback.pl -f 'mysql-bin.000001' -o '/tmp/t.sql' -u 'user' -p 'pwd' -i
  23.    shell> perl binlog-rollback.pl -f 'mysql-bin.000001' -o '/tmp/t.sql' -u 'user' -p 'pwd' --debug
  24.    shell> perl binlog-rollback.pl -f 'mysql-bin.000001' -o '/tmp/t.sql' -h '192.168.1.2' -u 'user' -p 'pwd' -P
  25.    shell> perl binlog-rollback.pl -f 'mysql-bin.000001' -o '/tmp/t.sql' -u 'user' -p 'pwd' --start-position=
  26.    shell> perl binlog-rollback.pl -f 'mysql-bin.000001' -o '/tmp/t.sql' -u 'user' -p 'pwd' --start-position= --stop-position=
  27.    shell> perl binlog-rollback.pl -f 'mysql-bin.000001' -o '/tmp/t.sql' -u 'user' -p 'pwd' -d 'db1,db2'
  28.    shell> perl binlog-rollback.pl -f 'mysql-bin.0000*' -o '/tmp/t.sql' -u 'user' -p 'pwd' -d 'db1,db2' -T 'tb1,tb2'
  29. ==========================================================================================
  30. [root@localhost mysql3306]#

下面主要演示对一个表的增、删、修（Insert/Delete/Update）操作，基于Binlog为Row格式的反向解析。

细心看脚本的朋友都能看到这个脚本需要提供一个连接MySQL的用户，主要是为了获取表结构。下面我们测试一个普通用户并给予SELECT权限即可，默认是host是127.0.0.1，这个可以修改脚本，我这里按脚本默认的：

  1. <Test>[(none)]> GRANT SELECT ON *.* TO 'recovery'@'127.0.0.1' identified by '';
  2. Query OK, 0 rows affected (0.08 sec)
  3.  
  4. <Test>[(none)]> flush privileges;
  5. Query OK, 0 rows affected (0.04 sec)
  6.  
  7. <Test>[(none)]>

往xuanzhi库的表tb1里插入2行数据，记得binlog格式要为ROW：

  1. <Test>[xuanzhi]> show global variables like 'binlog_format';
  2. +---------------+-------+
  3. | Variable_name | Value |
  4. +---------------+-------+
  5. | binlog_format | ROW   |
  6. +---------------+-------+
  7. 1 row in set (0.00 sec)
  8.  
  9. <Test>[xuanzhi]> insert into xuanzhi.tb1 select 1,'aa';
  10. Query OK, 1 row affected (0.01 sec)
  11. Records: 1  Duplicates: 0  Warnings: 0
  12.  
  13. <Test>[xuanzhi]> insert into xuanzhi.tb1 select 2,'cc';
  14. Query OK, 1 row affected (0.01 sec)
  15. Records: 1  Duplicates: 0  Warnings: 0
  16.  
  17. <Test>[xuanzhi]> select * from xuanzhi.tb1;
  18. +------+------+
  19. | id   | name |
  20. +------+------+
  21. |    1 | aa   |
  22. |    2 | cc   |
  23. +------+------+
  24. 2 rows in set (0.00 sec)
  25.  
  26. <Test>[xuanzhi]>

为了看到运行脚本在不指定库看到的效果，我这里再往test库的user表插入两行数据：

  1. <Test>[xuanzhi]> insert into test.user select 1,'user1',20;
  2. Query OK, 1 row affected (0.03 sec)
  3. Records: 1  Duplicates: 0  Warnings: 0
  4.  
  5. <Test>[xuanzhi]> insert into test.user select 2,'user2',30;
  6. Query OK, 1 row affected (0.01 sec)
  7. Records: 1  Duplicates: 0  Warnings: 0
  8.  
  9. <Test>[xuanzhi]>

查看此时的此时处于那个binlog：

  1. <Test>[xuanzhi]> show master status;
  2. +----------------------+----------+--------------+------------------+-------------------+
  3. | File                 | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
  4. +----------------------+----------+--------------+------------------+-------------------+
  5. | localhost-bin.000023 |      936 |              |                  |                   |
  6. +----------------------+----------+--------------+------------------+-------------------+
  7. 1 row in set (0.00 sec)
  8.  
  9. <Test>[xuanzhi]>

下面运行脚本 binlog-rollback.pl ，不指定任何库和表的情况下，这时表把binlog里所有DML操作都生成反向的SQL（最新的DML会生成在输入文件的最前面）：

  1. [root@localhost mysql3306]# perl binlog-rollback.pl -f 'localhost-bin.000023'  -o '/data/t.sql' -u 'recovery' -p ''
  2. mysql: [Warning] Using a password on the command line interface can be insecure.
  3. mysql: [Warning] Using a password on the command line interface can be insecure.
  4. [root@localhost mysql3306]#

我们查看输出的文件：/data/t.sql

  1. [root@localhost mysql3306]# cat /data/t.sql
  2. DELETE FROM `test`.`user` WHERE `id`= AND `name`='user2' AND `age`=;
  3. DELETE FROM `test`.`user` WHERE `id`= AND `name`='user1' AND `age`=;
  4. DELETE FROM `xuanzhi`.`tb1` WHERE `id`= AND `name`='bb';
  5. DELETE FROM `xuanzhi`.`tb1` WHERE `id`= AND `name`='aa';

可以看到，INSERT操作的反向操作就是DELETE，这里把所有库的DML操作都查出来了，在后面会演示找单个库或者表所产生的反向SQL。

下面模拟运维人员、开发人员或者DBA误操删除数据，分别在不同的库删除一条记录：

  1. <Test>[xuanzhi]> delete from xuanzhi.tb1 where id=2;
  2. Query OK, 1 row affected (0.06 sec)
  3.  
  4. <Test>[xuanzhi]> delete from test.user where id=1;
  5. Query OK, 1 row affected (0.00 sec)
  6.  
  7. <Test>[xuanzhi]>

这个时候发现自己删除错了，需要恢复，刚好这些数据不在最新的备份里，正常的恢复方法有两种：

一、是基于最新的完整备份+binlog进行数据恢复了，这时需要把备份导回去，还要找出Binlog DELETE前的pos位置，再进行binlog恢复，恢复完后再把记录恢复到误操的环境上。如果表很大，这时间要很久。
二、因为Binlog格式为ROW时，记录了行的修改，所以DELETE是可以看到所有列的值的，把binlog解析出来，找到被DELETE的记录，通过各种处理再恢复回去，但binlog不能基于一个库或表级别的解析，只能整个binlog解析再进行操作。

以上的方法都比较消耗时间，当然使用binlog-rollback.pl脚本有点类似第二种方法，但是binlog-rollback.pl可以指定库或表进行反向解析，还可以指定POS点，效率相当更高一些。

下面我们运行 binlog-rollback.pl 脚本，生成删除数据语句的反向SQL：

  1. [root@localhost mysql3306]# perl binlog-rollback.pl -f 'localhost-bin.000023'  -o '/data/t.sql' -u 'recovery' -p ''
  2. mysql: [Warning] Using a password on the command line interface can be insecure.
  3. mysql: [Warning] Using a password on the command line interface can be insecure.
  4. [root@localhost mysql3306]#

再次查看输出文件：

  1. [root@localhost mysql3306]# cat /data/t.sql
  2. INSERT INTO `test`.`user` SET `id`=, `name`='user1', `age`=;
  3. INSERT INTO `xuanzhi`.`tb1` SET `id`=, `name`='bb';
  4. DELETE FROM `test`.`user` WHERE `id`= AND `name`='user2' AND `age`=;
  5. DELETE FROM `test`.`user` WHERE `id`= AND `name`='user1' AND `age`=;
  6. DELETE FROM `xuanzhi`.`tb1` WHERE `id`= AND `name`='bb';
  7. DELETE FROM `xuanzhi`.`tb1` WHERE `id`= AND `name`='aa';
  8.  
  9. [root@localhost mysql3306]#

刚刚DELETE的2条记录已经生成了反向INSERT语句，这样恢复就简单多啦：

  1. INSERT INTO `test`.`user` SET `id`=, `name`='user1', `age`=;
  2. INSERT INTO `xuanzhi`.`tb1` SET `id`=, `name`='bb';

下面我们模拟修改数据的时候，误修改了，如下：

  1. <Test>[xuanzhi]> select * from xuanzhi.tb1;
  2. +------+------+
  3. | id   | name |
  4. +------+------+
  5. |    1 | aa   |
  6. +------+------+
  7. 1 row in set (0.00 sec)
  8.  
  9. <Test>[xuanzhi]> select * from test.user;
  10. +------+-------+------+
  11. | id   | name  | age  |
  12. +------+-------+------+
  13. |    2 | user2 |   30 |
  14. +------+-------+------+
  15. 1 row in set (0.00 sec)
  16.  
  17. <Test>[xuanzhi]> update  xuanzhi.tb1 set name = 'MySQL' where id=1;
  18. Query OK, 1 row affected (0.00 sec)
  19. Rows matched: 1  Changed: 1  Warnings: 0
  20.  
  21. <Test>[xuanzhi]> update test.user set age = 20 where id = 2;
  22. Query OK, 1 row affected (0.01 sec)
  23. Rows matched: 1  Changed: 1  Warnings: 0
  24.  
  25. <Test>[xuanzhi]>

这个时候发现修改错数据了，需要还原，同样可以使用脚本binlog-rollback.pl 进行对所在Binlog的DML生成反向的SQL，进行恢复：

  1. [root@localhost mysql3306]# perl binlog-rollback.pl -f 'localhost-bin.000023'  -o '/data/t.sql' -u 'recovery' -p ''
  2. mysql: [Warning] Using a password on the command line interface can be insecure.
  3. mysql: [Warning] Using a password on the command line interface can be insecure.
  4. [root@localhost mysql3306]#

再查看输出文件：

  1. [root@localhost mysql3306]# cat /data/t.sql
  2. UPDATE `test`.`user` SET `id`=, `name`='user2', `age`= WHERE `id`= AND `name`='user2' AND `age`=;
  3. UPDATE `xuanzhi`.`tb1` SET `id`=, `name`='aa' WHERE `id`= AND `name`='MySQL';
  4. INSERT INTO `test`.`user` SET `id`=, `name`='user1', `age`=;
  5. INSERT INTO `xuanzhi`.`tb1` SET `id`=, `name`='bb';
  6. DELETE FROM `test`.`user` WHERE `id`= AND `name`='user2' AND `age`=;
  7. DELETE FROM `test`.`user` WHERE `id`= AND `name`='user1' AND `age`=;
  8. DELETE FROM `xuanzhi`.`tb1` WHERE `id`= AND `name`='bb';
  9. DELETE FROM `xuanzhi`.`tb1` WHERE `id`= AND `name`='aa';
  10.  
  11. [root@localhost mysql3306]#

可以看到生成了反向的UPDATE语句：

  1. UPDATE `test`.`user` SET `id`=, `name`='user2', `age`= WHERE `id`= AND `name`='user2' AND `age`=;
  2. UPDATE `xuanzhi`.`tb1` SET `id`=, `name`='aa' WHERE `id`= AND `name`='MySQL';

下面进行指定库的反向解析，参数为（-d）

  1. [root@localhost mysql3306]# perl binlog-rollback.pl -f 'localhost-bin.000023'  -o '/data/t.sql' -u 'recovery' -p '' -d 'xuanzhi'
  2. mysql: [Warning] Using a password on the command line interface can be insecure.
  3. [root@localhost mysql3306]# cat /data/t.sql
  4. UPDATE `xuanzhi`.`tb1` SET `id`=, `name`='aa' WHERE `id`= AND `name`='MySQL';
  5. INSERT INTO `xuanzhi`.`tb1` SET `id`=, `name`='bb';
  6. DELETE FROM `xuanzhi`.`tb1` WHERE `id`= AND `name`='bb';
  7. DELETE FROM `xuanzhi`.`tb1` WHERE `id`= AND `name`='aa';
  8.  
  9. [root@localhost mysql3306]#

可以看到输入的文件只含xuanzhi库的所有DML的反向SQL。

下面进行指定库下某个表的反向解析，参数为：-T （为了看到效果在xuanzhi库下的tb2表删除一些记录）：

  1. <Test>[xuanzhi]> select * from tb2;
  2. +------+------+
  3. | id   | name |
  4. +------+------+
  5. |    1 | aa   |
  6. |    2 | bb   |
  7. |    3 | cc   |
  8. +------+------+
  9. 3 rows in set (0.04 sec)
  10.  
  11. <Test>[xuanzhi]> delete from xuanzhi.tb2 where id <2;
  12. Query OK, 1 row affected (0.02 sec)
  13.  
  14. <Test>[xuanzhi]>

这个时候应该如果只指定xuanzhi库,那么tb1和tb2的DML操作的反向操作都会记录下来：

  1. [root@localhost mysql3306]# perl binlog-rollback.pl -f 'localhost-bin.000023'  -o '/data/t.sql' -u 'recovery' -p '' -d 'xuanzhi'
  2. mysql: [Warning] Using a password on the command line interface can be insecure.
  3. mysql: [Warning] Using a password on the command line interface can be insecure.
  4. [root@localhost mysql3306]# cat /data/t.sql
  5. INSERT INTO `xuanzhi`.`tb2` SET `id`=, `name`='aa';
  6. UPDATE `xuanzhi`.`tb1` SET `id`=, `name`='aa' WHERE `id`= AND `name`='MySQL';
  7. INSERT INTO `xuanzhi`.`tb1` SET `id`=, `name`='bb';
  8. DELETE FROM `xuanzhi`.`tb1` WHERE `id`= AND `name`='bb';
  9. DELETE FROM `xuanzhi`.`tb1` WHERE `id`= AND `name`='aa';
  10.  
  11. [root@localhost mysql3306]#

指定单个表tb2：

  1. [root@localhost mysql3306]# perl binlog-rollback.pl -f 'localhost-bin.000023'  -o '/data/t.sql' -u 'recovery' -p '' -d 'xuanzhi' -T 'tb2'
  2. mysql: [Warning] Using a password on the command line interface can be insecure.
  3. [root@localhost mysql3306]# cat /data/t.sql
  4. INSERT INTO `xuanzhi`.`tb2` SET `id`=, `name`='aa';
  5.  
  6. [root@localhost mysql3306]#

因为上面删除了一条tb2的数据，所有这个文件就对应生成一条tb2的INSERT记录

下面进行POS点生成反向SQL：（--start-position=  --stop-position=）

  1. # at
  2. #  :: server id   end_log_pos  CRC32 0xb67ef6ba   Query   thread_id=    exec_time=     error_code=
  3. SET TIMESTAMP=/*!*/;
  4. BEGIN
  5. /*!*/;
  6. # at
  7. #  :: server id   end_log_pos  CRC32 0x219a127c   Table_map: `test`.`user` mapped to number
  8. # at
  9. #  :: server id   end_log_pos  CRC32 0xf5e0d39e   Update_rows: table id  flags: STMT_END_F
  10.  
  11. BINLOG '
  12. K+TdVhPqlBMAMwAAAJMGAAAAAEoAAAAAAAEABHRlc3QABHVzZXIAAwP+AwL+Hgd8Epoh
  13. K+TdVh/qlBMAQgAAANUGAAAAAEoAAAAAAAEAAgAD///4AgAAAAV1c2VyMh4AAAD4AgAAAAV1c2Vy
  14. MhQAAACe0+D1
  15. '/*!*/;
  16. ### UPDATE `test`.`user`
  17. ### WHERE
  18. ###   @=
  19. ###   @='user2'
  20. ###   @=
  21. ### SET
  22. ###   @=
  23. ###   @='user2'
  24. ###   @=
  25. # at
  26. #  :: server id   end_log_pos  CRC32 0x1e62cb77   Xid =
  27. COMMIT/*!*/;
  28. # at
  29. #  :: server id   end_log_pos  CRC32 0x04dfe1f0   Query   thread_id=    exec_time=     error_code=
  30. SET TIMESTAMP=/*!*/;
  31. BEGIN
  32. /*!*/;
  33. # at
  34. #  :: server id   end_log_pos  CRC32 0x897ae6bd   Table_map: `xuanzhi`.`tb2` mapped to number
  35. # at
  36. #  :: server id   end_log_pos  CRC32 0xea61aff0   Delete_rows: table id  flags: STMT_END_F
  37.  
  38. BINLOG '
  39. QOfdVhPqlBMANAAAAHMHAAAAAEYAAAAAAAEAB3h1YW56aGkAA3RiMgACA/4C/goDveZ6iQ==
  40. QOfdViDqlBMAKwAAAJ4HAAAAAEYAAAAAAAEAAgAC//wBAAAAAmFh8K9h6g==
  41. '/*!*/;
  42. ### DELETE FROM `xuanzhi`.`tb2`
  43. ### WHERE
  44. ###   @=
  45. ###   @='aa'
  46. # at
  47. #  :: server id   end_log_pos  CRC32 0x49e1ce9c   Xid =
  48. COMMIT/*!*/;

从上面的binlog可以看到开始的--start-position=1557 结束的--stop-position=1981，这一段binlog里做了UPDATE `test`.`user` ... 和 DELETE FROM `xuanzhi`.`tb2` ... 的操作，那么用binlog-rollback.pl应该会生成一个UPDATE和一个INSERT语句

  1. [root@localhost mysql3306]# perl binlog-rollback.pl -f 'localhost-bin.000023'  -o '/data/t.sql' -u 'recovery' -p '123456' --start-position=1557  --stop-position=1981
  2. mysql: [Warning] Using a password on the command line interface can be insecure.
  3. mysql: [Warning] Using a password on the command line interface can be insecure.
  4. [root@localhost mysql3306]# cat /data/t.sql
  5. INSERT INTO `xuanzhi`.`tb2` SET `id`=1, `name`='aa';
  6. UPDATE `test`.`user` SET `id`=2, `name`='user2', `age`=30 WHERE `id`=2 AND `name`='user2' AND `age`=20;
  7.  
  8. [root@localhost mysql3306]#

更多的测试，就看同学们了，有测试不当的地方请告诉我，大家一起学习。

总结： 一、感谢那些有分享精神的大神们，让我们学到了更多的东西，但开源的脚本需要多测试。

         二、误操的情况，时有发生，所以我们要做好备份，做好一些数据恢复的测试。

         三、该脚本在处理比较在的binlog时，会经常出现些小问题

作者：陆炫志

出处：xuanzhi的博客 http://www.cnblogs.com/xuanzhi201111

您的支持是对博主最大的鼓励，感谢您的认真阅读。本文版权归作者所有，欢迎转载，但请保留该声明。

MySQL基于ROW格式的数据恢复的更多相关文章

  1. 浅析MySQL基于ROW格式的二进制日志

    上文分析的二进制日志实际上是基于STATEMENT格式的,下面我们来看看基于ROW格式的二进制日志,毕竟,两者对应的binlog事件类型也不一样,同时,很多童鞋反映基于ROW格式的二进制日志无法查到原 ...

  2. my42_Mysql基于ROW格式的主从同步

    模拟主从update事务,从库跳过部分update事务后,再次开始同步的现象 主库 mysql> select * from dbamngdb.isNodeOK; +----+--------- ...

  3. MySQL Binlog Mixed模式记录成Row格式

    背景: 一个简单的主从结构,主的binlog format是Mixed模式,在执行一条简单的导入语句时,通过mysqlbinlog导出发现记录的Binlog全部变成了Row的格式(明明设置的是Mixe ...

  4. my15_ mysql binlog格式从mixed修改为row格式

    由于主库繁忙,就在从库上修改binlog格式 1. 从库切日志mysql> flush logs;Query OK, 0 rows affected (0.00 sec) mysql> f ...

  5. 基于GTID环境的数据恢复

    下面说一个线上环境的数据恢复案例,线上环境误操作的事情,随时都有可能发生的,这时候运维DBA或者DBA会想,没人会傻到把数据库干掉的吧?又或者没有会闲得蛋痛删除了几条数据又想恢复这么无聊吧?适适这样的 ...

  6. Mysql基于GTID主从复制

    Mysql5.6基于GTID全局事务的复制 什么是GTID?   GTID(Global Transaction Identifiers)是全局事务标识 当使用GTIDS时,在主上提交的每一个事务都会 ...

  7. MySQL二进制日志格式对复制的影响

    复制的分类 基于SQL语句的复制 - SBR 主库二进制日志格式使用STATEMENT 在MySQL 5.1之前仅存在SBR模式, 又称之为逻辑复制. 主库记录CUD操作的SQL语句, 从库会读取并重 ...

  8. MySQL基于binlog主从复制

    MySQL复制介绍 默认情况 下复制是异步进行的,从库也不需要一直连接到主库来同步数据 MySQL复制的数据粒度可以是主实例上所有的数据库,也可以是指定的一个或多个数据库 ,也可以是一个数据库里的指定 ...

  9. Mysql基于GTID复制模式-运维小结 (完整篇)

    先来看mysql5.6主从同步操作时遇到的一个报错:mysql> change master to master_host='192.168.10.59',master_user='repli' ...

随机推荐

  1. ieda 运行web--导入其它jar包

    == 配置 或选择tomcat 1.打开IntelliJ IDEA,点击菜单的“Run”->“Edit Configurations...”,然后会打开运行设置弹框.2.在弹框中,点击左上角的“ ...

  2. POJ 3253 Fence Repair （贪心）

    题意:将一块木板切成N块,长度分别为:a1,a2,……an,每次切割木板的开销为当前木板的长度.求出按照要求将木板切割完毕后的最小开销. 思路:比较奇特的贪心 每次切割都会将当前木板一分为二,可以按切 ...

  3. MySQL软件基本管理

    1. 忘记密码 windows平台下,5.6.43版本mysql # 关闭mysql # 在cmd中执行:mysqld --skip-grant-tables # 在cmd中执行:mysql # 执行 ...

  4. Linux内存分配小结--malloc、brk、mmap【转】

    转自:https://blog.csdn.net/gfgdsg/article/details/42709943 http://blog.163.com/xychenbaihu@yeah/blog/s ...

  5. hibernate框架学习之一级缓存

    l缓存是存储数据的临时空间,减少从数据库中查询数据的次数 lHibernate中提供有两种缓存机制 •一级缓存(Hibernate自身携带) •二级缓存(使用外部技术) lHibernate的一级缓存 ...

  6. 训练报告 (2014-2015) 2014, Samara SAU ACM ICPC Quarterfinal Qualification Contest

    Solved A Gym 100488A Yet Another Goat in the Garden   B Gym 100488B Impossible to Guess Solved C Gym ...

  7. C# 对话框使用整理

    1.保存文件对话框 SaveFileDialog saveFile = new SaveFileDialog(); saveFile.Title = "save file"; sa ...

  8. 修改JDK版本配置

    我使用的maven是3.0.5版本的,在创建项目的时候,默认使用的jdk为1.5版本 在项目的pom.xml中添加如下配置可修改使用的jdk版本. <properties> <!-- ...

  9. 【转】Jmeter中使用CSV Data Set Config参数化不重复数据执行N遍

    Jmeter中使用CSV Data Set Config参数化不重复数据执行N遍 要求: 今天要测试上千条数据,且每条数据要求执行多次,(模拟多用户多次抽奖) 1.用户id有175个,且没有任何排序规 ...

  10. UpdatePanel1里面使用FileUpload控件

    最近做项目过程中,遇到了UpdatePanel1里面放了一个FileUpload控件,结果从后台就获取不到上传的文件了,找了好久才找到原因.原因: 加了红色部分后立马获取到了.