CentOS6.8下MySQL MHA架构搭建笔记
转载请注明出处，本文地址：http://www.cnblogs.com/ajiangg/p/6552855.html
以下是CentOS6.8下MySQL MHA架构搭建笔记
IP资源规划：
192.168.206.139 master
192.168.206.140 slave01(备用master)
192.168.206.141 slave02
192.168.206.142 manager
192.168.206.145 VIP
一、准备工作：
1.关闭Selinux
[root@localhost ~]# vi /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
[root@localhost ~]# reboot
2.关闭防火墙
[root@localhost ~]# chkconfig iptables off
[root@localhost ~]# service iptables stop
3.安装epel yum源
[root@localhost tmp]# rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
[root@localhost tmp]# rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-
4.安装MHA node所需的perl模块（DBD:mysql）
[root@localhost tmp]# yum -y install perl-DBD-MySQL ncftp perl-Config-Tiny perl-Log-Dispatch perl-Parallel-ForkManager perl-Time-HiRes perl-devel
二、Replication搭建
1.建立replication用户(master和slave01)
mysql> GRANT REPLICATION SLAVE ON *.* TO 'u_repl'@'192.168.206.%' IDENTIFIED BY 'replpass' with grant option;
2.建立MHA管理用用户
mysql> GRANT ALL PRIVILEGES ON *.* to 'root'@'192.168.206.%' IDENTIFIED BY 'checkpass';
3.安装半同步复制插件
mysql> INSTALL PLUGIN rpl_semi_sync_master SONAME 'semisync_master.so';
mysql> INSTALL PLUGIN rpl_semi_sync_slave SONAME 'semisync_slave.so';
4.修改my.cnf配置文件
[root@localhost tmp]# vi /etc/my.cnf
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html [mysqld]
#
# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at % of total RAM for dedicated server, else %.
# innodb_buffer_pool_size = 128M
#
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
#
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
# join_buffer_size = 128M
# sort_buffer_size = 2M
# read_rnd_buffer_size = 2M
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
server_id=
log_bin=/var/lib/mysql/mysql-bin
binlog-format=ROW
binlog-checksum=CRC32
log_slave_updates=true
gtid-mode=on
enforce-gtid-consistency=true
master_info_repository=TABLE
sync-master-info=
slave-parallel-workers=
master-verify-checksum=
slave-sql-verify-checksum=
relay_log=/var/lib/mysql/localhost-relay-bin
relay_log_purge=
relay_log_recovery=
rpl_semi_sync_master_enabled=ON
rpl_semi_sync_slave_enabled=ON # Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links= log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
5.master备份slave还原（略），查找master Binlog位置
[root@localhost tmp]# mysql -uroot -p
mysql> show master status\G;
*************************** . row ***************************
File: mysql-bin.
Position:
Binlog_Do_DB:
Binlog_Ignore_DB:
Executed_Gtid_Set:
row in set (0.00 sec)
6.各slave上执行脚本，建立主从配置。(MySQL5.7可以直接使用master_auto_position=1参数)
mysql> change master to master_host='192.168.206.139',master_user='u_repl',master_password='replpass',master_log_file='mysql-bin.000001' ,master_log_pos=154;
mysql> start slave;
三、开始安装配置
1.各节点使用ssh-keygen实现三台机器之间相互免密钥登录
Memo:如果是直接通过修改/root/.ssh/authorized_keys，需要在安装manager的服务器上把自己的key也加进去，否则masterha_check_ssh验证通不过。
[root@localhost tmp]# ssh-keygen -t rsa
[root@localhost tmp]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
d5:0d::::::f0:e4:::::0d:1b:ba root@localhost.localdomain
The key's randomart image is:
+--[ RSA ]----+
| o+.=B&|
| .oo*oB=|
| ...o o +|
| .. o |
| SE |
| |
| |
| |
| |
+-----------------+
[root@localhost .ssh]# ssh-copy-id -i /root/.ssh/id_rsa.pub root@192.168.206.140
[root@localhost .ssh]# ssh-copy-id -i /root/.ssh/id_rsa.pub root@192.168.206.140
root@192.168.206.140's password:
Now try logging into the machine, with "ssh 'root@192.168.206.140'", and check in: .ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting.
[root@localhost .ssh]# ssh-copy-id -i /root/.ssh/id_rsa.pub root@192.168.206.141
[root@localhost .ssh]# ssh-copy-id -i /root/.ssh/id_rsa.pub root@192.168.206.142
2.所有节点(包括manager)下载安装mha4mysql-node-0.56
由于google被墙，我是直接从网上下载的rpm文件，具体下载过程略。
[root@localhost tmp]# yum localinstall mha4mysql-node-0.56-.el6.noarch.rpm
[root@localhost tmp]# yum localinstall mha4mysql-node-0.56-.el6.noarch.rpm
Loaded plugins: fastestmirror, refresh-packagekit, security
Setting up Local Package Process
Examining mha4mysql-node-0.56-.el6.noarch.rpm: mha4mysql-node-0.56-.el6.noarch
Marking mha4mysql-node-0.56-.el6.noarch.rpm to be installed
Loading mirror speeds from cached hostfile
* base: ftp.sjtu.edu.cn
* epel: mirrors.tuna.tsinghua.edu.cn
* extras: centos.ustc.edu.cn
* updates: ftp.sjtu.edu.cn
Resolving Dependencies
--> Running transaction check
---> Package mha4mysql-node.noarch :0.56-.el6 will be installed
--> Finished Dependency Resolution Dependencies Resolved ==============================================================================================================================================================================================
Package Arch Version Repository Size
==============================================================================================================================================================================================
Installing:
mha4mysql-node noarch 0.56-.el6 /mha4mysql-node-0.56-.el6.noarch k Transaction Summary
==============================================================================================================================================================================================
Install Package(s) Total size: k
Installed size: k
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : mha4mysql-node-0.56-.el6.noarch /
Verifying : mha4mysql-node-0.56-.el6.noarch / Installed:
mha4mysql-node.noarch :0.56-.el6 Complete!
确认安装内容
[root@localhost tmp]# rpm -ql mha4mysql-node
[root@localhost tmp]# rpm -ql mha4mysql-node
/usr/bin/apply_diff_relay_logs
/usr/bin/filter_mysqlbinlog
/usr/bin/purge_relay_logs
/usr/bin/save_binary_logs
/usr/share/man/man1/apply_diff_relay_logs..gz
/usr/share/man/man1/filter_mysqlbinlog..gz
/usr/share/man/man1/purge_relay_logs..gz
/usr/share/man/man1/save_binary_logs..gz
/usr/share/perl5/vendor_perl/MHA/BinlogHeaderParser.pm
/usr/share/perl5/vendor_perl/MHA/BinlogManager.pm
/usr/share/perl5/vendor_perl/MHA/BinlogPosFindManager.pm
/usr/share/perl5/vendor_perl/MHA/BinlogPosFinder.pm
/usr/share/perl5/vendor_perl/MHA/BinlogPosFinderElp.pm
/usr/share/perl5/vendor_perl/MHA/BinlogPosFinderXid.pm
/usr/share/perl5/vendor_perl/MHA/NodeConst.pm
/usr/share/perl5/vendor_perl/MHA/NodeUtil.pm
/usr/share/perl5/vendor_perl/MHA/SlaveUtil.pm
3.manager节点安装mha4mysql-manager-0.56
[root@localhost tmp]# yum localinstall mha4mysql-manager-0.56-.el6.noarch.rpm
[root@localhost tmp]# yum localinstall mha4mysql-manager-0.56-.el6.noarch.rpm
Loaded plugins: fastestmirror, refresh-packagekit, security
Setting up Local Package Process
Examining mha4mysql-manager-0.56-.el6.noarch.rpm: mha4mysql-manager-0.56-.el6.noarch
Marking mha4mysql-manager-0.56-.el6.noarch.rpm to be installed
Loading mirror speeds from cached hostfile
* base: mirrors..com
* epel: mirror01.idc.hinet.net
* extras: mirrors.cn99.com
* updates: mirrors.cn99.com
Resolving Dependencies
--> Running transaction check
---> Package mha4mysql-manager.noarch :0.56-.el6 will be installed
--> Finished Dependency Resolution Dependencies Resolved ==============================================================================================================================================================================================
Package Arch Version Repository Size
==============================================================================================================================================================================================
Installing:
mha4mysql-manager noarch 0.56-.el6 /mha4mysql-manager-0.56-.el6.noarch k Transaction Summary
==============================================================================================================================================================================================
Install Package(s) Total size: k
Installed size: k
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : mha4mysql-manager-0.56-.el6.noarch /
Verifying : mha4mysql-manager-0.56-.el6.noarch / Installed:
mha4mysql-manager.noarch :0.56-.el6 Complete!
确认安装内容
[root@localhost tmp]# rpm -ql mha4mysql-manager
[root@localhost tmp]# rpm -ql mha4mysql-node
/usr/bin/apply_diff_relay_logs
/usr/bin/filter_mysqlbinlog
/usr/bin/purge_relay_logs
/usr/bin/save_binary_logs
/usr/share/man/man1/apply_diff_relay_logs..gz
/usr/share/man/man1/filter_mysqlbinlog..gz
/usr/share/man/man1/purge_relay_logs..gz
/usr/share/man/man1/save_binary_logs..gz
/usr/share/perl5/vendor_perl/MHA/BinlogHeaderParser.pm
/usr/share/perl5/vendor_perl/MHA/BinlogManager.pm
/usr/share/perl5/vendor_perl/MHA/BinlogPosFindManager.pm
/usr/share/perl5/vendor_perl/MHA/BinlogPosFinder.pm
/usr/share/perl5/vendor_perl/MHA/BinlogPosFinderElp.pm
/usr/share/perl5/vendor_perl/MHA/BinlogPosFinderXid.pm
/usr/share/perl5/vendor_perl/MHA/NodeConst.pm
/usr/share/perl5/vendor_perl/MHA/NodeUtil.pm
/usr/share/perl5/vendor_perl/MHA/SlaveUtil.pm
[root@localhost tmp]# rpm -ql mha4mysql-manager
/usr/bin/masterha_check_repl
/usr/bin/masterha_check_ssh
/usr/bin/masterha_check_status
/usr/bin/masterha_conf_host
/usr/bin/masterha_manager
/usr/bin/masterha_master_monitor
/usr/bin/masterha_master_switch
/usr/bin/masterha_secondary_check
/usr/bin/masterha_stop
/usr/share/man/man1/masterha_check_repl..gz
/usr/share/man/man1/masterha_check_ssh..gz
/usr/share/man/man1/masterha_check_status..gz
/usr/share/man/man1/masterha_conf_host..gz
/usr/share/man/man1/masterha_manager..gz
/usr/share/man/man1/masterha_master_monitor..gz
/usr/share/man/man1/masterha_master_switch..gz
/usr/share/man/man1/masterha_secondary_check..gz
/usr/share/man/man1/masterha_stop..gz
/usr/share/perl5/vendor_perl/MHA/Config.pm
/usr/share/perl5/vendor_perl/MHA/DBHelper.pm
/usr/share/perl5/vendor_perl/MHA/FileStatus.pm
/usr/share/perl5/vendor_perl/MHA/HealthCheck.pm
/usr/share/perl5/vendor_perl/MHA/ManagerAdmin.pm
/usr/share/perl5/vendor_perl/MHA/ManagerAdminWrapper.pm
/usr/share/perl5/vendor_perl/MHA/ManagerConst.pm
/usr/share/perl5/vendor_perl/MHA/ManagerUtil.pm
/usr/share/perl5/vendor_perl/MHA/MasterFailover.pm
/usr/share/perl5/vendor_perl/MHA/MasterMonitor.pm
/usr/share/perl5/vendor_perl/MHA/MasterRotate.pm
/usr/share/perl5/vendor_perl/MHA/SSHCheck.pm
/usr/share/perl5/vendor_perl/MHA/Server.pm
/usr/share/perl5/vendor_perl/MHA/ServerManager.pm
4.在manager节点管理MHA配置文件
前面采用yum安装的，没有samples文件夹，所以另外down了一份mha4mysql-manager-0.56.tar.gz包（具体过程略）
[root@localhost tmp]# tar xf mha4mysql-manager-0.56.tar.gz
[root@localhost tmp]# cd mha4mysql-manager-0.56
[root@localhost mha4mysql-manager-0.56]# mkdir -p /etc/mha/{app1,scripts}
[root@localhost mha4mysql-manager-0.56]# cp -r samples/conf/* /etc/mha/
[root@localhost mha4mysql-manager-0.56]# cp -r samples/scripts/* /etc/mha/scripts/
[root@localhost mha4mysql-manager-0.56]# mv /etc/mha/app1.cnf /etc/mha/app1/
[root@localhost mha4mysql-manager-0.56]# mv /etc/mha/masterha_default.cnf /etc/masterha_default.cnf
5.在manager上设置全局配置
修改master_ip_failover脚本，添加VIP漂移相关三个参数
[root@localhost mha4mysql-manager-0.56]# vi /etc/mha/scripts/master_ip_failover
[root@localhost scripts]# vi master_ip_failover #!/usr/bin/env perl # Copyright (C) DeNA Co.,Ltd.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc.,
# Franklin Street, Fifth Floor, Boston, MA - USA ## Note: This is a sample script and is not complete. Modify the script based on your environment. use strict;
use warnings FATAL => 'all'; use Getopt::Long;
use MHA::DBHelper; my (
$command, $ssh_user, $orig_master_host,
$orig_master_ip, $orig_master_port, $new_master_host,
$new_master_ip, $new_master_port, $new_master_user,
$new_master_password,
$virtual_ip, $orig_master_vip_eth, $new_master_vip_eth
);
GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
'new_master_user=s' => \$new_master_user,
'new_master_password=s' => \$new_master_password,
'virtual_ip=s' => \$virtual_ip,
'orig_master_vip_eth=s' => \$orig_master_vip_eth,
'new_master_vip_eth=s' => \$new_master_vip_eth
); exit &main(); sub main {
if ( $command eq "stop" || $command eq "stopssh" ) { # $orig_master_host, $orig_master_ip, $orig_master_port are passed.
# If you manage master ip address at global catalog database,
# invalidate orig_master_ip here.
my $exit_code = ;
eval { # updating global catalog, etc
`ssh $orig_master_host -o "ConnectTimeout=5" '/sbin/ifconfig $orig_master_vip_eth down'`;
$exit_code = ;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) { # all arguments are passed.
# If you manage master ip address at global catalog database,
# activate new_master_ip here.
# You can also grant write access (create user, set read_only=, etc) here.
my $exit_code = ;
eval {
my $new_master_handler = new MHA::DBHelper(); # args: hostname, port, user, password, raise_error_or_not
$new_master_handler->connect( $new_master_ip, $new_master_port,
$new_master_user, $new_master_password, ); ## Set read_only= on the new master
$new_master_handler->disable_log_bin_local();
print "Set read_only=0 on the new master.\n";
$new_master_handler->disable_read_only(); ## Creating an app user on the new master
print "Creating app user on the new master..\n";
# FIXME_xxx_create_user( $new_master_handler->{dbh} );
$new_master_handler->enable_log_bin_local();
$new_master_handler->disconnect(); ## Update master ip on the catalog database, etc
# FIXME_xxx; my $real_eth=$new_master_vip_eth;
$real_eth=~ s/:.*//g;
my $new_master_network= `ssh $new_master_host '/sbin/ifconfig $real_eth'`;
my $vip_netmask = join(".", $new_master_network =~ /Mask\:(\d+)\.(\d+)\.(\d+)\.(\d+)/ );
my $vip_broadcast = join(".", $new_master_network =~ /Bcast\:(\d+)\.(\d+)\.(\d+)\.(\d+)/ );
`ssh $new_master_host -o "ConnectTimeout=15" '/sbin/ifconfig $new_master_vip_eth $virtual_ip netmask $vip_netmask up; /sbin/arping -c 3 -A $virtual_ip; ping -I $virtual_ip -b $vip_broadcast -c 3 -c 3 -i 0.01 -q -W 10 > /dev/null'`; $exit_code = ;
};
if ($@) {
warn $@; # If you want to continue failover, exit .
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "status" ) { # do nothing
exit ;
}
else {
&usage();
exit ;
}
} sub usage {
"Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port --virtual_ip=ip --orig_master_vip_eth=eth --new_master_vip_eth=eth\n";
}
修改master_ip_online_change脚本，添加VIP漂移相关三个参数
[root@localhost scripts]# vi master_ip_online_change
[root@localhost scripts]# vi master_ip_online_change #!/usr/bin/env perl # Copyright (C) DeNA Co.,Ltd.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc.,
# Franklin Street, Fifth Floor, Boston, MA - USA ## Note: This is a sample script and is not complete. Modify the script based on your environment. use strict;
use warnings FATAL => 'all'; use Getopt::Long;
use MHA::DBHelper;
use MHA::NodeUtil;
use Time::HiRes qw( sleep gettimeofday tv_interval );
use Data::Dumper; my $_tstart;
my $_running_interval = 0.1;
my (
$command, $orig_master_is_new_slave, $orig_master_host,
$orig_master_ip, $orig_master_port, $orig_master_user,
$orig_master_password, $orig_master_ssh_user, $new_master_host,
$new_master_ip, $new_master_port, $new_master_user,
$new_master_password, $new_master_ssh_user,
$virtual_ip, $orig_master_vip_eth, $new_master_vip_eth
);
GetOptions(
'command=s' => \$command,
'orig_master_is_new_slave' => \$orig_master_is_new_slave,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'orig_master_user=s' => \$orig_master_user,
'orig_master_password=s' => \$orig_master_password,
'orig_master_ssh_user=s' => \$orig_master_ssh_user,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
'new_master_user=s' => \$new_master_user,
'new_master_password=s' => \$new_master_password,
'new_master_ssh_user=s' => \$new_master_ssh_user,
'virtual_ip=s' => \$virtual_ip,
'orig_master_vip_eth=s' => \$orig_master_vip_eth,
'new_master_vip_eth=s' => \$new_master_vip_eth
); exit &main(); sub current_time_us {
my ( $sec, $microsec ) = gettimeofday();
my $curdate = localtime($sec);
return $curdate . " " . sprintf( "%06d", $microsec );
} sub sleep_until {
my $elapsed = tv_interval($_tstart);
if ( $_running_interval > $elapsed ) {
sleep( $_running_interval - $elapsed );
}
} sub get_threads_util {
my $dbh = shift;
my $my_connection_id = shift;
my $running_time_threshold = shift;
my $type = shift;
$running_time_threshold = unless ($running_time_threshold);
$type = unless ($type);
my @threads; my $sth = $dbh->prepare("SHOW PROCESSLIST");
$sth->execute(); while ( my $ref = $sth->fetchrow_hashref() ) {
my $id = $ref->{Id};
my $user = $ref->{User};
my $host = $ref->{Host};
my $command = $ref->{Command};
my $state = $ref->{State};
my $query_time = $ref->{Time};
my $info = $ref->{Info};
$info =~ s/^\s*(.*?)\s*$/$/ if defined($info);
next if ( $my_connection_id == $id );
next if ( defined($query_time) && $query_time < $running_time_threshold );
next if ( defined($command) && $command eq "Binlog Dump" );
next if ( defined($user) && $user eq "system user" );
next
if ( defined($command)
&& $command eq "Sleep"
&& defined($query_time)
&& $query_time >= ); if ( $type >= ) {
next if ( defined($command) && $command eq "Sleep" );
next if ( defined($command) && $command eq "Connect" );
} if ( $type >= ) {
next if ( defined($info) && $info =~ m/^select/i );
next if ( defined($info) && $info =~ m/^show/i );
} push @threads, $ref;
}
return @threads;
} sub main {
if ( $command eq "stop" ) {
## Gracefully killing connections on the current master
# . Set read_only= on the new master
# . DROP USER so that no app user can establish new connections
# . Set read_only= on the current master
# . Kill current queries
# * Any database access failure will result in script die.
my $exit_code = ;
eval {
## Setting read_only= on the new master (to avoid accident)
my $new_master_handler = new MHA::DBHelper(); # args: hostname, port, user, password, raise_error(die_on_error)_or_not
$new_master_handler->connect( $new_master_ip, $new_master_port,
$new_master_user, $new_master_password, );
print current_time_us() . " Set read_only on the new master.. ";
$new_master_handler->enable_read_only();
if ( $new_master_handler->is_read_only() ) {
print "ok.\n";
}
else {
die "Failed!\n";
}
$new_master_handler->disconnect(); # Connecting to the orig master, die if any database error happens
my $orig_master_handler = new MHA::DBHelper();
$orig_master_handler->connect( $orig_master_ip, $orig_master_port,
$orig_master_user, $orig_master_password, ); ## Drop application user so that nobody can connect. Disabling per-session binlog beforehand
$orig_master_handler->disable_log_bin_local();
print current_time_us() . " Drpping app user on the orig master..\n";
# FIXME_xxx_drop_app_user($orig_master_handler); ## Waiting for N * milliseconds so that current connections can exit
my $time_until_read_only = ;
$_tstart = [gettimeofday];
my @threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
while ( $time_until_read_only > && $#threads >= ) {
if ( $time_until_read_only % == ) {
printf
"%s Waiting all running %d threads are disconnected.. (max %d milliseconds)\n",
current_time_us(), $#threads + , $time_until_read_only * ;
if ( $#threads < ) {
print Data::Dumper->new( [$_] )->Indent()->Terse()->Dump . "\n"
foreach (@threads);
}
}
sleep_until();
$_tstart = [gettimeofday];
$time_until_read_only--;
@threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
} ## Setting read_only= on the current master so that nobody(except SUPER) can write
print current_time_us() . " Set read_only=1 on the orig master.. ";
$orig_master_handler->enable_read_only();
if ( $orig_master_handler->is_read_only() ) {
print "ok.\n";
}
else {
die "Failed!\n";
} ## Waiting for M * milliseconds so that current update queries can complete
my $time_until_kill_threads = ;
@threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
while ( $time_until_kill_threads > && $#threads >= ) {
if ( $time_until_kill_threads % == ) {
printf
"%s Waiting all running %d queries are disconnected.. (max %d milliseconds)\n",
current_time_us(), $#threads + , $time_until_kill_threads * ;
if ( $#threads < ) {
print Data::Dumper->new( [$_] )->Indent()->Terse()->Dump . "\n"
foreach (@threads);
}
}
sleep_until();
$_tstart = [gettimeofday];
$time_until_kill_threads--;
@threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
} ## Terminating all threads
print current_time_us() . " Killing all application threads..\n";
$orig_master_handler->kill_threads(@threads) if ( $#threads >= );
print current_time_us() . " done.\n";
$orig_master_handler->enable_log_bin_local();
$orig_master_handler->disconnect(); ## After finishing the script, MHA executes FLUSH TABLES WITH READ LOCK
$exit_code = ;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
## Activating master ip on the new master
# . Create app user with write privileges
# . Moving backup script if needed
# . Register new master's ip to the catalog database # We don't return error even though activating updatable accounts/ip failed so that we don't interrupt slaves' recovery.
# If exit code is or , MHA does not abort
my $exit_code = ;
eval {
my $new_master_handler = new MHA::DBHelper(); # args: hostname, port, user, password, raise_error_or_not
$new_master_handler->connect( $new_master_ip, $new_master_port,
$new_master_user, $new_master_password, ); ## Set read_only= on the new master
$new_master_handler->disable_log_bin_local();
print current_time_us() . " Set read_only=0 on the new master.\n";
$new_master_handler->disable_read_only(); ## Creating an app user on the new master
print current_time_us() . " Creating app user on the new master..\n";
# FIXME_xxx_create_app_user($new_master_handler);
$new_master_handler->enable_log_bin_local();
$new_master_handler->disconnect(); ## Update master ip on the catalog database, etc
`ssh $orig_master_host -o "ConnectTimeout=5" '/sbin/ifconfig $orig_master_vip_eth down'`;
my $real_eth=$new_master_vip_eth;
$real_eth=~ s/:.*//g;
my $new_master_network= `ssh $new_master_host '/sbin/ifconfig $real_eth'`;
my $vip_netmask = join(".", $new_master_network =~ /Mask\:(\d+)\.(\d+)\.(\d+)\.(\d+)/ );
my $vip_broadcast = join(".", $new_master_network =~ /Bcast\:(\d+)\.(\d+)\.(\d+)\.(\d+)/ );
`ssh $new_master_host -o "ConnectTimeout=15" '/sbin/ifconfig $new_master_vip_eth $virtual_ip netmask $vip_netmask up; /sbin/arping -c 3 -A $virtual_ip; ping -I $virtual_ip -b $vip_broadcast -c 3 -c 3 -i 0.01 -q -W 10 > /dev/null'`; $exit_code = ;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "status" ) { # do nothing
exit ;
}
else {
&usage();
exit ;
}
} sub usage {
"Usage: master_ip_online_change --command=start|stop|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port --virtual_ip=ip --orig_master_vip_eth=eth --new_master_vip_eth=eth\n";
die;
}
修改masterha_default.cnf全局配置
[root@localhost mha4mysql-manager-0.56]# vi /etc/masterha_default.cnf
[root@localhost scripts]# vi /etc/masterha_default.cnf
[server default]
user=root
password=checkpass
ssh_user=root
repl_user=u_repl
repl_password=replpass
master_binlog_dir= /var/lib/mysql,/var/log/mysql
remote_workdir=/data/log/masterha
secondary_check_script= masterha_secondary_check -s 192.168.206.140 -s 192.168.206.139 --user=root --master_host=192.168.206.139 --master_port=
ping_interval=
master_ip_failover_script= /etc/mha/scripts/master_ip_failover --virtual_ip=192.168.206.145 --orig_master_vip_eth=eth0: --new_master_vip_eth=eth0:
# shutdown_script= /etc/mha/scripts/power_manager
# report_script= /etc/mha/scripts/send_report
master_ip_online_change_script= /etc/mha/scripts/master_ip_online_change --virtual_ip=192.168.206.145 --orig_master_vip_eth=eth0: --new_master_vip_eth=eth0:
创建日志目录：
[root@localhost mha4mysql-manager-0.56]# mkdir -p /var/log/mha/app1
配置app1.cnf：
[root@localhost mha4mysql-manager-0.56]# vi /etc/mha/app1/app1.cnf
[root@localhost mha4mysql-manager-0.56]# vi /etc/mha/app1/app1.cnf [server default]
manager_workdir=/var/log/mha/app1
manager_log=/var/log/mha/app1/manager.log [server1]
hostname=192.168.206.139
master_binlog_dir=/var/lib/mysql/
candidate_master= [server2]
hostname=192.168.206.140
master_binlog_dir=/var/lib/mysql/
candidate_master= [server3]
hostname=192.168.206.141
master_binlog_dir=/var/lib/mysql/
no_master=
6.验证安装内容
验证ssh信任
[root@localhost mha4mysql-manager-0.56]# masterha_check_ssh --conf=/etc/mha/app1/app1.cnf
[root@localhost mha4mysql-manager-0.56]# masterha_check_ssh --conf=/etc/mha/app1/app1.cnf
Thu Mar :: - [info] Reading default configuration from /etc/masterha_default.cnf..
Thu Mar :: - [info] Reading application default configuration from /etc/mha/app1/app1.cnf..
Thu Mar :: - [info] Reading server configuration from /etc/mha/app1/app1.cnf..
Thu Mar :: - [info] Starting SSH connection tests..
Thu Mar :: - [debug]
Thu Mar :: - [debug] Connecting via SSH from root@192.168.206.140(192.168.206.140:) to root@192.168.206.139(192.168.206.139:)..
Thu Mar :: - [debug] ok.
Thu Mar :: - [debug] Connecting via SSH from root@192.168.206.140(192.168.206.140:) to root@192.168.206.141(192.168.206.141:)..
Thu Mar :: - [debug] ok.
Thu Mar :: - [debug]
Thu Mar :: - [debug] Connecting via SSH from root@192.168.206.139(192.168.206.139:) to root@192.168.206.140(192.168.206.140:)..
Thu Mar :: - [debug] ok.
Thu Mar :: - [debug] Connecting via SSH from root@192.168.206.139(192.168.206.139:) to root@192.168.206.141(192.168.206.141:)..
Thu Mar :: - [debug] ok.
Thu Mar :: - [debug]
Thu Mar :: - [debug] Connecting via SSH from root@192.168.206.141(192.168.206.141:) to root@192.168.206.139(192.168.206.139:)..
Thu Mar :: - [debug] ok.
Thu Mar :: - [debug] Connecting via SSH from root@192.168.206.141(192.168.206.141:) to root@192.168.206.140(192.168.206.140:)..
Thu Mar :: - [debug] ok.
Thu Mar :: - [info] All SSH connection tests passed successfully.
验证主从复制
[root@localhost mha4mysql-manager-0.56]# masterha_check_repl --conf=/etc/mha/app1/app1.cnf
[root@localhost mha4mysql-manager-0.56]# masterha_check_repl --conf=/etc/mha/app1/app1.cnf
Thu Mar :: - [info] Reading default configuration from /etc/masterha_default.cnf..
Thu Mar :: - [info] Reading application default configuration from /etc/mha/app1/app1.cnf..
Thu Mar :: - [info] Reading server configuration from /etc/mha/app1/app1.cnf..
Thu Mar :: - [info] MHA::MasterMonitor version 0.56.
Thu Mar :: - [info] GTID failover mode =
Thu Mar :: - [info] Dead Servers:
Thu Mar :: - [info] Alive Servers:
Thu Mar :: - [info] 192.168.206.139(192.168.206.139:)
Thu Mar :: - [info] 192.168.206.140(192.168.206.140:)
Thu Mar :: - [info] 192.168.206.141(192.168.206.141:)
Thu Mar :: - [info] Alive Slaves:
Thu Mar :: - [info] 192.168.206.140(192.168.206.140:) Version=5.7.-log (oldest major version between slaves) log-bin:enabled
Thu Mar :: - [info] GTID ON
Thu Mar :: - [info] Replicating from 192.168.206.139(192.168.206.139:)
Thu Mar :: - [info] Primary candidate for the new Master (candidate_master is set)
Thu Mar :: - [info] 192.168.206.141(192.168.206.141:) Version=5.7.-log (oldest major version between slaves) log-bin:enabled
Thu Mar :: - [info] GTID ON
Thu Mar :: - [info] Replicating from 192.168.206.139(192.168.206.139:)
Thu Mar :: - [info] Not candidate for the new Master (no_master is set)
Thu Mar :: - [info] Current Alive Master: 192.168.206.139(192.168.206.139:)
Thu Mar :: - [info] Checking slave configurations..
Thu Mar :: - [info] read_only= is not set on slave 192.168.206.140(192.168.206.140:).
Thu Mar :: - [info] read_only= is not set on slave 192.168.206.141(192.168.206.141:).
Thu Mar :: - [info] Checking replication filtering settings..
Thu Mar :: - [info] binlog_do_db= , binlog_ignore_db=
Thu Mar :: - [info] Replication filtering check ok.
Thu Mar :: - [info] GTID (with auto-pos) is supported. Skipping all SSH and Node package checking.
Thu Mar :: - [info] Checking SSH publickey authentication settings on the current master..
Thu Mar :: - [warning] HealthCheck: Got timeout on checking SSH connection to 192.168.206.139! at /usr/share/perl5/vendor_perl/MHA/HealthCheck.pm line .
Thu Mar :: - [info]
192.168.206.139(192.168.206.139:) (current master)
+--192.168.206.140(192.168.206.140:)
+--192.168.206.141(192.168.206.141:) Thu Mar :: - [info] Checking replication health on 192.168.206.140..
Thu Mar :: - [info] ok.
Thu Mar :: - [info] Checking replication health on 192.168.206.141..
Thu Mar :: - [info] ok.
Thu Mar :: - [info] Checking master_ip_failover_script status:
Thu Mar :: - [info] /etc/mha/scripts/master_ip_failover --virtual_ip=192.168.206.145 --orig_master_vip_eth=eth0: --new_master_vip_eth=eth0: --command=status --ssh_user=root --orig_master_host=192.168.206.139 --orig_master_ip=192.168.206.139 --orig_master_port=
Thu Mar :: - [info] OK.
Thu Mar :: - [warning] shutdown_script is not defined.
Thu Mar :: - [info] Got exit code (Not master dead). MySQL Replication Health is OK.
四、启动MHA、测试MHA故障转移
启动MHA
[root@localhost mha4mysql-manager-0.56]# masterha_manager --conf=/etc/mha/app1/app1.cnf
[root@localhost mha4mysql-manager-0.56]# masterha_manager --conf=/etc/mha/app1/app1.cnf
Thu Mar :: - [info] Reading default configuration from /etc/masterha_default.cnf..
Thu Mar :: - [info] Reading application default configuration from /etc/mha/app1/app1.cnf..
Thu Mar :: - [info] Reading server configuration from /etc/mha/app1/app1.cnf..
或者采用后台启动
[root@localhost mha4mysql-manager-0.56]# nohup masterha_manager --conf=/etc/mha/app1/app1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/mha/app1/manager.log >& &
[root@localhost mha4mysql-manager-0.56]# nohup masterha_manager --conf=/etc/mha/app1/app1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/mha/app1/manager.log >& &
[]
[] Exit nohup masterha_manager --conf=/etc/mha/app1/app1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/mha/app1/manager.log >&
配置master的VIP
[root@localhost mysql]# /sbin/ifconfig eth0: 192.168.206.145 netmask 255.255.255.0 up
确认VIP
[root@localhost mysql]# ifconfig
[root@localhost mysql]# ifconfig
eth0 Link encap:Ethernet HWaddr :0C:::A4:6E
inet addr:192.168.206.139 Bcast:192.168.206.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe81:a46e/ Scope:Link
UP BROADCAST RUNNING MULTICAST MTU: Metric:
RX packets: errors: dropped: overruns: frame:
TX packets: errors: dropped: overruns: carrier:
collisions: txqueuelen:
RX bytes: (1.5 MiB) TX bytes: (1018.8 KiB) eth0: Link encap:Ethernet HWaddr :0C:::A4:6E
inet addr:192.168.206.145 Bcast:192.168.206.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU: Metric: lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::/ Scope:Host
UP LOOPBACK RUNNING MTU: Metric:
RX packets: errors: dropped: overruns: frame:
TX packets: errors: dropped: overruns: carrier:
collisions: txqueuelen:
RX bytes: (240.0 b) TX bytes: (240.0 b)
确认MHA启动状态：
[root@localhost ~]# masterha_check_status --conf=/etc/mha/app1/app1.cnf
[root@localhost ~]# masterha_check_status --conf=/etc/mha/app1/app1.cnf
app1 (pid:) is running(:PING_OK), master:192.168.206.139
进程确认：
[root@localhost scripts]# ps aux|grep mha
[root@localhost scripts]# ps aux|grep mha
root 1.8 0.9 pts/ S : : perl /usr/bin/masterha_manager --conf=/etc/mha/app1/app1.cnf --remove_dead_master_conf --ignore_last_failover
root 0.0 0.0 pts/ S+ : : grep mha
关闭MHA Manager监控：
[root@localhost mha4mysql-manager-0.56]# masterha_stop --conf=/etc/mha/app1/app1.cnf
[root@localhost mha4mysql-manager-0.56]# masterha_stop --conf=/etc/mha/app1/app1.cnf
Stopped app1 successfully.
[]+ Exit nohup masterha_manager --conf=/etc/mha/app1/app1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/mha/app1/manager.log >&
主动切换之交互模式
[root@localhost scripts]# masterha_master_switch --master_state=alive --conf=/etc/mha/app1/app1.cnf --new_master_host=192.168.206.140
[root@localhost scripts]# masterha_master_switch --master_state=alive --conf=/etc/mha/app1/app1.cnf --new_master_host=192.168.206.140
Thu Mar :: - [info] MHA::MasterRotate version 0.56.
Thu Mar :: - [info] Starting online master switch..
Thu Mar :: - [info]
Thu Mar :: - [info] * Phase : Configuration Check Phase..
Thu Mar :: - [info]
Thu Mar :: - [info] Reading default configuration from /etc/masterha_default.cnf..
Thu Mar :: - [info] Reading application default configuration from /etc/mha/app1/app1.cnf..
Thu Mar :: - [info] Reading server configuration from /etc/mha/app1/app1.cnf..
Thu Mar :: - [info] GTID failover mode =
Thu Mar :: - [info] Current Alive Master: 192.168.206.139(192.168.206.139:)
Thu Mar :: - [info] Alive Slaves:
Thu Mar :: - [info] 192.168.206.140(192.168.206.140:) Version=5.7.-log (oldest major version between slaves) log-bin:enabled
Thu Mar :: - [info] GTID ON
Thu Mar :: - [info] Replicating from 192.168.206.139(192.168.206.139:)
Thu Mar :: - [info] Primary candidate for the new Master (candidate_master is set)
Thu Mar :: - [info] 192.168.206.141(192.168.206.141:) Version=5.7.-log (oldest major version between slaves) log-bin:enabled
Thu Mar :: - [info] GTID ON
Thu Mar :: - [info] Replicating from 192.168.206.139(192.168.206.139:)
Thu Mar :: - [info] Not candidate for the new Master (no_master is set) It is better to execute FLUSH NO_WRITE_TO_BINLOG TABLES on the master before switching. Is it ok to execute on 192.168.206.139(192.168.206.139:)? (YES/no): yes
Thu Mar :: - [info] Executing FLUSH NO_WRITE_TO_BINLOG TABLES. This may take long time..
Thu Mar :: - [info] ok.
Thu Mar :: - [info] Checking MHA is not monitoring or doing failover..
Thu Mar :: - [info] Checking replication health on 192.168.206.140..
Thu Mar :: - [info] ok.
Thu Mar :: - [info] Checking replication health on 192.168.206.141..
Thu Mar :: - [info] ok.
Thu Mar :: - [info] 192.168.206.140 can be new master.
Thu Mar :: - [info]
From:
192.168.206.139(192.168.206.139:) (current master)
+--192.168.206.140(192.168.206.140:)
+--192.168.206.141(192.168.206.141:) To:
192.168.206.140(192.168.206.140:) (new master)
+--192.168.206.141(192.168.206.141:) Starting master switch from 192.168.206.139(192.168.206.139:) to 192.168.206.140(192.168.206.140:)? (yes/NO): yes
Thu Mar :: - [info] Checking whether 192.168.206.140(192.168.206.140:) is ok for the new master..
Thu Mar :: - [info] ok.
Thu Mar :: - [info] ** Phase : Configuration Check Phase completed.
Thu Mar :: - [info]
Thu Mar :: - [info] * Phase : Rejecting updates Phase..
Thu Mar :: - [info]
Thu Mar :: - [info] Executing master ip online change script to disable write on the current master:
Thu Mar :: - [info] /etc/mha/scripts/master_ip_online_change --virtual_ip=192.168.206.145 --orig_master_vip_eth=eth0: --new_master_vip_eth=eth0: --command=stop --orig_master_host=192.168.206.139 --orig_master_ip=192.168.206.139 --orig_master_port= --orig_master_user='root' --orig_master_password='replpass' --new_master_host=192.168.206.140 --new_master_ip=192.168.206.140 --new_master_port= --new_master_user='root' --new_master_password='replpass' --orig_master_ssh_user=root --new_master_ssh_user=root
Thu Mar :: Set read_only on the new master.. ok.
Thu Mar :: Drpping app user on the orig master..
Thu Mar :: Waiting all running threads are disconnected.. (max milliseconds)
{'Time' => '','Command' => 'Binlog Dump GTID','db' => undef,'Id' => '','Info' => undef,'User' => 'u_repl','State' => 'Master has sent all binlog to slave; waiting for more updates','Host' => '192.168.206.141:59184'}
Thu Mar :: Waiting all running threads are disconnected.. (max milliseconds)
{'Time' => '','Command' => 'Binlog Dump GTID','db' => undef,'Id' => '','Info' => undef,'User' => 'u_repl','State' => 'Master has sent all binlog to slave; waiting for more updates','Host' => '192.168.206.141:59184'}
Thu Mar :: Waiting all running threads are disconnected.. (max milliseconds)
{'Time' => '','Command' => 'Binlog Dump GTID','db' => undef,'Id' => '','Info' => undef,'User' => 'u_repl','State' => 'Master has sent all binlog to slave; waiting for more updates','Host' => '192.168.206.141:59184'}
Thu Mar :: Set read_only= on the orig master.. ok.
Thu Mar :: Waiting all running queries are disconnected.. (max milliseconds)
{'Time' => '','Command' => 'Binlog Dump GTID','db' => undef,'Id' => '','Info' => undef,'User' => 'u_repl','State' => 'Master has sent all binlog to slave; waiting for more updates','Host' => '192.168.206.141:59184'}
Thu Mar :: Killing all application threads..
Thu Mar :: done.
Thu Mar :: - [info] ok.
Thu Mar :: - [info] Locking all tables on the orig master to reject updates from everybody (including root):
Thu Mar :: - [info] Executing FLUSH TABLES WITH READ LOCK..
Thu Mar :: - [info] ok.
Thu Mar :: - [info] Orig master binlog:pos is mysql-bin.:.
Thu Mar :: - [info] Waiting to execute all relay logs on 192.168.206.140(192.168.206.140:)..
Thu Mar :: - [info] master_pos_wait(mysql-bin.:) completed on 192.168.206.140(192.168.206.140:). Executed events.
Thu Mar :: - [info] done.
Thu Mar :: - [info] Getting new master's binlog name and position..
Thu Mar :: - [info] mysql-bin.:
Thu Mar :: - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='192.168.206.140', MASTER_PORT=, MASTER_AUTO_POSITION=, MASTER_USER='u_repl', MASTER_PASSWORD='xxx';
Thu Mar :: - [info] Executing master ip online change script to allow write on the new master:
Thu Mar :: - [info] /etc/mha/scripts/master_ip_online_change --virtual_ip=192.168.206.145 --orig_master_vip_eth=eth0: --new_master_vip_eth=eth0: --command=start --orig_master_host=192.168.206.139 --orig_master_ip=192.168.206.139 --orig_master_port= --orig_master_user='root' --orig_master_password='replpass' --new_master_host=192.168.206.140 --new_master_ip=192.168.206.140 --new_master_port= --new_master_user='root' --new_master_password='replpass' --orig_master_ssh_user=root --new_master_ssh_user=root
Thu Mar :: Set read_only= on the new master.
Thu Mar :: Creating app user on the new master..
Thu Mar :: - [info] ok.
Thu Mar :: - [info]
Thu Mar :: - [info] * Switching slaves in parallel..
Thu Mar :: - [info]
Thu Mar :: - [info] -- Slave switch on host 192.168.206.141(192.168.206.141:) started, pid:
Thu Mar :: - [info]
Thu Mar :: - [info] Log messages from 192.168.206.141 ...
Thu Mar :: - [info]
Thu Mar :: - [info] Waiting to execute all relay logs on 192.168.206.141(192.168.206.141:)..
Thu Mar :: - [info] master_pos_wait(mysql-bin.:) completed on 192.168.206.141(192.168.206.141:). Executed events.
Thu Mar :: - [info] done.
Thu Mar :: - [info] Resetting slave 192.168.206.141(192.168.206.141:) and starting replication from the new master 192.168.206.140(192.168.206.140:)..
Thu Mar :: - [info] Executed CHANGE MASTER.
Thu Mar :: - [info] Slave started.
Thu Mar :: - [info] End of log messages from 192.168.206.141 ...
Thu Mar :: - [info]
Thu Mar :: - [info] -- Slave switch on host 192.168.206.141(192.168.206.141:) succeeded.
Thu Mar :: - [info] Unlocking all tables on the orig master:
Thu Mar :: - [info] Executing UNLOCK TABLES..
Thu Mar :: - [info] ok.
Thu Mar :: - [info] All new slave servers switched successfully.
Thu Mar :: - [info]
Thu Mar :: - [info] * Phase : New master cleanup phase..
Thu Mar :: - [info]
Thu Mar :: - [info] 192.168.206.140: Resetting slave info succeeded.
Thu Mar :: - [info] Switching master to 192.168.206.140(192.168.206.140:) completed successfully.
重新将旧的master加入复制，指向新的master
mysql> change master to master_host='192.168.206.140',master_user='u_repl',master_password='replpass',master_auto_position=;
mysql> start slave;
主动切换之非交互模式：
[root@localhost scripts]# masterha_master_switch --master_state=alive --conf=/etc/mha/app1/app1.cnf --new_master_host=192.168.206.139 --interactive=
[root@localhost scripts]# masterha_master_switch --master_state=alive --conf=/etc/mha/app1/app1.cnf --new_master_host=192.168.206.139 --interactive=
Thu Mar :: - [info] MHA::MasterRotate version 0.56.
Thu Mar :: - [info] Starting online master switch..
Thu Mar :: - [info]
Thu Mar :: - [info] * Phase : Configuration Check Phase..
Thu Mar :: - [info]
Thu Mar :: - [info] Reading default configuration from /etc/masterha_default.cnf..
Thu Mar :: - [info] Reading application default configuration from /etc/mha/app1/app1.cnf..
Thu Mar :: - [info] Reading server configuration from /etc/mha/app1/app1.cnf..
Thu Mar :: - [info] GTID failover mode =
Thu Mar :: - [info] Current Alive Master: 192.168.206.140(192.168.206.140:)
Thu Mar :: - [info] Alive Slaves:
Thu Mar :: - [info] 192.168.206.139(192.168.206.139:) Version=5.7.-log (oldest major version between slaves) log-bin:enabled
Thu Mar :: - [info] GTID ON
Thu Mar :: - [info] Replicating from 192.168.206.140(192.168.206.140:)
Thu Mar :: - [info] Primary candidate for the new Master (candidate_master is set)
Thu Mar :: - [info] 192.168.206.141(192.168.206.141:) Version=5.7.-log (oldest major version between slaves) log-bin:enabled
Thu Mar :: - [info] GTID ON
Thu Mar :: - [info] Replicating from 192.168.206.140(192.168.206.140:)
Thu Mar :: - [info] Not candidate for the new Master (no_master is set)
Thu Mar :: - [info] Executing FLUSH NO_WRITE_TO_BINLOG TABLES. This may take long time..
Thu Mar :: - [info] ok.
Thu Mar :: - [info] Checking MHA is not monitoring or doing failover..
Thu Mar :: - [info] Checking replication health on 192.168.206.139..
Thu Mar :: - [info] ok.
Thu Mar :: - [info] Checking replication health on 192.168.206.141..
Thu Mar :: - [info] ok.
Thu Mar :: - [info] 192.168.206.139 can be new master.
Thu Mar :: - [info]
From:
192.168.206.140(192.168.206.140:) (current master)
+--192.168.206.139(192.168.206.139:)
+--192.168.206.141(192.168.206.141:) To:
192.168.206.139(192.168.206.139:) (new master)
+--192.168.206.141(192.168.206.141:)
Thu Mar :: - [info] Checking whether 192.168.206.139(192.168.206.139:) is ok for the new master..
Thu Mar :: - [info] ok.
Thu Mar :: - [info] ** Phase : Configuration Check Phase completed.
Thu Mar :: - [info]
Thu Mar :: - [info] * Phase : Rejecting updates Phase..
Thu Mar :: - [info]
Thu Mar :: - [info] Executing master ip online change script to disable write on the current master:
Thu Mar :: - [info] /etc/mha/scripts/master_ip_online_change --virtual_ip=192.168.206.145 --orig_master_vip_eth=eth0: --new_master_vip_eth=eth0: --command=stop --orig_master_host=192.168.206.140 --orig_master_ip=192.168.206.140 --orig_master_port= --orig_master_user='root' --orig_master_password='replpass' --new_master_host=192.168.206.139 --new_master_ip=192.168.206.139 --new_master_port= --new_master_user='root' --new_master_password='replpass' --orig_master_ssh_user=root --new_master_ssh_user=root
Thu Mar :: Set read_only on the new master.. ok.
Thu Mar :: Drpping app user on the orig master..
Thu Mar :: Waiting all running threads are disconnected.. (max milliseconds)
{'Time' => '','Command' => 'Binlog Dump GTID','db' => undef,'Id' => '','Info' => undef,'User' => 'u_repl','State' => 'Master has sent all binlog to slave; waiting for more updates','Host' => '192.168.206.141:48684'}
Thu Mar :: Waiting all running threads are disconnected.. (max milliseconds)
{'Time' => '','Command' => 'Binlog Dump GTID','db' => undef,'Id' => '','Info' => undef,'User' => 'u_repl','State' => 'Master has sent all binlog to slave; waiting for more updates','Host' => '192.168.206.141:48684'}
Thu Mar :: Waiting all running threads are disconnected.. (max milliseconds)
{'Time' => '','Command' => 'Binlog Dump GTID','db' => undef,'Id' => '','Info' => undef,'User' => 'u_repl','State' => 'Master has sent all binlog to slave; waiting for more updates','Host' => '192.168.206.141:48684'}
Thu Mar :: Set read_only= on the orig master.. ok.
Thu Mar :: Waiting all running queries are disconnected.. (max milliseconds)
{'Time' => '','Command' => 'Binlog Dump GTID','db' => undef,'Id' => '','Info' => undef,'User' => 'u_repl','State' => 'Master has sent all binlog to slave; waiting for more updates','Host' => '192.168.206.141:48684'}
Thu Mar :: Killing all application threads..
Thu Mar :: done.
Thu Mar :: - [info] ok.
Thu Mar :: - [info] Locking all tables on the orig master to reject updates from everybody (including root):
Thu Mar :: - [info] Executing FLUSH TABLES WITH READ LOCK..
Thu Mar :: - [info] ok.
Thu Mar :: - [info] Orig master binlog:pos is mysql-bin.:.
Thu Mar :: - [info] Waiting to execute all relay logs on 192.168.206.139(192.168.206.139:)..
Thu Mar :: - [info] master_pos_wait(mysql-bin.:) completed on 192.168.206.139(192.168.206.139:). Executed events.
Thu Mar :: - [info] done.
Thu Mar :: - [info] Getting new master's binlog name and position..
Thu Mar :: - [info] mysql-bin.:
Thu Mar :: - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='192.168.206.139', MASTER_PORT=, MASTER_AUTO_POSITION=, MASTER_USER='u_repl', MASTER_PASSWORD='xxx';
Thu Mar :: - [info] Executing master ip online change script to allow write on the new master:
Thu Mar :: - [info] /etc/mha/scripts/master_ip_online_change --virtual_ip=192.168.206.145 --orig_master_vip_eth=eth0: --new_master_vip_eth=eth0: --command=start --orig_master_host=192.168.206.140 --orig_master_ip=192.168.206.140 --orig_master_port= --orig_master_user='root' --orig_master_password='replpass' --new_master_host=192.168.206.139 --new_master_ip=192.168.206.139 --new_master_port= --new_master_user='root' --new_master_password='replpass' --orig_master_ssh_user=root --new_master_ssh_user=root
Thu Mar :: Set read_only= on the new master.
Thu Mar :: Creating app user on the new master..
Thu Mar :: - [info] ok.
Thu Mar :: - [info]
Thu Mar :: - [info] * Switching slaves in parallel..
Thu Mar :: - [info]
Thu Mar :: - [info] -- Slave switch on host 192.168.206.141(192.168.206.141:) started, pid:
Thu Mar :: - [info]
Thu Mar :: - [info] Log messages from 192.168.206.141 ...
Thu Mar :: - [info]
Thu Mar :: - [info] Waiting to execute all relay logs on 192.168.206.141(192.168.206.141:)..
Thu Mar :: - [info] master_pos_wait(mysql-bin.:) completed on 192.168.206.141(192.168.206.141:). Executed events.
Thu Mar :: - [info] done.
Thu Mar :: - [info] Resetting slave 192.168.206.141(192.168.206.141:) and starting replication from the new master 192.168.206.139(192.168.206.139:)..
Thu Mar :: - [info] Executed CHANGE MASTER.
Thu Mar :: - [info] Slave started.
Thu Mar :: - [info] End of log messages from 192.168.206.141 ...
Thu Mar :: - [info]
Thu Mar :: - [info] -- Slave switch on host 192.168.206.141(192.168.206.141:) succeeded.
Thu Mar :: - [info] Unlocking all tables on the orig master:
Thu Mar :: - [info] Executing UNLOCK TABLES..
Thu Mar :: - [info] ok.
Thu Mar :: - [info] All new slave servers switched successfully.
Thu Mar :: - [info]
Thu Mar :: - [info] * Phase : New master cleanup phase..
Thu Mar :: - [info]
Thu Mar :: - [info] 192.168.206.139: Resetting slave info succeeded.
Thu Mar :: - [info] Switching master to 192.168.206.139(192.168.206.139:) completed successfully.
Master192.168.206.140宕机模式下切换主从：
masterha_master_switch --master_state=dead --conf=/etc/mha/app1/app1.cnf --dead_master_host=192.168.206.140 --dead_master_ip=192.168.206.140 --dead_master_port= --new_master_host=192.168.206.139
[root@localhost scripts]# masterha_master_switch --master_state=dead --conf=/etc/mha/app1/app1.cnf --dead_master_host=192.168.206.140 --dead_master_ip=192.168.206.140 --dead_master_port= --new_master_host=192.168.206.139
Thu Mar :: - [info] Reading default configuration from /etc/masterha_default.cnf..
Thu Mar :: - [info] Reading application default configuration from /etc/mha/app1/app1.cnf..
Thu Mar :: - [info] Reading server configuration from /etc/mha/app1/app1.cnf..
Thu Mar :: - [info] MHA::MasterFailover version 0.56.
Thu Mar :: - [info] Starting master failover.
Thu Mar :: - [info]
Thu Mar :: - [info] * Phase : Configuration Check Phase..
Thu Mar :: - [info]
Thu Mar :: - [info] GTID failover mode =
Thu Mar :: - [info] Dead Servers:
Thu Mar :: - [info] 192.168.206.140(192.168.206.140:)
Thu Mar :: - [info] Checking master reachability via MySQL(double check)...
Thu Mar :: - [info] ok.
Thu Mar :: - [info] Alive Servers:
Thu Mar :: - [info] 192.168.206.139(192.168.206.139:)
Thu Mar :: - [info] 192.168.206.141(192.168.206.141:)
Thu Mar :: - [info] Alive Slaves:
Thu Mar :: - [info] 192.168.206.139(192.168.206.139:) Version=5.7.-log (oldest major version between slaves) log-bin:enabled
Thu Mar :: - [info] GTID ON
Thu Mar :: - [info] Replicating from 192.168.206.140(192.168.206.140:)
Thu Mar :: - [info] Primary candidate for the new Master (candidate_master is set)
Thu Mar :: - [info] 192.168.206.141(192.168.206.141:) Version=5.7.-log (oldest major version between slaves) log-bin:enabled
Thu Mar :: - [info] GTID ON
Thu Mar :: - [info] Replicating from 192.168.206.140(192.168.206.140:)
Thu Mar :: - [info] Not candidate for the new Master (no_master is set)
Master 192.168.206.140(192.168.206.140:) is dead. Proceed? (yes/NO): yes
Thu Mar :: - [info] Starting GTID based failover.
Thu Mar :: - [info]
Thu Mar :: - [info] ** Phase : Configuration Check Phase completed.
Thu Mar :: - [info]
Thu Mar :: - [info] * Phase : Dead Master Shutdown Phase..
Thu Mar :: - [info]
Thu Mar :: - [info] HealthCheck: SSH to 192.168.206.140 is reachable.
Thu Mar :: - [info] Forcing shutdown so that applications never connect to the current master..
Thu Mar :: - [info] Executing master IP deactivation script:
Thu Mar :: - [info] /etc/mha/scripts/master_ip_failover --virtual_ip=192.168.206.145 --orig_master_vip_eth=eth0: --new_master_vip_eth=eth0: --orig_master_host=192.168.206.140 --orig_master_ip=192.168.206.140 --orig_master_port= --command=stopssh --ssh_user=root
Thu Mar :: - [info] done.
Thu Mar :: - [warning] shutdown_script is not set. Skipping explicit shutting down of the dead master.
Thu Mar :: - [info] * Phase : Dead Master Shutdown Phase completed.
Thu Mar :: - [info]
Thu Mar :: - [info] * Phase : Master Recovery Phase..
Thu Mar :: - [info]
Thu Mar :: - [info] * Phase 3.1: Getting Latest Slaves Phase..
Thu Mar :: - [info]
Thu Mar :: - [info] The latest binary log file/position on all slaves is mysql-bin.:
Thu Mar :: - [info] Latest slaves (Slaves that received relay log files to the latest):
Thu Mar :: - [info] 192.168.206.139(192.168.206.139:) Version=5.7.-log (oldest major version between slaves) log-bin:enabled
Thu Mar :: - [info] GTID ON
Thu Mar :: - [info] Replicating from 192.168.206.140(192.168.206.140:)
Thu Mar :: - [info] Primary candidate for the new Master (candidate_master is set)
Thu Mar :: - [info] 192.168.206.141(192.168.206.141:) Version=5.7.-log (oldest major version between slaves) log-bin:enabled
Thu Mar :: - [info] GTID ON
Thu Mar :: - [info] Replicating from 192.168.206.140(192.168.206.140:)
Thu Mar :: - [info] Not candidate for the new Master (no_master is set)
Thu Mar :: - [info] The oldest binary log file/position on all slaves is mysql-bin.:
Thu Mar :: - [info] Oldest slaves:
Thu Mar :: - [info] 192.168.206.139(192.168.206.139:) Version=5.7.-log (oldest major version between slaves) log-bin:enabled
Thu Mar :: - [info] GTID ON
Thu Mar :: - [info] Replicating from 192.168.206.140(192.168.206.140:)
Thu Mar :: - [info] Primary candidate for the new Master (candidate_master is set)
Thu Mar :: - [info] 192.168.206.141(192.168.206.141:) Version=5.7.-log (oldest major version between slaves) log-bin:enabled
Thu Mar :: - [info] GTID ON
Thu Mar :: - [info] Replicating from 192.168.206.140(192.168.206.140:)
Thu Mar :: - [info] Not candidate for the new Master (no_master is set)
Thu Mar :: - [info]
Thu Mar :: - [info] * Phase 3.3: Determining New Master Phase..
Thu Mar :: - [info]
Thu Mar :: - [info] 192.168.206.139 can be new master.
Thu Mar :: - [info] New master is 192.168.206.139(192.168.206.139:)
Thu Mar :: - [info] Starting master failover..
Thu Mar :: - [info]
From:
192.168.206.140(192.168.206.140:) (current master)
+--192.168.206.139(192.168.206.139:)
+--192.168.206.141(192.168.206.141:) To:
192.168.206.139(192.168.206.139:) (new master)
+--192.168.206.141(192.168.206.141:) Starting master switch from 192.168.206.140(192.168.206.140:) to 192.168.206.139(192.168.206.139:)? (yes/NO): yes
Thu Mar :: - [info] New master decided manually is 192.168.206.139(192.168.206.139:)
Thu Mar :: - [info]
Thu Mar :: - [info] * Phase 3.3: New Master Recovery Phase..
Thu Mar :: - [info]
Thu Mar :: - [info] Waiting all logs to be applied..
Thu Mar :: - [info] done.
Thu Mar :: - [info] Getting new master's binlog name and position..
Thu Mar :: - [info] mysql-bin.:
Thu Mar :: - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='192.168.206.139', MASTER_PORT=, MASTER_AUTO_POSITION=, MASTER_USER='u_repl', MASTER_PASSWORD='xxx';
Thu Mar :: - [info] Master Recovery succeeded. File:Pos:Exec_Gtid_Set: mysql-bin., , 347cbac6--11e7-b957-000c2981a46e:-,
9e2c7c0f--11e7--000c29ab7544:-
Thu Mar :: - [info] Executing master IP activate script:
Thu Mar :: - [info] /etc/mha/scripts/master_ip_failover --virtual_ip=192.168.206.145 --orig_master_vip_eth=eth0: --new_master_vip_eth=eth0: --command=start --ssh_user=root --orig_master_host=192.168.206.140 --orig_master_ip=192.168.206.140 --orig_master_port= --new_master_host=192.168.206.139 --new_master_ip=192.168.206.139 --new_master_port= --new_master_user='root' --new_master_password='replpass'
Set read_only= on the new master.
Creating app user on the new master..
Undefined subroutine &main::FIXME_xxx_create_user called at /etc/mha/scripts/master_ip_failover line .
Thu Mar :: - [error][/usr/share/perl5/vendor_perl/MHA/MasterFailover.pm, ln1588] Failed to activate master IP address for 192.168.206.139(192.168.206.139:) with return code :
Thu Mar :: - [warning] Proceeding.
Thu Mar :: - [info] ** Finished master recovery successfully.
Thu Mar :: - [info] * Phase : Master Recovery Phase completed.
Thu Mar :: - [info]
Thu Mar :: - [info] * Phase : Slaves Recovery Phase..
Thu Mar :: - [info]
Thu Mar :: - [info]
Thu Mar :: - [info] * Phase 4.1: Starting Slaves in parallel..
Thu Mar :: - [info]
Thu Mar :: - [info] -- Slave recovery on host 192.168.206.141(192.168.206.141:) started, pid: . Check tmp log /var/log/mha/app1/192.168..141_3306_20170316080936.log if it takes time..
Thu Mar :: - [info]
Thu Mar :: - [info] Log messages from 192.168.206.141 ...
Thu Mar :: - [info]
Thu Mar :: - [info] Resetting slave 192.168.206.141(192.168.206.141:) and starting replication from the new master 192.168.206.139(192.168.206.139:)..
Thu Mar :: - [info] Executed CHANGE MASTER.
Thu Mar :: - [info] Slave started.
Thu Mar :: - [info] gtid_wait(347cbac6--11e7-b957-000c2981a46e:-,
9e2c7c0f--11e7--000c29ab7544:-) completed on 192.168.206.141(192.168.206.141:). Executed events.
Thu Mar :: - [info] End of log messages from 192.168.206.141.
Thu Mar :: - [info] -- Slave on host 192.168.206.141(192.168.206.141:) started.
Thu Mar :: - [info] All new slave servers recovered successfully.
Thu Mar :: - [info]
Thu Mar :: - [info] * Phase : New master cleanup phase..
Thu Mar :: - [info]
Thu Mar :: - [info] Resetting slave info on the new master..
Thu Mar :: - [info] 192.168.206.139: Resetting slave info succeeded.
Thu Mar :: - [info] Master failover to 192.168.206.139(192.168.206.139:) completed successfully.
Thu Mar :: - [info] ----- Failover Report ----- app1: MySQL Master failover 192.168.206.140(192.168.206.140:) to 192.168.206.139(192.168.206.139:) succeeded Master 192.168.206.140(192.168.206.140:) is down! Check MHA Manager logs at localhost.localdomain for details. Started manual(interactive) failover.
Invalidated master IP address on 192.168.206.140(192.168.206.140:)
Selected 192.168.206.139(192.168.206.139:) as a new master.
192.168.206.139(192.168.206.139:): OK: Applying all logs succeeded.
Failed to activate master IP address for 192.168.206.139(192.168.206.139:) with return code :
192.168.206.141(192.168.206.141:): OK: Slave started, replicating from 192.168.206.139(192.168.206.139:)
192.168.206.139(192.168.206.139:): Resetting slave info succeeded.
Master failover to 192.168.206.139(192.168.206.139:) completed successfully.
本文地址：http://www.cnblogs.com/ajiangg/p/6552855.html
参考资源：https://code.google.com/p/mysql-master-ha/wiki/TableOfContents?tm=6
http://k-1blog.com/development/program/post-7414/
http://www.cnblogs.com/galengao/p/5764163.html
CentOS6.8下MySQL MHA架构搭建笔记的更多相关文章
- CENTOS6.6 下mysql MHA架构搭建
本文来自我的github pages博客http://galengao.github.io/ 即www.gaohuirong.cn 摘要: 本篇是自己搭建的一篇mysql MHA文章 前面的安装步骤基 ...
- CENTOS6.6下mysql MMM架构搭建
本文来自我的github pages博客http://galengao.github.io/ 即www.gaohuirong.cn MMM(Master-Master replication mana ...
- mysql MHA架构搭建过程
[环境介绍] 系统环境:Red Hat Enterprise Linux 7 + 5.7.18 + MHA version 0.57 系统 IP 主机名 备注 版本 xx系统 192.168.142. ...
- ubuntu下mysql的环境搭建及使用
ubuntu下mysql的环境搭建及使用 环境安装 使用如下命令分别安装服务端程序,客户端程序,及客户端依赖库 sudo apt-get install mysql-server sudo apt-g ...
- ubuntu 下 mysql数据库的搭建 及 数据迁移
1.mysql的安装 我是使用apt-get直接安装的 :sudo apt-get install mysql-server sudo apt-get install mysql-client 2.配 ...
- [转] Linux学习之CentOS(十三)--CentOS6.4下Mysql数据库的安装与配置
from: http://www.cnblogs.com/xiaoluo501395377/archive/2013/04/07/3003278.html 如果要在Linux上做j2ee开发,首先得 ...
- Linux学习之CentOS(十三)--CentOS6.4下Mysql数据库的安装与配置
原文:http://www.cnblogs.com/xiaoluo501395377/archive/2013/04/07/3003278.html 如果要在Linux上做j2ee开发,首先得搭建好j ...
- MySQL MHA环境搭建
MHA功能: 1,从故障的mysql保存二进制日志时间(binlog events);2,识别含有最新更新的slave:3,应用差异的中继日志(relay log)到其他的slave:4,应用从mas ...
- Linux学习之CentOS(十三)--CentOS6.4下Mysql数据库的安装与配置（转）
原文地址:http://www.cnblogs.com/xiaoluo501395377/archive/2013/04/07/3003278.html 如果要在Linux上做j2ee开发,首先得搭建 ...
随机推荐
- find命令高级参数
find 命令参数大全 转自[https://www.cnblogs.com/yorkyang/p/6294894.html] Linux中find常见用法示例 ·find path -op ...
- python练习一—文本转化渲染为html
想学习python已经很久了,以前使用ArcGIS的时候学习过一些简单的python语法,用来进行一些简单的GIS数据处理,但是后来并没有用到工作中也就荒废了,后来断断续续看过一些,最近想学习一门新的 ...
- python理解描述符(descriptor)
Descriptor基础 python中的描述符可以用来定义触发自动执行的代码,它像是一个对象属性操作(访问.赋值.删除)的代理类一样.前面介绍过的property是描述符的一种. 大致流程是这样的: ...
- Logback中使用TurboFilter实现日志级别等内容的动态修改
可能看到这个标题,读者会问:要修改日志的级别,不是直接修改log.xxx就好了吗?为何要搞那么复杂呢?所以,先说一下场景,为什么要通过TurboFilter去动态的修改日志级别.我们在使用Java开发 ...
- Python正则进阶
目录 1.Python正则表达式模块 1.1 正则表达式处理字符串主要有四大功能 1.2 Python中re模块使用正则表达式的两种方法 1.3 正则表达式对象的常用方法 1.4 匹配对象的属性与方法 ...
- centos7+openvpn+easy3.0
openvpn介绍 OpenVPN 是一个基于 OpenSSL库的应用层 VPN 实现.和传统 VPN 相比,它的优点是简单易用.vpn直译就是虚拟专用通道,是提供企业之间或者公司之间安全数据传输的 ...
- 多线程(二)ThreadLocal
ThreadLocal public class Demo extends Thread{ static int i = 0; public Integer getNext(){ i++; retur ...
- Docker 系列五（Docker Compose 项目）.
一.概念 Docker Compose 是官方编排项目之一,负责快速的部署分布式应用.它允许用户通过一个单独的 docker-compose.yml 模板文件(YAML格式)来定义一种相关联的应用容器 ...
- 函数多个返回值与unpack的用法
-- return the index of max number and himself -- 函数可以返回多个值 function get_max( T ) ] for i, v in ipair ...
- linux查看用户、创建用户、设置密码、修改用户、删除用户命令
查看用户 /etc/passwd /etc/shadow id alex ' |passwd --stdin alex # 设置密码,不需要交互 [root@localhost ~]# tail -l ...