3. ClustrixDB 操作

测试数据库

一、 测试分片，存储信息

测试前：

clxm@p2cn1uclx101m_10.248.100.241 /data]$ clx stat
Cluster Name: cle69e350c2c16d729
Cluster Version: 9.1.4
Cluster Status: OK
Cluster Size: 3 nodes - 8 CPUs per Node
Current Node: ip-10-1-3-151 - nid 3

nid | Hostname | Status | IP Address | TPS | Used | Total
----+----------------+--------+-------------+-----+-----------------+--------
1 | ip-10-1-3-88 | OK | 10.1.3.88 | 0 | 145.2M (0.19%) | 74.0G
2 | ip-10-1-3-242 | OK | 10.1.3.242 | 0 | 144.7M (0.19%) | 74.0G
3 | ip-10-1-3-151 | OK | 10.1.3.151 | 0 | 144.8M (0.19%) | 74.0G
----+----------------+--------+-------------+-----+-----------------+--------
0 | 434.7M (0.19%) | 222.0G

测试：(造10000条数据)

create database testdb;

use testdb;

CREATE TABLE userinfo_uuid
(
uuid varchar(36) NOT NULL,
name varchar(64) NOT NULL DEFAULT '',
email varchar(64) NOT NULL DEFAULT '',
password varchar(64) NOT NULL DEFAULT '',
dob date DEFAULT NULL,
address varchar(255) NOT NULL DEFAULT '',
city varchar(64) NOT NULL DEFAULT '',
state_id tinyint unsigned NOT NULL DEFAULT '0',
zip varchar(8) NOT NULL DEFAULT '',
country_id smallint unsigned NOT NULL DEFAULT '0',
gender enum('M','F') NOT NULL DEFAULT 'M',
account_type varchar(32) NOT NULL DEFAULT '',
verified tinyint NOT NULL DEFAULT '0',
allow_mall tinyint unsigned NOT NULL DEFAULT '0',
parrent_account int unsigned NOT NULL DEFAULT '0',
closest_airport varchar(3) NOT NULL DEFAULT '',
PRIMARY KEY(uuid),
UNIQUE KEY email (email),
KEY country_id (country_id),
KEY state_id (state_id),
KEY state_id_2 (state_id,city,address)
)ENGINE=InnoDB;

DROP FUNCTION IF EXISTS rand_string;
DELIMITER $$
CREATE FUNCTION rand_string(n INT)
RETURNS VARCHAR(255)
BEGIN
DECLARE chars_str varchar(100) DEFAULT 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789';
DECLARE return_str varchar(255) DEFAULT '';
DECLARE i INT DEFAULT 0;
WHILE i < n DO
SET return_str = concat(return_str,substring(chars_str , FLOOR(1 + RAND()*62 ),1));
SET i = i +1;
END WHILE;
RETURN return_str;
END $$
DELIMITER ;

DELIMITER $$

DROP PROCEDURE IF EXISTS `insert_userinfo_uuid`$$

CREATE DEFINER=`root`@`127.0.0.1` PROCEDURE `insert_userinfo_uuid`(IN item INTEGER)
BEGIN
DECLARE counter INT;
SET counter = item;
WHILE counter >= 1 DO
insert into userinfo_uuid (uuid,name,email,password,dob,address,city,state_id,zip,country_id,gender,account_type,verified,allow_mall,parrent_account,closest_airport)
values(uuid(),rand_string(64), rand_string(64), rand_string(64), '2010-10-10', rand_string(255), rand_string(64), ceil(rand() * 100), rand_string(8),
ceil(rand() * 100), 'M', rand_string(32), 0, 0, 0, rand_string(3));
SET counter = counter - 1;
END WHILE;
END$$

DELIMITER ;

MySQL [testdb]> call `insert_userinfo_uuid`(10000);
Query OK, 1 row affected (13 min 45.81 sec)

测试后：

clxm@p2cn1uclx101m_10.248.100.241 /opt/clustrix]$ clx stat
Cluster Name: cle69e350c2c16d729
Cluster Version: 9.1.4
Cluster Status: OK
Cluster Size: 3 nodes - 8 CPUs per Node
Current Node: ip-10-1-3-151 - nid 3

nid | Hostname | Status | IP Address | TPS | Used | Total
----+----------------+--------+-------------+-----+-----------------+--------
1 | ip-10-1-3-88 | OK | 10.1.3.88 | 0 | 161.9M (0.21%) | 74.0G
2 | ip-10-1-3-242 | OK | 10.1.3.242 | 0 | 161.5M (0.21%) | 74.0G
3 | ip-10-1-3-151 | OK | 10.1.3.151 | 0 | 161.3M (0.21%) | 74.0G
----+----------------+--------+-------------+-----+-----------------+--------
0 | 484.7M (0.21%) | 222.0G

二、在线DDL操作

一千万数据（模拟插数据，验证不会锁表）

MySQL [scloud]> alter table sbtest1 add d varchar(10);
Query OK, 0 rows affected (48.01 sec)

监视ALTER的进度

要查看ALTER进程的状态，请使用此SQL。

sql> select * from system.alter_progress;

主从同步会出现延迟

MySQL [(none)]> show slave status\G

*************************** 1. row ***************************

Slave_Name: default

Slave_Status: Running

Master_Host: 10.1.1.23

Master_Port: 5306

Master_User: clx_repl

Master_Log_File: clustrix-bin

Slave_Enabled: Enabled

Log_File_Seq: 20

Log_File_Pos: 21568627

Last_Error: no error

Connection_Status: Connected

Relay_Log_Bytes_Read: 0

Relay_Log_Current_Bytes: 43719291

  Seconds_Behind_Master: 95

1 row in set (0.00 sec)

ALTER完成后，以高并发性访问或具有N * 10 ^ 7行以上的表的群集性能可能会下降。ClustrixDB存储每个表的统计信息和查询计划，并缓存这些值。ClustrixDB对该信息的定期刷新和刷新可能不会足够快地发生，因此可能会影响性能。为避免这种情况，请 使用以下命令在ALTER完成后立即刷新高速缓存  ：

sql> ALTER TABLE ...

shell> clx cmd'mysql system -e“ call pdcache_update（）”'; 

shell> clx cmd'mysql system -e“ call qpc_flush（）”'; 

请注意，qpc_flush和pdcache_update是基于每个节点完成的。因此，应使用clx cmd实用程序执行这些操作，以确保它们在所有节点上运行。

建议将这些命令作为命令&&命令&&命令在外壳中链接在一起，以避免在ALTER完成和缓存刷新之间出现延迟。

四、在线添加从库

主库: 10.1.1.23:5306

从库: 10.1.3.88:5306

主库开启binlog

MySQL [(none)]> CREATE BINLOG 'clustrix-bin' FORMAT='ROW'

主库创建同步账号

MySQL [(none)]> create user 'clx_repl'@'%' identified by '123123';

MySQL [(none)]> Grant REPLICATION SLAVE ON *.* to 'clx_repl'@'%';

主库模拟创建数据

scloudusr@p2cn1uclx101m_10.248.100.241 ~]$ sysbench --mysql-host=localhost --mysql-port=5306 --mysql-user=root --mysql-password= --mysql-db=scloud --table_size=1000000 oltp_insert prepare

主库备份数据

MySQL [(none)]>  backup scloud.* to "sftp://scloudusr:Scloud201!@10.1.3.88:2022/tmp/backup" ;

从库恢复数据

RESTORE * FROM "sftp://scloudusr:Scloud201!@10.1.3.88:2022/tmp/backup" REPLICAS = 1;   （快速恢复使用1个副本）

从库备份文件找到主库file和pos信息

cat /tmp/backup/metadata/clustrix-bin.000002:103179247

从库配置同步信息

MySQL [(none)]> stop slave;
MySQL [(none)]> CHANGE MASTER TO MASTER_LOG_FILE = 'clustrix-bin.000002', MASTER_LOG_POS = 103179247, MASTER_HOST = '10.1.1.23', MASTER_USER = 'clx_repl', MASTER_PASSWORD = '123123', MASTER_PORT = 5306;
MySQL [(none)]> start slave;
MySQL [(none)]> show slave status\G

MySQL [scloud]> show slave status\G

*************************** 1. row ***************************

Slave_Name: default

Slave_Status: Running

Master_Host: 10.1.1.23

Master_Port: 5306

Master_User: clx_repl

Master_Log_File: clustrix-bin

Slave_Enabled: Enabled

Log_File_Seq: 3

Log_File_Pos: 26232091

Last_Error: no error

Connection_Status: Connected

Relay_Log_Bytes_Read: 0

Relay_Log_Current_Bytes: 12483

  Seconds_Behind_Master: 225

1 row in set (0.00 sec)

五、数据库账号迁移

导出命令：clustrix_clone_users -H localhost -u root > user_dump.sql 

导入命令：mysql -u root -h localhost < user_dump.sql

clxm@p2cn1uclx101m_10.248.100.241 ~]$ clustrix_clone_users -H localhost -u root > user_dump.sql
clxm@p2cn1uclx101m_10.248.100.241 ~]$ ll
total 4
-rw-r--r-- 1 clxm clxm 946 Nov 26 07:28 user_dump.sql
clxm@p2cn1uclx101m_10.248.100.241 ~]$ more user_dump.sql
--
-- Clustrix Users dumpfile ver: 197:46eb173adf75
-- Host: localhost
--

GRANT OSAUTH, REPLICATION CLIENT, SHUTDOWN ON *.* TO 'clustrix'@'localhost';

GRANT ALL PRIVILEGES ON `clustrix_dbi`.* TO 'clustrix'@'localhost';

GRANT ALL PRIVILEGES ON `clustrix_statd`.* TO 'clustrix'@'localhost';

GRANT SELECT ON `information_schema`.* TO 'clustrix'@'localhost';

GRANT SELECT ON `system`.* TO 'clustrix'@'localhost';

GRANT ALL PRIVILEGES ON *.* TO 'clustrix_ui'@'127.0.0.1' WITH GRANT OPTION;

GRANT OSAUTH, SHUTDOWN ON *.* TO 'clxm'@'localhost';

GRANT SELECT ON `system`.* TO 'clxm'@'localhost';

GRANT ALL PRIVILEGES ON *.* TO 'clx_maint'@'127.0.0.1' WITH GRANT OPTION;

GRANT ALL PRIVILEGES ON *.* TO 'clx_support'@'127.0.0.1' WITH GRANT OPTION;

GRANT ALL PRIVILEGES ON *.* TO 'clx_view_definer'@'127.0.0.1' WITH GRANT OPTION;

GRANT ALL PRIVILEGES ON *.* TO 'hogan'@'%';

GRANT ALL PRIVILEGES ON *.* TO 'mysql_slave'@'' WITH GRANT OPTION;
clxm@p2cn1uclx101m_10.248.100.241 ~]$ mysql -u root -h localhost < user_dump.sql

问题： 密码没有导出

六、锁等待，kill 会话操作

模拟锁表

在一个节点执行

MySQL [scloud]> begin; select * from scloud.sbtest1 where id = 4 for update;
Query OK, 0 rows affected (0.00 sec)

+----+----------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+------+
| id | k | c | pad | d |
+----+----------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+------+
| 4 | 50201319 | 37002370280-58842166667-00026392672-77506866252-09658311935-56926959306-83464667271-94685475868-28264244556-14550208498 | 63947013338-98809887124-59806726763-79831528812-45582457048 | NULL |
+----+----------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+------+
1 row in set (0.00 sec)

在另一个节点执行（会出现卡住等待锁超时）

MySQL [(none)]> begin; select * from scloud.sbtest1 where id = 4 for update;
Query OK, 0 rows affected (0.00 sec)

找到被锁SQL的session_id然后kill掉

MySQL [system]> select concat('kill ',session_id, ';') from sessions s join lockman l on s.xid = l.waiter;
+---------------------------------+
| concat('kill ',session_id, ';') |
+---------------------------------+
| kill 19458; |
+---------------------------------+
1 row in set (0.00 sec)

MySQL [system]> kill 19458;
Query OK, 0 rows affected (0.00 sec)

七、数据库升级

原版本：5.0.45-clustrix-9.1.4

升级后版本；5.0.45-clustrix-9.2

升级用户：clustrix

前提条件： 配置集群中 clustrix 用户免密

**升级操作不能回退，只能升，不能降。提前测试环境测试升级操作。

**主从环境，先升级从库。

1、修改/etc/profile,注释掉后五行 （三个节点都操作）

#df -h
#set -o vi
#stty erase ^H
#set -o vi
#stty erase ^H

修改之后执行 source /etc/profile

2、修改 clustrix用户环境变量，添加 :/opt/clustrix/bin

clustrix@p2cn1uclx101m_10.248.100.241 ~]$ cd

clustrix@p2cn1uclx101m_10.248.100.241 ~]$ cat .bash_profile

# .bash_profile

# Get the aliases and functions

if [ -f ~/.bashrc ]; then

. ~/.bashrc

fi

# User specific environment and startup programs

PATH=$PATH:$HOME/.local/bin:$HOME/bin:/opt/clustrix/bin

export PATH

clustrix@p2cn1uclx101m_10.248.100.241 ~]$ ./clustrix-clxnode-9.2-upgrade.el7.sh  install

Extracting New Cluster Management Tools...  Done

Configuring Upgrade Authentication...  Done

Checking for Node Clock Sync...  Done

Verifying Payload MD5 Checksum...  Done

Verifying dependencies...  Done

Checking for Available Disk Space...  Done

Pushing package to cluster...  Done

Extracting and Verifying Checksums on Upgrade...  Done

Preparing UI for upgrade...  Done

Installing Upgrade...  Done

Disabling Cluster Alerts...  Done

Stopping Database...  Done

Switching Symlinks to point at 9_2...  Done

Restarting database...2  Done

= = = = = = = = = = = = = = = = = = = = = =

ClustrixDB is now ready for use.

= = = = = = = = = = = = = = = = = = = = = =

Updating Clx UI...  Done

Enabling Cluster Alerts...  Done

Install completed successfully!

clustrix@p2cn1uclx101m_10.248.100.241 ~]$ mysql -uroot

Welcome to the MariaDB monitor.  Commands end with ; or \g.

Your MySQL connection id is 18434

Server version: 5.0.45-clustrix-9.2 

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [(none)]>

clustrix@p2cn1uclx101m_10.248.100.241 ~]$ ./clustrix-clxnode-9.2-upgrade.el7.sh  install

Extracting New Cluster Management Tools...

Done

Configuring Upgrade Authentication...  ERROR

Warning: CLX Command Failed

Fix CLX and try again.

2、/etc/profile 问题

clustrix@p2cn1uclx101m_10.248.100.241 ~]$ ./clustrix-clxnode-9.2-upgrade.el7.sh  install

Extracting New Cluster Management Tools...  Done

Configuring Upgrade Authentication...  Done

stty: standard input: Inappropriate ioctl for device

stty: standard input: Inappropriate ioctl for device

stty: standard input: Inappropriate ioctl for device

stty: standard input: Inappropriate ioctl for device

Checking for Node Clock Sync...stty: standard input: Inappropriate ioctl for device

stty: standard input: Inappropriate ioctl for device

stty: standard input: Inappropriate ioctl for device

stty: standard input: Inappropriate ioctl for device

Warning, non-integer retured by clx cmd 'date +%s': 'FilesystemSizeUsedAvailUse%Mountedon'

Check output of this command manually and fix the problem

 

八、剔除节点

原集群状态

clxm@p2cn1uclx101m_10.248.100.241 ~]$ clx stat
Cluster Name: cle69e350c2c16d729
Cluster Version: 9.2
Cluster Status: OK
Cluster Size: 3 nodes - 8 CPUs per Node
Current Node: ip-10-1-3-88 - nid 1

nid | Hostname | Status | IP Address | TPS | Used | Total
----+----------------+--------+-------------+-----+----------------+--------
1 | ip-10-1-3-88 | OK | 10.1.3.88 | 1 | 9.6G (12.9%) | 73.9G
2 | ip-10-1-3-242 | OK | 10.1.3.242 | 0 | 9.7G (13.1%) | 73.9G
3 | ip-10-1-3-151 | OK | 10.1.3.151 | 0 | 9.5G (12.8%) | 73.9G
----+----------------+--------+-------------+-----+----------------+--------
1 | 28.7G (13.0%) | 221.6G

剔除操作：

 

MySQL [system]> select * from nodeinfo;
+--------+----------------------------+----------------------------+----------------------------+-----------------------------------------------+------------+-------------------+------------+-----------------+------------+------------+---------+--------------+----------------+-------------------+-------+------------+----------+------+
| nodeid | started | ntptime | up_since | hostname | iface_name | iface_mac_addr | iface_ip | iface_connected | mysql_port | be_ip | be_port | control_port | healthmon_port | pnid | cores | first_pass | be_iface | zone |
+--------+----------------------------+----------------------------+----------------------------+-----------------------------------------------+------------+-------------------+------------+-----------------+------------+------------+---------+--------------+----------------+-------------------+-------+------------+----------+------+
| 2 | 2019-11-26 08:51:25.438532 | 2019-11-26 09:12:17.992518 | 2019-11-26 00:34:51.992518 | ip-10-1-3-242.cn-northwest-1.compute.internal | eth0 | 0a:27:62:3d:4b:78 | 10.1.3.242 | 1 | 5306 | 10.1.3.242 | 24378 | 2048 | 3581 | p94f8dacba3bda6a2 | 8 | 0 | eth0 | 0 |
| 3 | 2019-11-26 08:51:24.795313 | 2019-11-26 09:12:17.186910 | 2019-11-26 00:34:50.186910 | ip-10-1-3-151.cn-northwest-1.compute.internal | eth0 | 0a:36:35:de:83:9c | 10.1.3.151 | 1 | 5306 | 10.1.3.151 | 24378 | 2048 | 3581 | pfe8cd271ed7991fa | 8 | 0 | eth0 | 0 |
| 1 | 2019-11-26 08:51:25.519211 | 2019-11-26 09:12:18.003852 | 2019-11-26 00:34:52.003852 | ip-10-1-3-88.cn-northwest-1.compute.internal | eth0 | 0a:a4:d1:f0:b2:32 | 10.1.3.88 | 1 | 5306 | 10.1.3.88 | 24378 | 2048 | 3581 | p6259ec9897eb8bd9 | 8 | 0 | eth0 | 0 |
+--------+----------------------------+----------------------------+----------------------------+-----------------------------------------------+------------+-------------------+------------+-----------------+------------+------------+---------+--------------+----------------+-------------------+-------+------------+----------+------+
3 rows in set (0.00 sec)

MySQL [system]> ALTER CLUSTER drop '3';
Query OK, 0 rows affected (0.01 sec)

剔除后状态

clxm@p2cn1uclx101m_10.248.100.241 ~]$ clx stat
Cluster Name: cle69e350c2c16d729
Cluster Version: 9.2
Cluster Status: OK
Cluster Size: 2 nodes - 8 CPUs per Node
Current Node: ip-10-1-3-88 - nid 1

nid | Hostname | Status | IP Address | TPS | Used | Total
----+----------------+--------+-------------+-----+----------------+--------
1 | ip-10-1-3-88 | OK | 10.1.3.88 | 0 | 14.3G (19.3%) | 73.9G
2 | ip-10-1-3-242 | OK | 10.1.3.242 | 18 | 14.4G (19.5%) | 73.9G
----+----------------+--------+-------------+-----+----------------+--------
18 | 28.7G (19.4%) | 147.8G

剔除后做rebalance操作，数据总量不变