MyCat水平分库

一.什么是水平分库

将一张表水平切分到多个库中

1.1分片原则

1.需要分片的表是少数的
2.能不切分尽量不要切分
3.日志表可以采取归档方式
4.选择合适的切分规则和分片建，确保数据分片均匀，否则依然存在性能瓶颈
5.尽量避免跨分片join操作，保证关联操作表在同一分片

1.2分片后如何处理查询

1.根据简单分片规则，对分片键进行路由到正确的后端物理数据库
2.如果不是使用分片键的话，将会遍历后端数据库，极大消耗性能

二.水平切分步骤

2.1根据业务状态确定要进行水平切分的表

读写频繁，访问量非常大的表才需要切分，一般是订单表

如何选择分片键

1.尽可能的比较均匀分布数据到各个节点上，自增长主键并不是很好的选择，因为不会用于查询
2.业务字段是最频繁的或者最重要的查询条件

2.2分析业务模型选择分片键及分片算法

一般分片键选择的是频繁作为查询的字段，关键是能保证分片后的数据分布均匀，常用简单取模算法

对订单相关表进行水平切分
不仅仅是订单表，经常与订单表关联查询的表也需要一并分片，避免跨分片查询，大表不适合作为全局表
订单号，可以保证分片均匀，但是实际业务很少根据订单号来查询
下单人ID，业务查询更多，更适合，但是分片并不均匀，但不严重，值得考虑
采用简单取模分片算法，可以保证数据尽可能均匀

2.3适用mycat部署分片集群

1.使用schema.xml配置逻辑库及逻辑表

<!-- <table name="order_master" primaryKey="order_id" dataNode="dn_orderdb" /> -->

	<table name="order_master" primaryKey="order_id" dataNode="dn_orderdb01,dn_orderdb02,dn_orderdb03,dn_orderdb04" rule="order_master"/>

<!-- 只需要增加表所在的物理节点schema名称和指定规则rule -->

<dataNode name="dn_orderdb01" dataHost="mysql10143" database="orderdb01" />

<dataNode name="dn_orderdb02" dataHost="mysql10143" database="orderdb02" />

<dataNode name="dn_orderdb03" dataHost="mysql10144" database="orderdb03" />

<dataNode name="dn_orderdb04" dataHost="mysql10144" database="orderdb04" />

2.使用rule.xml配置分片表的分片规则

    <tableRule name="order_master">

        <rule>

            <columns>customer_id</columns>

            <algorithm>mod-long</algorithm>

        </rule>

    </tableRule>

    <function name="mod-long" class="org.opencloudb.route.function.PartitionByMod">

    <!-- 分片算法，唯一；JAVA类路径，1.6是"io.mycat.route.function.PartitionByMod"-->

        <property name="count">4</property>

    </function>

3.验证

# 现在逻辑库上进行查看，此时是没有数据的

# 需要提前建库建表，否则报错表不存在

app_imooc@172.16.10.142 00:26:  [imooc_db]> select * from order_master;

Empty set (0.08 sec)

# 在逻辑库imooc_db上插入5条数据，正常是orderdb02 2条数据（节点索引顺序为1），其他各1条数据

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70001,90001,'yzw1',1,1,1,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70002,90002,'yzw2',2,2,2,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70003,90003,'yzw3',3,3,3,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70004,90004,'yzw4',4,4,4,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70005,90005,'yzw5',5,5,5,'address1',1,20);

# truncate table报错

app_imooc@172.16.10.142 10:51:  [imooc_db]> truncate table order_master;

ERROR 1105 (HY000): DROP command denied to user 'bm_mycat'@'172.16.10.142' for table 'order_master'

# 逻辑库查看插入结果

app_imooc@172.16.10.142 10:50:  [imooc_db]> select order_id,order_sn,customer_id from order_master;

+----------+----------+-------------+

| order_id | order_sn | customer_id |

+----------+----------+-------------+

|    80005 |    70002 |       90002 |

|    80005 |    70003 |       90003 |

|    80005 |    70004 |       90004 |

|    80006 |    70001 |       90001 |

|    80007 |    70005 |       90005 |

+----------+----------+-------------+

5 rows in set (0.00 sec)

# 在物理库查看是否有这4条数据，验证成功

root@localhost 10:52:  [orderdb01]> select order_id,order_sn,customer_id from order_master;

+----------+----------+-------------+

| order_id | order_sn | customer_id |

+----------+----------+-------------+

|    80005 |    70004 |       90004 |

+----------+----------+-------------+

1 row in set (0.00 sec)

root@localhost 10:52:  [orderdb02]> select order_id,order_sn,customer_id from order_master;

+----------+----------+-------------+

| order_id | order_sn | customer_id |

+----------+----------+-------------+

|    80006 |    70001 |       90001 |

|    80007 |    70005 |       90005 |

+----------+----------+-------------+

2 rows in set (0.00 sec)

root@localhost 10:05:  [orderdb03]> select order_id,order_sn,customer_id from order_master;

+----------+----------+-------------+

| order_id | order_sn | customer_id |

+----------+----------+-------------+

|    80005 |    70002 |       90002 |

+----------+----------+-------------+

1 row in set (0.00 sec)

root@localhost 10:52:  [orderdb04]> select order_id,order_sn,customer_id from order_master;

+----------+----------+-------------+

| order_id | order_sn | customer_id |

+----------+----------+-------------+

|    80005 |    70003 |       90003 |

+----------+----------+-------------+

1 row in set (0.00 sec)

4.使用server.xml配置访问权限

#配置用户登录

2.4测试分片集群，采用应用端双写方式进行

2.5业务及数据迁移

三.全局自增ID

分片表中的自增ID在逻辑表中有重复
第三方给ID或者使用mycat自增ID

3.1全局自增ID方法

1.本地文件方式：适用服务器本地磁盘的方式
2.数据库方式：适用数据库存储的方式
3.本地时间戳方式：适用时间戳
4.分布式zookeeper生成ID

3.2本地文件全局ID

优点：本地加载，读取速度快，配置简单
缺点：集群部署无法使用，不同的mycat无法保证id唯一，使mycat变成了有状态的中间件

配置方法

1.server.xml增加属性

<property name="sequnceHandlerType>0</property>

2.sequence_conf.properties配置

#default global sequence

GLOBAL.HISIDS=

GLOBAL.MINID=10001

GLOBAL.MAXID=20000

GLOBAL.CURID=10000

# self define sequence

ORDER_MASTER.HISIDS=

ORDER_MASTER.MINID=1001

ORDER_MASTER.MAXID=2000

ORDER_MASTER.CURID=1000

# 以上配置文件中，自定义表名必须大写书写HISIDS：表示使用过的历史分段(一般无特殊需要可不配置)

# MINID ：最小ID 值

# MAXID ：表示最大ID 值

# CURID ：表示当前ID 值。

# 当 sequence_conf.properties的配置名字与 表名一致的时候sql可以不包含ID字段（此处表名为id_local_file），逻辑表需要增加属性autoIncrement="true"

3.插入数据

# mycat restart

# delete from order_master;

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70001,90001,'yzw1',1,1,1,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70002,90002,'yzw2',2,2,2,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70003,90003,'yzw3',3,3,3,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70004,90004,'yzw4',4,4,4,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70005,90005,'yzw5',5,5,5,'address1',1,20);

# 此时逻辑表中的order_id唯一

app_imooc@172.16.10.142 11:35:  [imooc_db]> select order_id,order_sn,customer_id from order_master;

+----------+----------+-------------+

| order_id | order_sn | customer_id |

+----------+----------+-------------+

|    10004 |    70004 |       90004 |

|    10003 |    70003 |       90003 |

|    10001 |    70001 |       90001 |

|    10005 |    70005 |       90005 |

|    10002 |    70002 |       90002 |

+----------+----------+-------------+

5 rows in set (0.01 sec)

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70001,90001,'yzw1',1,1,1,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70002,90002,'yzw2',2,2,2,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70003,90003,'yzw3',3,3,3,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70004,90004,'yzw4',4,4,4,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70005,90005,'yzw5',5,5,5,'address1',1,20);

# 此时逻辑表中的order_id依然是唯一，但是ID值使用的是自定义表自增属性的值

app_imooc@172.16.10.142 11:48:  [imooc_db]> select order_id,order_sn,customer_id from order_master;

+----------+----------+-------------+

| order_id | order_sn | customer_id |

+----------+----------+-------------+

|     1009 |    70003 |       90003 |

|     1008 |    70002 |       90002 |

|     1007 |    70001 |       90001 |

|     1011 |    70005 |       90005 |

|     1010 |    70004 |       90004 |

+----------+----------+-------------+

5 rows in set (0.00 sec)

# 不指定自增字段插入

# 逻辑表必须增加自增属性，否则不指定自增字段插入则使用分片表本身自增id

#<table name="order_master" primaryKey="order_id" autoIncrement="true" dataNode="dn_orderdb01,dn_orderdb02,dn_orderdb03,dn_orderdb04" rule="order_master"/>

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70001,90001,'yzw1',1,1,1,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70002,90002,'yzw2',2,2,2,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70003,90003,'yzw3',3,3,3,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70004,90004,'yzw4',4,4,4,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70005,90005,'yzw5',5,5,5,'address1',1,20);

app_imooc@172.16.10.142 11:58:  [imooc_db]> select order_id,order_sn,customer_id from order_master;

+----------+----------+-------------+

| order_id | order_sn | customer_id |

+----------+----------+-------------+

|     1001 |    70001 |       90001 |

|     1005 |    70005 |       90005 |

|     1003 |    70003 |       90003 |

|     1004 |    70004 |       90004 |

|     1002 |    70002 |       90002 |

+----------+----------+-------------+

5 rows in set (0.01 sec)

3.3数据库方式

优点：mycat重启后，sequence值不会被初始化，因为从数据库中取，每次取完按自增步长增加
缺点：sequence数据库主从切换后，存在mycat适用缓存序列号，可能存在ID重复风险，可以手动增加步长避免

配置方法

1.server.xml增加属性

<property name="sequnceHandlerType">1</property>

2.sequence_db_conf.properties

#sequence stored in datanode

GLOBAL=dn_test

ORDER_MASTER=dn_test

datanode必须要mycat能访问的数据库，并不一定需要在分片表的数据库中

3.schema.xml增加主机节点和数据节点，让mycat能访问上面配置的数据节点
4.在对应节点的数据库中增加函数和表，用户必须授予表、函数、存储过程的访问权限

# 需要全局ID的表明需要写入表中，大写

INSERT INTO mycat_sequence values('ORDER_MASTER', 1, 100);

root@master 11:48:  [test]> select * from mycat_sequence;

+--------------+---------------+-----------+

| NAME         | current_value | increment |

+--------------+---------------+-----------+

| GLOBAL       |        100000 |       100 |

| ORDER_MASTER |             1 |       100 |

+--------------+---------------+-----------+

2 rows in set (0.00 sec)

5.schema.xml增加配置，适用全局自增IDautoIncrement="true"

<table name="order_master" primaryKey="order_id" autoIncrement="true" dataNode="dn_orderdb01,dn_orderdb02,dn_orderdb03,dn_orderdb04" rule="order_master"/>

6.在逻辑库中插入数据

# 适用ID表里面的全局ID插入

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70001,90001,'yzw1',1,1,1,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70002,90002,'yzw2',2,2,2,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70003,90003,'yzw3',3,3,3,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70004,90004,'yzw4',4,4,4,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70005,90005,'yzw5',5,5,5,'address1',1,20);

# 使用ID表里面的对应表ID插入

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70001,90001,'yzw1',1,1,1,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70002,90002,'yzw2',2,2,2,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70003,90003,'yzw3',3,3,3,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70004,90004,'yzw4',4,4,4,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70005,90005,'yzw5',5,5,5,'address1',1,20);

# 不指定id列，效果同上

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70001,90001,'yzw1',1,1,1,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70002,90002,'yzw2',2,2,2,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70003,90003,'yzw3',3,3,3,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70004,90004,'yzw4',4,4,4,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70005,90005,'yzw5',5,5,5,'address1',1,20);

3.4本地时间戳方式

ID= 64 位二进制 (42(毫秒)+5(机器 ID)+5(业务编码)+12(重复累加)

优点：与mycat无关，跟时间有关系
缺点：字段最大长度必须大于等于18，int无法满足

配置方法

1.server.xml增加属性

<property name="sequnceHandlerType">2</property>

2.sequence_time_conf.properties

#sequence depend on TIME

WORKID=01

DATAACENTERID=01

# 0-31任意整数

3.在对应节点的数据库中增加函数和表，用户必须授予表、函数、存储过程的访问权限

# 需要全局ID的表明需要写入表中，大写

INSERT INTO mycat_sequence values('ORDER_MASTER', 1, 100);

root@master 11:48:  [test]> select * from mycat_sequence;

+--------------+---------------+-----------+

| NAME         | current_value | increment |

+--------------+---------------+-----------+

| GLOBAL       |        100000 |       100 |

| ORDER_MASTER |             1 |       100 |

+--------------+---------------+-----------+

2 rows in set (0.00 sec)

5.schema.xml增加配置，适用全局自增IDautoIncrement="true"

<table name="order_master" primaryKey="order_id" autoIncrement="true" dataNode="dn_orderdb01,dn_orderdb02,dn_orderdb03,dn_orderdb04" rule="order_master"/>

6.在逻辑库中插入数据

# int无法满足

ERROR 1264 (22003): Out of range value for column 'order_id' at row 

app_imooc@172.16.10.142 12:32:  [imooc_db]> select next value for MYCATSEQ_ORDER_MASTER;

+---------------------+

| 1025962766269288448 |

+---------------------+

| 1025962766269288448 |

+---------------------+

1 row in set (0.00 sec)

app_imooc@172.16.10.142 12:32:  [imooc_db]> select next value for MYCATSEQ_GLOBAL;

+---------------------+

| 1025962827724230656 |

+---------------------+

| 1025962827724230656 |

+---------------------+

1 row in set (0.00 sec)

1

# 适用ID表里面的全局ID插入

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70001,90001,'yzw1',1,1,1,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70002,90002,'yzw2',2,2,2,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70003,90003,'yzw3',3,3,3,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70004,90004,'yzw4',4,4,4,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70005,90005,'yzw5',5,5,5,'address1',1,20);

# 使用ID表里面的对应表ID插入

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70001,90001,'yzw1',1,1,1,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70002,90002,'yzw2',2,2,2,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70003,90003,'yzw3',3,3,3,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70004,90004,'yzw4',4,4,4,'address1',1,20);

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70005,90005,'yzw5',5,5,5,'address1',1,20);

# 不指定id列，效果同上

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70001,90001,'yzw1',1,1,1,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70002,90002,'yzw2',2,2,2,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70003,90003,'yzw3',3,3,3,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70004,90004,'yzw4',4,4,4,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70005,90005,'yzw5',5,5,5,'address1',1,20);

select order_id,order_sn,customer_id from order_master;

3.5分布式zookeeper生成ID 未测试，1.5没有默认配置文件

基于ZK 与本地配置的分布式ID 生成器(可以通过ZK 获取集群（机房）唯一InstanceID，也可以通过配置文件配置InstanceID)ID 结构：long 64 位，ID 最大可占63 位

current time millis(微秒时间戳38 位,可以使用17 年)

instanceId（实例ID，可以通过ZK 或者配置文件获取，5 位，也就是十进制0-31）

threadId（线程ID，9 位）

increment(自增,6 位)

一共63 位，可以承受单机房单机器单线程1000*(2^6)=640000 的并发。

优点：无悲观锁，无强竞争，吞吐量更高
缺点：对zookeeper集群的要求增加。

配置方法

1.server.xml增加属性

<property name="sequnceHandlerType">3</property>

2.sequence_distributed_conf.properties

INSTANCEID=01

CLUSTERID=01

3.在对应节点的数据库中增加函数和表，用户必须授予表、函数、存储过程的访问权限

# 需要全局ID的表明需要写入表中，大写

INSERT INTO mycat_sequence values('ORDER_MASTER', 1, 100);

root@master 11:48:  [test]> select * from mycat_sequence;

+--------------+---------------+-----------+

| NAME         | current_value | increment |

+--------------+---------------+-----------+

| GLOBAL       |        100000 |       100 |

| ORDER_MASTER |             1 |       100 |

+--------------+---------------+-----------+

2 rows in set (0.00 sec)

5.schema.xml增加配置，适用全局自增IDautoIncrement="true"

<table name="order_master" primaryKey="order_id" autoIncrement="true" dataNode="dn_orderdb01,dn_orderdb02,dn_orderdb03,dn_orderdb04" rule="order_master"/>

6.在逻辑库中插入数据

四.ER关系

跨分片查询存在跨节点问题

app_imooc@172.16.10.142 13:55:  [imooc_db]> select * from order_master m join order_detail o on m.order_id=o.order_id;

ERROR 1064 (HY000): invalid route in sql, multi tables found but datanode has no intersection  sql:select * from order_master m join order_detail o on m.order_id=o.order_id

根据ER关系，也需要对order_detail这张表进行分片

4.1配置步骤

1.在原先order_master所在4个分配数据库(orderdb01/02/03/04)中建立order_detail表结构

CREATE TABLE `order_detail` (

  `order_detail_id` int(10) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增主键ID,订单详情表ID',

  `order_id` int(10) unsigned NOT NULL COMMENT '订单表ID',

  `product_id` int(10) unsigned NOT NULL COMMENT '订单商品ID',

  `product_name` varchar(50) NOT NULL COMMENT '商品名称',

  `product_cnt` int(11) NOT NULL DEFAULT '1' COMMENT '购买商品数量',

  `product_price` decimal(8,2) NOT NULL COMMENT '购买商品单价',

  `average_cost` decimal(8,2) NOT NULL DEFAULT '0.00' COMMENT '平均成本价格',

  `weight` float DEFAULT NULL COMMENT '商品重量',

  `fee_money` decimal(8,2) NOT NULL DEFAULT '0.00' COMMENT '优惠分摊金额',

  `w_id` int(10) unsigned NOT NULL COMMENT '仓库ID',

  `modified_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '最后修改时间',

  PRIMARY KEY (`order_detail_id`)

) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='订单详情表';

2.schema.xml将全局分片表进行ER分片，在父表中增加子表信息

<table name="order_master" primaryKey="order_id" autoIncrement="true" dataNode="dn_orderdb01,dn_orderdb02,dn_orderdb03,dn_orderdb04" rule="order_master">

    <childTable name="order_detail" primaryKey="order_detail_id" joinKey="order_id" parentKey="order_id" autoIncrement="true" />

    <!-- 指定关联子表的表名、主键，及父表关联key、父表的主键，同时也需要指定全局ID -->

</table>

3.配置表order_detail全局自增ID(使用数据库方式)

# server.xml

# sequence_db_conf.properties增加order_detail自增ID

#sequence stored in datanode

GLOBAL=dn_test

ORDER_MASTER=dn_test

ORDER_DETAIL=dn_test

# 数据库中增加分片表信息

insert into mycat_sequence values('ORDER_DETAIL',1,1);

4.插入数据

# 先插入order_master

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70001,90001,'yzw1',1,1,1,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70002,90002,'yzw2',2,2,2,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70003,90003,'yzw3',3,3,3,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70004,90004,'yzw4',4,4,4,'address1',1,20);

insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70005,90005,'yzw5',5,5,5,'address1',1,20);

app_imooc@172.16.10.142 14:46:  [imooc_db]> select order_sn,customer_id,order_id from order_master;

+----------+-------------+----------+

| order_sn | customer_id | order_id |

+----------+-------------+----------+

|    70005 |       90005 |        5 |

|    70001 |       90001 |        6 |

|    70005 |       90005 |       10 |

|    70004 |       90004 |        4 |

|    70004 |       90004 |        9 |

|    70003 |       90003 |        3 |

|    70003 |       90003 |        8 |

|    70002 |       90002 |        2 |

|    70002 |       90002 |        7 |

+----------+-------------+----------+

9 rows in set (0.00 sec)

# 根据order_id插入order_detail

insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,2,12,'商品2',1,22.2,12,1,10,2);

insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,3,13,'商品3',1,22.2,12,1,10,2);

insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,4,14,'商品4',1,22.2,12,1,10,2);

insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,5,15,'商品5',1,22.2,12,1,10,2);

insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,6,16,'商品6',1,22.2,12,1,10,2);

insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,7,17,'商品7',1,22.2,12,1,10,2);

insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,8,18,'商品8',1,22.2,12,1,10,2);

insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,9,19,'商品9',1,22.2,12,1,10,2);

insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,10,20,'商品10',1,22.2,12,1,10,2);

insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,5,21,'商品11',1,22.2,12,1,10,2);

insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,6,12,'商品2',1,22.2,12,1,10,2);

insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,8,12,'商品2',1,22.2,12,1,10,2);

insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,8,12,'商品2',1,22.2,12,1,10,2);

5.此时再进行分片关联查询验证数据

select m.order_id,o.order_detail_id,o.product_id,o.product_name from order_master m join order_detail o on m.order_id=o.order_id;

+----------+-----------------+------------+--------------+

| order_id | order_detail_id | product_id | product_name |

+----------+-----------------+------------+--------------+

|        3 |               4 |         13 | 商品3        |

|        8 |               9 |         18 | 商品8        |

|        8 |              14 |         12 | 商品2        |

|        8 |              15 |         12 | 商品2        |

|        2 |               3 |         12 | 商品2        |

|        7 |               8 |         17 | 商品7        |

|        4 |               5 |         14 | 商品4        |

|        9 |              10 |         19 | 商品9        |

|        5 |               6 |         15 | 商品5        |

|        6 |               7 |         16 | 商品6        |

|       10 |              11 |         20 | 商品10       |

|        5 |              12 |         21 | 商品11       |

|        6 |              13 |         12 | 商品2        |

+----------+-----------------+------------+--------------+

13 rows in set (0.00 sec)

6.在其中一个分片上可以验证，order_id一致的表落在同一个分片

root@localhost 15:05:  [orderdb02]> select order_sn,customer_id,order_id,mod(customer_id,4) from order_master;

+----------+-------------+----------+

| order_sn | customer_id | order_id |

+----------+-------------+----------+

|    70005 |       90005 |        5 |

|    70001 |       90001 |        6 |

|    70005 |       90005 |       10 |

+----------+-------------+----------+

3 rows in set (0.00 sec)

root@localhost 15:06:  [orderdb02]> select order_detail_id,product_id,order_id from order_detail;

+-----------------+------------+----------+

| order_detail_id | product_id | order_id |

+-----------------+------------+----------+

|               6 |         15 |        5 |

|               7 |         16 |        6 |

|              11 |         20 |       10 |

|              12 |         21 |        5 |

|              13 |         12 |        6 |

+-----------------+------------+----------+

5 rows in set (0.00 sec)

MyCat水平分库的更多相关文章

01.SQLServer性能优化之---水平分库扩展
汇总篇:http://www.cnblogs.com/dunitian/p/4822808.html#tsql 第一次引入文件组的概念:http://www.cnblogs.com/dunitian/ ...
【Java EE 学习 77 下】【数据采集系统第九天】【使用spring实现答案水平分库】【未解决问题：分库查询问题】
之前说过,如果一个数据库中要存储的数据量整体比较小,但是其中一个表存储的数据比较多,比如日志表,这时候就要考虑分表存储了:但是如果一个数据库整体存储的容量就比较大,该怎么办呢?这时候就需要考虑分库了, ...
02.SQLServer性能优化之---水平分库扩展
汇总篇:http://www.cnblogs.com/dunitian/p/4822808.html#tsql 第一次引入文件组的概念:http://www.cnblogs.com/dunitian/ ...
SQLServer性能优化之---水平分库扩展
汇总篇:http://www.cnblogs.com/dunitian/p/4822808.html#tsql 第一次引入文件组的概念:http://www.cnblogs.com/dunitia ...
MyCat垂直分库
一.什么是垂直分库将一类功能的表从一个实例切分到另一个实例,横向扩展实例,增加写负载目标:将1个实例的4类表拆分多4个实例中二.垂直切分步骤 2.1收集分析业务模块间的关系,能分几个库 2.2全 ...
Sharding-JDBC 实现水平分库分表
1.需求分析
Docker安装Mycat和Mysql进行水平分库分表实战【图文教学】
一.前言小编最近公司有个新的需求,数据量比较大,要涉及到分库分表.大概了解了一些主流的中间件,使用和网上资料比较多的是Mycat和sharding-jdbc,小编比较倾向于Mycat.原因很简单就是 ...
Mycat配置分库分表（垂直分库、水平分表）、全局序列
1. Mycat相关文章 Linux安装Mycat1.6.7.4并实现Mysql数据库读写分离简单配置 Linux安装Mysql8.0.20并配置主从复制(一主一从,双主双从) Docke ...
MySQL：如何使用MyCAT实现分库分表？
分库分表介绍随着微服务这种架构的兴起,我们应用从一个完整的大的应用,切分为很多可以独立提供服务的小应用.每个应用都有独立的数据库. 数据的切分分为两种: 垂直切分:按照业务模块进行切分,将不同模块的 ...

随机推荐

Python（3）
使用除法来缩减数字,使用余数法来计算个数. class Solution: def hammingWeight(self, n: int) -> int: count = 0 while Tru ...
Python 替换文本中的某些词语
https://stackoverflow.com/questions/39086/search-and-replace-a-line-in-a-file-in-python from tempfil ...
Ubuntu 设置 log 级别
Linux环境下使用rsyslog管理日志 rsyslog linux运维 linux 22.7k 次阅读 · 读完需要 22 分钟在 Linux 系统中,日志文件记录了系统中包括内核. ...
[redis]SDS和链表
一.SDS 1.SDS结构体 redis3.2之前:不管buf的字节数有多少,都用 4字节的len来储存长度,对于只存短字符串那么优点浪费空间,比如只存 name,则len=4 则只需要一个字节8位即 ...
SpringBoot应用操作Rabbitmq（fanout广播高级操作）
一.广播模式fanout.不需要指定路由key. 注:与topic和direct区别是:fanout广播模式会两个队列同时发送相同的消息,并非由交换器转发到某一个队列二.实战(广播模式) 1.引入m ...
OOM的起点到终点
前言 1.问题及现象线上日志反馈内存溢出问题.根据用户反馈,客户操作一段时间之后,APP 内存溢出崩溃. 2.分析过程 (1) 分析线上日志,发现主要分两种: 第一种如下,可能是某个死循环导致内存不 ...
Android Studio SVN配置忽略文件
1.用Android Studio创建一个项目,会在根目录和Module目录下自动生成.gitignore文件,貌似是Git的配置文件,和SVN没有关系. 2.打开Setting-Version Co ...
Highcharts的自适应DOM或者DIV，JS方法实现
那我们就按照官网的一分钟极速入门代码来说 // 图表配置 var options = { chart: { type: 'bar' //指定图表的类型,默认是折线图(line) }, title: { ...
Eclipse Collections：让Java Streams更上一层楼
\ 关键要点 \\ Eclipse Collections是一个高性能的Java集合框架,为原生JDK集合增加了丰富的功能.\\t Streams是JDK的一个非常受欢迎的功能,但它缺少了一些特性,严 ...
1745 Divisibility
Divisibility Time Limit: 1000MS Memory Limit: 10000K Total Submissions: 14084 Accepted: 4989 Descrip ...

MyCat水平分库

一.什么是水平分库

1.1分片原则

1.2分片后如何处理查询

二.水平切分步骤

2.1根据业务状态确定要进行水平切分的表

如何选择分片键

2.2分析业务模型选择分片键及分片算法

2.3适用mycat部署分片集群

2.4测试分片集群，采用应用端双写方式进行

2.5业务及数据迁移

三.全局自增ID

3.1全局自增ID方法

3.2本地文件全局ID

配置方法

3.3数据库方式

配置方法

3.4本地时间戳方式

配置方法

3.5分布式zookeeper生成ID 未测试，1.5没有默认配置文件

配置方法

四.ER关系

4.1配置步骤

MyCat水平分库的更多相关文章

随机推荐

热门专题