原文:数据库垂直拆分,水平拆分利器,cobar升级版mycat

1,关于Mycat

Mycat情报 
基于阿里的开源cobar ,可以用于生产系统中,目前在做如下的一些改进: 
非阻塞IO的实现,相对于目前的cobar,并发性能大大提升,而且不会陷入假死状态 
优化线程池的分配,目前cobar的线程池分配效率不高 
修复cobar一些BUG 
参考impala中的impala front部分的Java代码,实现高效的Map-Reduce,能够处理上亿的大数据量 
实现自动分片特性,目前cobar需要手工分片,并有一定的编程限制 
官方网站: 
https://github.com/MyCATApache/ 
好多文档,大多数都是word的,写的非常详细。 
https://github.com/MyCATApache/Mycat-doc 
Mycat是cobar重新优化开发的版本,和cobar的很多配置都类似。 
可以参考之前写的cobar安装: 
http://blog.csdn.net/freewebsys/article/details/44022421

2,安装Mycat服务

下载二进制安装文件 
https://github.com/MyCATApache/Mycat-download 
解压缩到/usr/local/mycat 
修改配置文件:

<?xml version="1.0"?>
<!DOCTYPE mycat:schema SYSTEM "schema.dtd">
<mycat:schema xmlns:mycat="http://org.opencloudb/">
#数据库名称是TESTDB,sqlMaxLimit设置limit防止错误sql查询大量数据
<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100">
#自动分库规则
<!-- auto sharding by id (long) -->
<table name="travelrecord" dataNode="dn1,dn2,dn3" rule="auto-sharding-long" />
#全局配置表,所有数据均同步到每个数据库。
<!-- global table is auto cloned to all defined data nodes ,so can join
with any table whose sharding node is in the same data node -->
<table name="company" primaryKey="ID" type="global" dataNode="dn1,dn2,dn3" />
<table name="goods" primaryKey="ID" type="global" dataNode="dn1,dn2" /> <!-- random sharding using mod sharind rule -->
<table name="hotnews" primaryKey="ID" dataNode="dn1,dn2,dn3" rule="mod-long" />
<table name="employee" primaryKey="ID" dataNode="dn1,dn2"
rule="sharding-by-intfile" />
#关联子表配置,不太明白
<table name="customer" primaryKey="ID" dataNode="dn1,dn2"
rule="sharding-by-intfile">
<childTable name="orders" primaryKey="ID" joinKey="customer_id"
parentKey="id">
<childTable name="order_items" joinKey="order_id"
parentKey="id" />
</childTable>
<childTable name="customer_addr" primaryKey="ID" joinKey="customer_id"
parentKey="id" />
</table>
</schema>
#配置多个dataNode,制定dataHost和数据库名称。
<dataNode name="dn1" dataHost="localhost1" database="db1" />
<dataNode name="dn2" dataHost="localhost1" database="db2" />
<dataNode name="dn3" dataHost="localhost1" database="db3" />
#配置dataHost的读写分配。同时mysql也要配置好,Master-Slave或Master-Master
<dataHost name="localhost1" maxCon="1000" minCon="10" balance="0"
writeType="0" dbType="mysql" dbDriver="native">
<heartbeat>select user()</heartbeat>
<!-- can have multi write hosts -->
<writeHost host="hostM1" url="localhost:3306" user="root"
password="123456">
<!-- can have multi read hosts -->
<!-- <readHost host="hostS1" url="localhost:3306" user="root" password="123456"
/> -->
</writeHost>
<!-- <writeHost host="hostM2" url="localhost:3316" user="root" password="123456"/> -->
</dataHost> </mycat:schema>

wrapper.conf是mycat的配置文件启动参数等 
rule.xml 是配置规则xml 
还有几个配置文件慢慢研究 
创建数据库

CREATE DATABASE db1 DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
CREATE DATABASE db2 DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
CREATE DATABASE db3 DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;

服务启动:/bin/mycat start 
/bin/mycat这个问题同时可以作为service,拷贝到/etc/init.d/目录下即可。

3,登陆mycat服务和管理端

登陆mycat服务:(端口8066)

# mysql -utest -ptest -h 127.0.0.1 -P8066
Warning: USING a password ON the command line interface can be insecure.
Welcome TO the MySQL monitor. Commands END WITH ; OR \g.
Your MySQL connection id IS 1
Server version: 5.5.8-mycat-1.3 MyCat Server (OpenCloundDB) Copyright (c) 2000, 2015, Oracle AND/OR its affiliates. ALL rights reserved. Oracle IS a registered trademark OF Oracle Corporation AND/OR its
affiliates. Other names may be trademarks OF their respective
owners. TYPE 'help;' OR '\h' FOR help. TYPE '\c' TO clear the CURRENT INPUT statement. mysql> SHOW DATABASES;
+----------+
| DATABASE |
+----------+
| TESTDB |
+----------+
1 ROW IN SET (0.02 sec) mysql> USE TESTDB;
Reading TABLE information FOR completion OF TABLE AND COLUMN names
You can turn off this feature TO GET a quicker startup WITH -A DATABASE changed
mysql> SHOW TABLES;
+------------------+
| TABLES IN TESTDB |
+------------------+
| company |
| customer |
| customer_addr |
| employee |
| goods |
| hotnews |
| orders |
| order_items |
| travelrecord |
+------------------+
9 ROWS IN SET (0.00 sec) mysql> CREATE TABLE company(id INT NOT NULL PRIMARY KEY,name VARCHAR(100));
Query OK, 0 ROWS affected (0.05 sec) mysql> EXPLAIN CREATE TABLE company(id INT NOT NULL PRIMARY KEY,name VARCHAR(100));
+-----------+---------------------------------------------------------------------+
| DATA_NODE | SQL |
+-----------+---------------------------------------------------------------------+
| dn1 | CREATE TABLE company(id INT NOT NULL PRIMARY KEY,name VARCHAR(100)) |
| dn2 | CREATE TABLE company(id INT NOT NULL PRIMARY KEY,name VARCHAR(100)) |
| dn3 | CREATE TABLE company(id INT NOT NULL PRIMARY KEY,name VARCHAR(100)) |
+-----------+---------------------------------------------------------------------+
3 ROWS IN SET (0.01 sec) mysql> INSERT INTO company(id,name) VALUES(1,'hp'); INSERT INTO company(id,name) VALUES(2,'ibm'); INSERT INTO company(id,name) VALUES(3,'oracle');
Query OK, 3 ROWS affected (0.37 sec) Query OK, 3 ROWS affected (0.01 sec) Query OK, 3 ROWS affected (0.00 sec) mysql> SELECT * FROM company ;
+----+--------+
| id | name |
+----+--------+
| 1 | hp |
| 2 | ibm |
| 3 | oracle |
+----+--------+
3 ROWS IN SET (0.01 sec)

使用管理端登陆:(端口9066)

#  mysql -utest -ptest -h 127.0.0.1 -P9066
Warning: USING a password ON the command line interface can be insecure.
Welcome TO the MySQL monitor. Commands END WITH ; OR \g.
Your MySQL connection id IS 2
Server version: 5.5.8-mycat-1.3 CobarManager@Alibaba Copyright (c) 2000, 2015, Oracle AND/OR its affiliates. ALL rights reserved. Oracle IS a registered trademark OF Oracle Corporation AND/OR its
affiliates. Other names may be trademarks OF their respective
owners. TYPE 'help;' OR '\h' FOR help. TYPE '\c' TO clear the CURRENT INPUT statement. mysql> SHOW @@help;
+--------------------------------------+-----------------------------------+
| STATEMENT | DESCRIPTION |
+--------------------------------------+-----------------------------------+
| clear @@slow WHERE datanode = ? | Clear slow SQL BY datanode |
| clear @@slow WHERE schema = ? | Clear slow SQL BY schema |
| KILL @@connection id1,id2,... | KILL the specified connections |
| offline | CHANGE MyCat STATUS TO OFF |
| online | CHANGE MyCat STATUS TO ON |
| reload @@config | Reload ALL config FROM file |
| reload @@route | Reload route config FROM file |
| reload @@USER | Reload USER config FROM file |
| ROLLBACK @@config | ROLLBACK ALL config FROM memory |
| ROLLBACK @@route | ROLLBACK route config FROM memory |
| ROLLBACK @@USER | ROLLBACK USER config FROM memory |
| SHOW @@backend | Report backend connection STATUS |
| SHOW @@cache | Report system cache usage |
| SHOW @@command | Report commands STATUS |
| SHOW @@connection | Report connection STATUS |
| SHOW @@connection.SQL | Report connection SQL |
| SHOW @@DATABASE | Report DATABASES |
| SHOW @@datanode | Report dataNodes |
| SHOW @@datanode WHERE schema = ? | Report dataNodes |
| SHOW @@datasource | Report dataSources |
| SHOW @@datasource WHERE dataNode = ? | Report dataSources |
| SHOW @@heartbeat | Report heartbeat STATUS |
| SHOW @@parser | Report parser STATUS |
| SHOW @@processor | Report processor STATUS |
| SHOW @@router | Report router STATUS |
| SHOW @@server | Report server STATUS |
| SHOW @@SESSION | Report front SESSION details |
| SHOW @@slow WHERE datanode = ? | Report datanode slow SQL |
| SHOW @@slow WHERE schema = ? | Report schema slow SQL |
| SHOW @@SQL WHERE id = ? | Report specify SQL |
| SHOW @@SQL.detail WHERE id = ? | Report EXECUTE detail STATUS |
| SHOW @@SQL.EXECUTE | Report EXECUTE STATUS |
| SHOW @@SQL.slow | Report slow SQL |
| SHOW @@threadpool | Report threadPool STATUS |
| SHOW @@TIME.CURRENT | Report CURRENT TIMESTAMP |
| SHOW @@TIME.startup | Report startup TIMESTAMP |
| SHOW @@version | Report Mycat Server version |
| stop @@heartbeat name:TIME | Pause dataNode heartbeat |
| switch @@datasource name:INDEX | Switch dataSource |
+--------------------------------------+-----------------------------------+
39 ROWS IN SET (0.03 sec) mysql> SHOW @@backend;
+------------+------+---------+-----------------+------+--------+--------+---------+------+--------+----------+------------+--------+---------+------------+
| processor | id | mysqlId | host | port | l_port | net_in | net_out | life | closed | borrowed | SEND_QUEUE | schema | txlevel | autocommit |
+------------+------+---------+-----------------+------+--------+--------+---------+------+--------+----------+------------+--------+---------+------------+
| Processor0 | 12 | 33 | 127.0.0.1 | 3306 | 42505 | 596 | 501 | 3384 | FALSE | FALSE | 0
....... 17 ROWS IN SET (0.03 sec) mysql> SHOW @@connection;
+------------+------+-----------+------+------------+--------+---------+--------+---------+---------------+-------------+------------+---------+------------+
| PROCESSOR | ID | HOST | PORT | LOCAL_PORT | SCHEMA | CHARSET | NET_IN | NET_OUT | ALIVE_TIME(S) | RECV_BUFFER | SEND_QUEUE | txlevel | autocommit |
+------------+------+-----------+------+------------+--------+---------+--------+---------+---------------+-------------+------------+---------+------------+
| Processor0 | 2 | 127.0.0.1 | 9066 | 35952 | NULL | utf8 | 156 | 4107 | 40 | 4096 | 0 | NULL | NULL |
| Processor0 | 1 | 127.0.0.1 | 8066 | 16525 | TESTDB | utf8 | 2005 | 5132 | 3461 | 4096 | 0 | 3 | TRUE |
+------------+------+-----------+------+------------+--------+---------+--------+---------+---------------+-------------+------------+---------+------------+
2 ROWS IN SET (0.02 sec) mysql> SHOW @@heartbeat;
+--------+-------+-----------------+------+---------+-------+--------+---------+--------------+---------------------+-------+
| NAME | TYPE | HOST | PORT | RS_CODE | RETRY | STATUS | TIMEOUT | EXECUTE_TIME | LAST_ACTIVE_TIME | STOP |
+--------+-------+-----------------+------+---------+-------+--------+---------+--------------+---------------------+-------+
| hostM1 | mysql | 127.0.0.1 | 3306 | 1 | 0 | idle | 30000 | 0,0,0 | 2015-03-03 14:45:00 | FALSE |
+--------+-------+-----------------+------+---------+-------+--------+---------+--------------+---------------------+-------+
1 ROW IN SET (0.01 sec) mysql> SHOW @@datanode;
+------+----------------+-------+-------+--------+------+------+---------+------------+----------+---------+---------------+
| NAME | DATHOST | INDEX | TYPE | ACTIVE | IDLE | SIZE | EXECUTE | TOTAL_TIME | MAX_TIME | MAX_SQL | RECOVERY_TIME |
+------+----------------+-------+-------+--------+------+------+---------+------------+----------+---------+---------------+
| dn1 | localhost1/db1 | 0 | mysql | 0 | 9 | 50 | 27 | 0 | 0 | 0 | -1 |
| dn2 | localhost1/db2 | 0 | mysql | 0 | 4 | 50 | 26 | 0 | 0 | 0 | -1 |
| dn3 | localhost1/db3 | 0 | mysql | 0 | 3 | 50 | 17 | 0 | 0 | 0 | -1 |
+------+----------------+-------+-------+--------+------+------+---------+------------+----------+---------+---------------+
3 ROWS IN SET (0.01 sec) mysql> SHOW @@cache;
+-------------------------------------+-------+------+--------+------+------+---------------+---------------+
| CACHE | MAX | CUR | ACCESS | HIT | PUT | LAST_ACCESS | LAST_PUT |
+-------------------------------------+-------+------+--------+------+------+---------------+---------------+
| SQLRouteCache | 10000 | 1 | 8 | 0 | 1 | 1425364473193 | 1425363891040 |
| TableID2DataNodeCache.TESTDB_ORDERS | 50000 | 0 | 0 | 0 | 0 | 0 | 0 |
| ER_SQL2PARENTID | 1000 | 0 | 0 | 0 | 0 | 0 | 0 |
+-------------------------------------+-------+------+--------+------+------+---------------+---------------+
3 ROWS IN SET (0.05 sec)

4,总结

初步安装测试了下mycat组件,还是非常不错的,确实比起cobar有不少的进步。 
同时他们开发团队也说了,解决了cobar之前存在的问题。 
有了全局表,这样可以使用join了,同时增加读写分离。 
规则增加了不少,能满足大部分需求。 
和cobar一样,mycat可以直接伪装成一个mysql服务器,对业务进行垂直拆分,水平拆分。平滑的进行数据扩展。 
保证在原有系统上进行优化。接下来继续研究mycat,非常好的解决方案。

数据库垂直拆分,水平拆分利器,cobar升级版mycat(转)的更多相关文章

  1. 用Spider引擎解决数据库垂直和水平拆分的问题

    作者介绍 张秀云,网名飞鸿无痕,现任职于腾讯,负责腾讯金融数据库的运维和优化工作.2007年开始从事运维方面的工作,经历过网络管理员.Linux运维工程师.DBA.分布式存储运维等多个IT职位.对Li ...

  2. mysql关于数据库表的水平拆分和垂直拆分

    最初知道水平垂直分表的时候是刚参加工作不久的时候,知道了这个概念,但是公司用户量和数据量始终没上来,所以也没用到过,知道有一天到了一家新公司后,这些才被应用到实际开发中,这里我就大概说说关于水平和垂直 ...

  3. 分布式数据库中间件TDDL、Amoeba、Cobar、MyCAT架构比较分

    比较了业界流行的MySQL分布式数据库中间件,关于每个产品的介绍,网上的资料比较多,本文只是对几款产品的架构进行比较,从中可以看出中间件发展和演进路线 框架比较 TDDL Amoeba Cobar M ...

  4. MYSQL水平拆分与垂直拆分

    目前很多互联网系统都存在单表数据量过大的问题,这就降低了查询速度,影响了客户体验.为了提高查询速度,我们可以优化sql语句,优化表结构和索引,不过对那些百万级千万级的数据库表,即便是优化过后,查询速度 ...

  5. mysql水平拆分与垂直拆分的详细介绍(转载http://www.cnblogs.com/nixi8/p/4524082.html)

      垂直 垂直拆分是指数据表列的拆分,把一张列比较多的表拆分为多张表 通常我们按以下原则进行垂直拆分: 把不常用的字段单独放在一张表; 把text,blob等大字段拆分出来放在附表中; 经常组合查询的 ...

  6. 【mysql的设计与优化专题(4)】表的垂直拆分和水平拆分

    垂直拆分 垂直拆分是指数据表列的拆分,把一张列比较多的表拆分为多张表 通常我们按以下原则进行垂直拆分: 把不常用的字段单独放在一张表; 把text,blob等大字段拆分出来放在附表中; 经常组合查询的 ...

  7. 表的垂直拆分和水平拆分-zz

    https://www.kancloud.cn/thinkphp/mysql-design-optimalize/39326 http://www.cnblogs.com/nixi8/tag/mysq ...

  8. Pycharm使用技巧:Split Vertically/Horizontally(垂直/水平拆分窗口)

    Split Vertically或者Split Horizontally可以把当前编辑窗口垂直或者水平拆分成两个. 使用: 在编辑窗口中打开你要展示的两个文件(如图中的  "郭靖" ...

  9. 数据库水平拆分和垂直拆分区别(以mysql为例)

    数据库水平拆分和垂直拆分区别(以mysql为例) 数据库水平拆分和垂直拆分区别(以mysql为例)   案例:     简单购物系统暂设涉及如下表: 1.产品表(数据量10w,稳定) 2.订单表(数据 ...

随机推荐

  1. 开源静态分析工具androguard体验

    原文链接:http://blog.csdn.net/xbalien29/article/details/21885297 虽然在windows端免费版的IDA.VTS等工具都可用来静态分析,但相对来说 ...

  2. sql中的笛卡尔积

    sql中的笛卡尔积 我们对数据库表进行操作时,经常会对多张表进行关联,多表连接查询大家肯定不会陌生,但是一不小心很容易出来庞大冗余的数据. 笛卡尔积数学概念 笛卡尔积是指在数学中,两个集合X和Y的笛卡 ...

  3. NIO之Charset类字符编码对象

    Charset类字符编码对象 介绍 java中使用Charset来表示编码对象 This class defines methods for creating decoders and encoder ...

  4. CSS3 :nth-child()伪类选择器

    CSS3 :nth-child()伪类选择器 CSS3的强大,让人惊叹,人们在惊喜之余,又不得不为其艰难的道路感到可惜:好 的标准只有得到行业浏览器的良好支持才算得上“标准”.CSS3标 准已提出数年 ...

  5. C语言中fgetc函数返回值为什么是int?

    学习C语言的,文件操作,大都会用到它. 它的函数原型: 这个函数的返回值,是返回读取的一个字节.如果读到文件末尾返回EOF.EOF其实就是一个宏#define EOF (-1)表示-1.既然返回的是一 ...

  6. Django学习之第三方储存服务器的使用

    最近,越来越多的公司采用第三方储存来作为视频,图片的储存工具. 国内的像七牛,阿里云的OSS,国外的像亚马逊的S3,微软的azure都是非常有名的第三方储存. 下面以阿里的OSS为例,来介绍第三储存的 ...

  7. win7安装RabbitMQ

    1.下载并安装erlang http://www.erlang.org/downloads 2.下载并安装RabbitMQ http://www.rabbitmq.com/install-window ...

  8. gulpfile.js(编译sass,压缩图片,自动刷新浏览器)

    var gulp = require('gulp'),     sass = require('gulp-sass'),     watch = require('gulp-watch'),      ...

  9. ../lib//libscsdblog.so: undefined reference to `pthread_atfork'

    代码中遇到这个问题,但是在makefile中已经添加了-lpthread. 最后发现问题时,引入库的顺序,把-lpthread放在最后就可以了.

  10. Visual Studio 调试小技巧-从查看窗口得到更多信息(转)

    原文地址:http://blog.csdn.net/cadcisdhht/article/details/5651488