mongodb的replication与shard分片结合使用详解

部署脚本

#!/bin/bash
#复制集配置
IP='10.0.0.12'  #主机ip
NA='rs3'  #复制集名称

if [ "$1" = "reset" ]
then
    kill -2 `ps -ef|grep -v grep |grep logpath|awk '{print $2}'`
    sleep 2
    rm -rf /mongodb/m{17,18,19,log}/*
    exit
fi

if [ "$1" = "install" ]
then
  mkdir -p /mongodb/m{17,18,19,log}/
  mongod --dbpath /mongodb/m17 --logpath /mongodb/mlog/m17.log --logappend --fork --port 27017 --replSet ${NA}  --smallfiles
  mongod --dbpath /mongodb/m18 --logpath /mongodb/mlog/m18.log --logappend --fork --port 27018 --replSet ${NA}  --smallfiles
  mongod --dbpath /mongodb/m19 --logpath /mongodb/mlog/m19.log --logappend --fork --port 27019 --replSet ${NA}  --smallfiles
   exit
 fi

 if [ "$1" = "repl" ]
then
mongo --port 27017 <<EOF

use admin;
var rsconf = {
_id:'${NA}',
members:[
{_id:0,host:'${IP}:27017'},
{_id:1,host:'${IP}:27018'},
{_id:2,host:'${IP}:27019'},
]
};
rs.initiate(rsconf);
<<
EOF
fi

部署情况

提前部署好两个复制集，部署脚本如上
现在部署情况如下
rs3:
  10.0.0.12  27017
  10.0.0.12  27018
  10.0.0.12  27019

rs4:
  10.0.0.13  27017
  10.0.0.13  27018
  10.0.0.13  27019

configsvr :
  10.0.0.11  27020

mongos：
  10.0.0.11  30000
  

部署configsvr和mongos

mongod --dbpath /mongodb/m20 --logpath /mongodb/mlog/m20.log --logappend --fork --port 27020 --configsvr
mongos --logpath /mongodb/mlog/m30.log --logappend --fork --port 30000 --configdb 10.0.0.11:27020 

[mongod@mcw01 ~]$ mongod --dbpath /mongodb/m20 --logpath /mongodb/mlog/m20.log --logappend --fork --port 27020 --configsvr
about to fork child process, waiting until server is ready for connections.
forked process: 20138
child process started successfully, parent exiting
[mongod@mcw01 ~]$ mongos --logpath /mongodb/mlog/m30.log --logappend --fork --port 30000 --configdb 10.0.0.11:27020
2022-03-05T15:18:39.286+0800 W SHARDING [main] Running a sharded cluster with fewer than 3 config servers should only be done for testing purposes and is not recommended for production.
about to fork child process, waiting until server is ready for connections.
forked process: 20158
child process started successfully, parent exiting
[mongod@mcw01 ~]$ 

添加分片

复制集和分片结合的。配置上没太多区别。就是在添加分片的时候，一个加上复制集名称，一个不加。一个复制集就是一个整体，当成分片的一个单元。复制集里的每个副本都是一个备份，其中写入的时候写到主副本就行了吧，复制集会实现其它备副本的复制。复制集中的副本是不是也可以设置在不同的主机上呢，这里设置的是同一台主机上，备份的安全性上考虑，肯定备份不要在同一个主机才安全。
sh.addShard('rs3/10.0.0.12:27017');

添加分片只需要添加复制集的主17就可以了，然后查看分片，可以看到主副本，以及其它副本的。
[mongod@mcw01 ~]$ mongo 10.0.0.11:30000
MongoDB shell version: 3.2.8
connecting to: 10.0.0.11:30000/test
mongos> sh.addShard('rs3/10.0.0.12:27017');
{ "shardAdded" : "rs3", "ok" : 1 }
mongos> sh.addShard('rs4/10.0.0.13:27017');
{ "shardAdded" : "rs4", "ok" : 1 }
mongos> sh.status();
--- Sharding Status ---
  sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("62230ecfe5cc911b886d1023")
}
  shards:  #写数据，分片上的数据是独有的，而同一个副本集不同副本中的数据是相同的，是个复制的数据。当数据分配存储到rs3分片时，18,19两个副本会从17主上复制一份过来作为备份用
    {  "_id" : "rs3",  "host" : "rs3/10.0.0.12:27017,10.0.0.12:27018,10.0.0.12:27019" }
    {  "_id" : "rs4",  "host" : "rs4/10.0.0.13:27017,10.0.0.13:27018,10.0.0.13:27019" }
  active mongoses:
    "3.2.8" : 1
  balancer:
    Currently enabled:  yes
    Currently running:  no
    Failed balancer rounds in last 5 attempts:  0
    Migration Results for the last 24 hours:
        No recent migrations
  databases:

mongos> 

设置分片库表，分片规则，插入数据，手动预分片

给shop开启分片，给shop下的user分片，根据userid分片。当userid是1000的时候切割一下，2000的时候切割一下......
mongos> show dbs;
config  0.000GB
mongos> sh.enableSharding('shop')
{ "ok" : 1 }
mongos> sh.shardCollection('shop.user',{userid:1})
{ "collectionsharded" : "shop.user", "ok" : 1 }
mongos> sh.splitAt('shop.user',{userid:10000})
{ "ok" : 1 }
mongos> sh.splitAt('shop.user',{userid:1000})
{ "ok" : 1 }
mongos> sh.splitAt('shop.user',{userid:2000})
{ "ok" : 1 }
mongos> sh.splitAt('shop.user',{userid:3000})
{ "ok" : 1 }
mongos> sh.splitAt('shop.user',{userid:4000})
{ "ok" : 1 }
mongos> 

查看已经是预分片了，并且res3复制集有，rs4复制集也有分片，各有三个，还是均衡的
mongos> sh.status();
--- Sharding Status ---
  sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("62230ecfe5cc911b886d1023")
}
  shards:
    {  "_id" : "rs3",  "host" : "rs3/10.0.0.12:27017,10.0.0.12:27018,10.0.0.12:27019" }
    {  "_id" : "rs4",  "host" : "rs4/10.0.0.13:27017,10.0.0.13:27018,10.0.0.13:27019" }
  active mongoses:
    "3.2.8" : 1
  balancer:
    Currently enabled:  yes
    Currently running:  no
    Failed balancer rounds in last 5 attempts:  0
    Migration Results for the last 24 hours:
        3 : Success
  databases:
    {  "_id" : "shop",  "primary" : "rs4",  "partitioned" : true }
        shop.user
            shard key: { "userid" : 1 }
            unique: false
            balancing: true
            chunks:
                rs3    3
                rs4    3
            { "userid" : { "$minKey" : 1 } } -->> { "userid" : 1000 } on : rs4 Timestamp(3, 0)
            { "userid" : 1000 } -->> { "userid" : 2000 } on : rs4 Timestamp(4, 0)
            { "userid" : 2000 } -->> { "userid" : 3000 } on : rs3 Timestamp(4, 1)
            { "userid" : 3000 } -->> { "userid" : 4000 } on : rs3 Timestamp(3, 4)
            { "userid" : 4000 } -->> { "userid" : 10000 } on : rs3 Timestamp(3, 5)
            { "userid" : 10000 } -->> { "userid" : { "$maxKey" : 1 } } on : rs4 Timestamp(2, 1) 

mongos>

我们在mongos上进入shop库,插入数据
mongos> use shop
switched to db shop
mongos> for(var i=1;i<=4000;i++){db.user.insert({userid:i,intro:'i am mcw, from china'})}
WriteResult({ "nInserted" : 1 })
mongos> 

查看插入数据的效果

首先看下rs3上的主副本：有两千到四千的数据
[mongod@mcw02 ~]$ mongo --port 27017
rs3:PRIMARY> show dbs;
local  0.000GB
shop   0.000GB
rs3:PRIMARY> use shop;
switched to db shop
rs3:PRIMARY> show tables;
user
rs3:PRIMARY> db.user.find().count();
2001
rs3:PRIMARY> db.user.find().skip(1999);
{ "_id" : ObjectId("622314a446d8d9b89fc9e710"), "userid" : 3999, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("622314a446d8d9b89fc9e711"), "userid" : 4000, "intro" : "i am mcw, from china" }
rs3:PRIMARY> db.user.find().limit(3);
{ "_id" : ObjectId("6223149d46d8d9b89fc9df41"), "userid" : 2000, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("6223149d46d8d9b89fc9df42"), "userid" : 2001, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("6223149d46d8d9b89fc9df43"), "userid" : 2002, "intro" : "i am mcw, from china" }
rs3:PRIMARY> 

看下rs3上的备副本：数据跟主副本时一致的
[mongod@mcw02 ~]$ mongo --port 27018
rs3:SECONDARY> use shop
switched to db shop
rs3:SECONDARY> db.user.find().count();
2001
rs3:SECONDARY> db.user.find().skip(1999);
{ "_id" : ObjectId("622314a446d8d9b89fc9e710"), "userid" : 3999, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("622314a446d8d9b89fc9e711"), "userid" : 4000, "intro" : "i am mcw, from china" }
rs3:SECONDARY> db.user.find().limit(3);
{ "_id" : ObjectId("6223149d46d8d9b89fc9df41"), "userid" : 2000, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("6223149d46d8d9b89fc9df42"), "userid" : 2001, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("6223149d46d8d9b89fc9df43"), "userid" : 2002, "intro" : "i am mcw, from china" }
rs3:SECONDARY> 

查看rs4上主副本：发现数据跟rs.status()是一样的。从1-1999都在rs4副本上
[mongod@mcw03 ~]$ mongo --port 27017
rs4:PRIMARY> use shop
switched to db shop
rs4:PRIMARY> db.user.find().count();
1999
rs4:PRIMARY> db.user.find().skip(1997);
{ "_id" : ObjectId("6223149d46d8d9b89fc9df3f"), "userid" : 1998, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("6223149d46d8d9b89fc9df40"), "userid" : 1999, "intro" : "i am mcw, from china" }
rs4:PRIMARY> db.user.find().limit(3);
{ "_id" : ObjectId("6223149446d8d9b89fc9d772"), "userid" : 1, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("6223149446d8d9b89fc9d773"), "userid" : 2, "intro" : "i am mcw, from china" }
{ "_id" : ObjectId("6223149446d8d9b89fc9d774"), "userid" : 3, "intro" : "i am mcw, from china" }
rs4:PRIMARY>