mongodb 3.4 学习 (四）分片
https://www.linode.com/docs/databases/mongodb/build-database-clusters-with-mongodb
由三部分组成
shard: 每个shard包括一个所有shard数据的子集. 每个shard能够部署成replica set
mongos: 做为查询路由, 提供clientt与sharded cluster的接口
config servers: 存储群集的metadata和配置，必须部署为replica set
# 生产环境部署
kvm-70-102 10.0.70.102 # mongos
kvm-70-103 10.0.70.103 # config
kvm-70-102 10.0.70.104 # shard1
kvm-70-103 10.0.70.105 # shard2
config
# 建立数据目录
mkdir -p /opt/mongo_{27021,27022,27023}
# 配置文件
cat > /etc/mongod_27021.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27021.log
storage:
dbPath: /opt/mongo_27021
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27021.pid
net:
port: 27021
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: config
sharding:
clusterRole: configsvr
EOF
# 配置文件
cat > /etc/mongod_27022.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27022.log
storage:
dbPath: /opt/mongo_27022
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27022.pid
net:
port: 27022
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: config
sharding:
clusterRole: configsvr
EOF
# 配置文件
cat > /etc/mongod_27023.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27023.log
storage:
dbPath: /opt/mongo_27023
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27023.pid
net:
port: 27023
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: config
sharding:
clusterRole: configsvr
EOF
# 启动复制集
mongod --config /etc/mongod_27021.conf
mongod --config /etc/mongod_27022.conf
mongod --config /etc/mongod_27023.conf
# 配置config
mongo --port 27021
rs.initiate(
{
_id: 'config',
configsvr: true,
members: [
{ _id : 0, host : '10.0.70.103:27021' },
{ _id : 1, host : '10.0.70.103:27022' },
{ _id : 2, host : '10.0.70.103:27023' }
]
}
)
{ "ok" : 1 }
shard1
# 建立数据目录
mkdir -p /opt/mongo_{27021,27022,27023}
# 配置文件
cat > /etc/mongod_27021.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27021.log
storage:
dbPath: /opt/mongo_27021
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27021.pid
net:
port: 27021
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard1
sharding:
clusterRole: shardsvr
EOF
# 配置文件
cat > /etc/mongod_27022.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27022.log
storage:
dbPath: /opt/mongo_27022
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27022.pid
net:
port: 27022
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard1
sharding:
clusterRole: shardsvr
EOF
# 配置文件
cat > /etc/mongod_27023.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27023.log
storage:
dbPath: /opt/mongo_27023
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27023.pid
net:
port: 27023
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard1
sharding:
clusterRole: shardsvr
EOF
# 启动复制集
mongod --config /etc/mongod_27021.conf
mongod --config /etc/mongod_27022.conf
mongod --config /etc/mongod_27023.conf
# 配置config
mongo --port 27021
rs.initiate(
{
_id: 'shard1',
members: [
{ _id : 0, host : '10.0.70.104:27021' },
{ _id : 1, host : '10.0.70.104:27022' },
{ _id : 2, host : '10.0.70.104:27023' }
]
}
)
{ "ok" : 1 }
shard2
# 建立数据目录
mkdir -p /opt/mongo_{27021,27022,27023}
# 配置文件
cat > /etc/mongod_27021.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27021.log
storage:
dbPath: /opt/mongo_27021
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27021.pid
net:
port: 27021
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard2
sharding:
clusterRole: shardsvr
EOF
# 配置文件
cat > /etc/mongod_27022.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27022.log
storage:
dbPath: /opt/mongo_27022
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27022.pid
net:
port: 27022
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard2
sharding:
clusterRole: shardsvr
EOF
# 配置文件
cat > /etc/mongod_27023.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27023.log
storage:
dbPath: /opt/mongo_27023
journal:
enabled: true
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27023.pid
net:
port: 27023
security:
keyFile: /etc/mongodb_product_key
replication:
oplogSizeMB: 2048
replSetName: shard2
sharding:
clusterRole: shardsvr
EOF
# 启动复制集
mongod --config /etc/mongod_27021.conf
mongod --config /etc/mongod_27022.conf
mongod --config /etc/mongod_27023.conf
# 配置config
mongo --port 27021
rs.initiate(
{
_id: 'shard2',
members: [
{ _id : 0, host : '10.0.70.105:27021' },
{ _id : 1, host : '10.0.70.105:27022' },
{ _id : 2, host : '10.0.70.105:27023' }
]
}
)
{ "ok" : 1 }
mongos
# 配置文件
cat > /etc/mongod_27021.conf << EOF
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod_27021.log
processManagement:
fork: true
pidFilePath: /var/run/mongodb/mongod_27021.pid
net:
port: 27021
security:
keyFile: /etc/mongodb_product_key
sharding:
configDB: config/10.0.70.103:27021,10.0.70.103:27022,10.0.70.103:27023
EOF
# 启动mongos
mongos --config /etc/mongod_27021.conf
# 配置
mongo --port 27021
# 添加分片
sh.addShard('shard1/10.0.70.104:27021')
sh.addShard('shard1/10.0.70.104:27022')
sh.addShard('shard1/10.0.70.104:27023')
sh.addShard('shard2/10.0.70.105:27021')
sh.addShard('shard2/10.0.70.105:27022')
sh.addShard('shard2/10.0.70.105:27023')
# 建立超级用户
use admin
db.createUser(
{
user: 'admin',
pwd: '@admin',
roles: [ { role: 'root', db: 'admin' } ]
}
);
# 认证登录
db.auth('admin', '@admin')
# app数据库demo表插入100000条数据
use app
for (var i = 1; i < 100000; i++) db.demo.insert({id: i, name: 'ken'})
WriteResult({ "nInserted" : 1 })
# 查看分片状态
sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("59261bca37ceff575b36ef09")
}
shards:
{ "_id" : "shard1", "host" : "shard1/10.0.70.104:27021,10.0.70.104:27022,10.0.70.104:27023", "state" : 1 }
{ "_id" : "shard2", "host" : "shard2/10.0.70.105:27021,10.0.70.105:27022,10.0.70.105:27023", "state" : 1 }
active mongoses:
"3.4.4" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Balancer lock taken at Thu May 25 2017 07:48:42 GMT+0800 (CST) by ConfigServer:Balancer
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
No recent migrations
databases:
{ "_id" : "app", "primary" : "shard1", "partitioned" : false }
# 结果显示没有分区，记录插在shard1服务器
# 配置分片
# 允许分片的数据库
sh.enableSharding('app')
# 为用做shard key的字段建立索引，实际中可以用uuid字段
use app
db.demo.createIndex({id: 1})
{
"raw" : {
"shard1/10.0.70.104:27021,10.0.70.104:27022,10.0.70.104:27023" : {
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 1,
"numIndexesAfter" : 2,
"ok" : 1,
"$gleStats" : {
"lastOpTime" : {
"ts" : Timestamp(1495675335, 1),
"t" : NumberLong(1)
},
"electionId" : ObjectId("7fffffff0000000000000001")
}
}
},
"ok" : 1
}
# 允许分片的collection，指定shard key的字段
sh.shardCollection('app.demo', { id : 'hashed' })
{ "collectionsharded" : "app.demo", "ok" : 1 }
# 插入数据
for (var i = 1; i < 10000; i++) db.demo.insert({id: i, name: 'ken'})
WriteResult({ "nInserted" : 1 })
# 查看状态，可以看到db与collection的分片状态
sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("59261bca37ceff575b36ef09")
}
shards:
{ "_id" : "shard1", "host" : "shard1/10.0.70.104:27021,10.0.70.104:27022,10.0.70.104:27023", "state" : 1 }
{ "_id" : "shard2", "host" : "shard2/10.0.70.105:27021,10.0.70.105:27022,10.0.70.105:27023", "state" : 1 }
active mongoses:
"3.4.4" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Balancer lock taken at Thu May 25 2017 07:48:42 GMT+0800 (CST) by ConfigServer:Balancer
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
1 : Success
databases:
{ "_id" : "app", "primary" : "shard1", "partitioned" : true }
app.demo
shard key: { "id" : "hashed" }
unique: false
balancing: true
chunks:
shard1 2
shard2 2
{ "id" : { "$minKey" : 1 } } -->> { "id" : NumberLong("-4611686018427387902") } on : shard1 Timestamp(2, 2)
{ "id" : NumberLong("-4611686018427387902") } -->> { "id" : NumberLong(0) } on : shard1 Timestamp(2, 3)
{ "id" : NumberLong(0) } -->> { "id" : NumberLong("4611686018427387902") } on : shard2 Timestamp(2, 4)
{ "id" : NumberLong("4611686018427387902") } -->> { "id" : { "$maxKey" : 1 } } on : shard2 Timestamp(2, 5)
# shard1
db.demo.find().count()
4992
# shard2
db.demo.find().count()
5007
如果在允许colletion前demo有数据，执行以下命令时会报错。
sh.shardCollection('app.demo', { id : 'hashed' })
{
"proposedKey" : {
"id" : "hashed"
},
"curIndexes" : [
{
"v" : 2,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "app.demo"
},
{
"v" : 2,
"key" : {
"id" : 1
},
"name" : "id_1",
"ns" : "app.demo"
}
],
"ok" : 0,
"errmsg" : "please create an index that starts with the shard key before sharding."
}
做法是将collection删除，重建索引。
chunksize默认是64M，如果分片过慢的话，可以适当调大，如512M。mongos的配置文件里。
Hashed Sharding
Ranged Sharding
mongodb 3.4 学习 (四）分片的更多相关文章
- 搭建高可用mongodb集群（四）—— 分片
按照上一节中<搭建高可用mongodb集群(三)—— 深入副本集>搭建后还有两个问题没有解决: 从节点每个上面的数据都是对数据库全量拷贝,从节点压力会不会过大? 数据压力大到机器支撑不了的 ...
- 搭建高可用mongodb集群（四）—— 分片
按照上一节中<搭建高可用mongodb集群(三)-- 深入副本集>搭建后还有两个问题没有解决: 从节点每个上面的数据都是对数据库全量拷贝,从节点压力会不会过大? 数据压力大到机器支撑不了的 ...
- MongoDB的使用学习之（七）MongoDB的聚合查询（两种方式）附项目源码
先来张在路上…… 铛铛铛……项目源码下载地址:http://files.cnblogs.com/ontheroad_lee/MongoDBDemo.rar 此项目是用Maven创建的,没有使用Mave ...
- TweenMax动画库学习(四)
目录 TweenMax动画库学习(一) TweenMax动画库学习(二) TweenMax动画库学习(三) Tw ...
- SVG 学习<四> 基础API
目录 SVG 学习<一>基础图形及线段 SVG 学习<二>进阶 SVG世界,视野,视窗 stroke属性 svg分组 SVG 学习<三>渐变 SVG 学习<四 ...
- Android JNI学习(四)——JNI的常用方法的中文API
本系列文章如下: Android JNI(一)——NDK与JNI基础 Android JNI学习(二)——实战JNI之“hello world” Android JNI学习(三)——Java与Nati ...
- mongodb 搭建集群（分片+副本集）
mongodb 搭建集群(分片+副本集) 一.搭建结构图: 二.搭建步骤:
- SCARA——OpenGL入门学习四（颜色）
OpenGL入门学习[四] 本次学习的是颜色的选择.终于要走出黑白的世界了~~ OpenGL支持两种颜色模式:一种是RGBA,一种是颜色索引模式. 无论哪种颜色模式,计算机都必须为每一个像素保存一些数 ...
- mongodb常用命令学习笔记
mongodb常用命令学习笔记 创建数据库 use DATABASE_NAME eg: use users; 如果数据库不存在,则创建数据库,否则切换到指定数据库.要显示刚刚创建的数据库,需要向数据库 ...
随机推荐
- HihoCoder - 1142 三分法练手
中文题面,原函数为三峰函数,先折半再三分 #include<bits/stdc++.h> using namespace std; const int maxn = 1e5+11; con ...
- Action执行时间过滤器
public class AccessStatisticsAttribute : ActionFilterAttribute { /// <summary> /// log4net 日志 ...
- 【研究】缓慢的http拒绝服务攻击
1 详细描述: 缓慢的http拒绝服务攻击是一种专门针对于Web的应用层拒绝服务攻击,攻击者操纵网络上的肉鸡,对目标Web服务器进行海量http request攻击,直到服务器带宽被打满,造成了拒绝服 ...
- DOC窗口之cd命令（windows）
cd的全称是Change Directory,改变文件夹,也就是切换路径.后面可以接驱动器符号.完整路径和相对路径. 通常win+R,输入cmd,便会看到以下界面, 通常,这个命令下并没有我们想要处理 ...
- google chrome 高版本 解除禁止添加本地扩展
1.下载chrome.adb文件,找不到的可以到https://pan.baidu.com/s/1j67vz_b6CwKfCH_uZzSfgA 这里下载2.Win+R 输入 gpedit.msc 3 ...
- windows 系统C盘暴增
系统: Windows Server 2012R2 问题: C盘200G的空间,暴增剩余3G.但是查看仅有69G的空间占用. 根源:Windows的虚拟文件,驱动器分页. 解决方案: 我的电脑-> ...
- Java 继承初探
Java继承的基础 Java中,被继承的类叫做超类,继承超类的类叫子类.(一个子类亦可以是另一个类的超类) 继承一个类,只需要用关键字 extends 把一个类的定义合并到另一个类中就可以了. 例子中 ...
- Java中多线程并发体系知识点汇总
一.多线程 1.操作系统有两个容易混淆的概念,进程和线程. 进程:一个计算机程序的运行实例,包含了需要执行的指令:有自己的独立地址空间,包含程序内容和数据:不同进程的地址空间是互相隔离的:进程拥有各种 ...
- MAYA 卸载工具，完美彻底清除干净maya各种残留注册表和文件
是不是遇到MAYA/CAD/3DSMAX/INVENTOR安装失败?AUTODESK系列软件着实令人头疼,MAYA/CAD/3DSMAX/INVENTOR安装失败之后不能完全卸载!!!(比如maya, ...
- Lua初探
官方网站:http://www.lua.org/ 三方模块:https://luarocks.org/ 介绍安装luarocks语法注释标示符关键字全局变量数据类型变量赋值索引循环流程控制函数可变 ...