[置顶] MongoDB 分布式操作——分片操作
MongoDB 分布式操作——分片操作
描述：
像其它分布式数据库一样，MongoDB同样支持分布式操作，且MongoDB将分布式已经集成到数据库中，其分布式体系如下图所示：
所谓的片，其实就是一个单独的MongoDB进程，它可以运行在不同的节点上，也可以运行在同一节点但监听不同的端口。同一个集合的数据可能在每个片中都存储的有数据，但它们在逻辑上整合成一个整体。分片的依据往往是根据表中的某一列或者是集合中的某一 组属性。且分片是系统自动的，但是分片的依据需要用户自己指定，分片的依据称之为“片键”。常见的片键选择为“随机片键”或者是“递增片键”。如果使用递增片键，数据的分布则会按照一定的顺序来写入数据，即当一个节点写满才会写入到下一个片键。而如果使用随机片键，数据则可能在任何一个片中都有数据。
对于分布式中片的管理（片与片之间的通信），MongoDB中通过mongos进程来进行管理，客户端对数据的操作，都由mongos进程来处理。对于节点、分片的管理，MongoDB由配置服务器来管理，配置服务器其实也是一个单独MongoDB进程。
那么，一般在什么时候使用分片呢？在单个节点的磁盘不足时会分片，另外，当单个mongod不能满足写数据的性能要求时，可以使用分片；当单台服务器内存太小，将大量的数据放到内存中速度会很慢，为了提高性能，我们可以使用分片操作。
以下是对于MongoDB分片操作的实验：
操作环境：
Win8 64位操作系统，虚拟机CentOS 32位操作系统。MongoDB 2.4.5
实验步骤：
（1）启动配置服务器（放置一些元数据）：
[root@h3 dbs]# mongod -dbpath /var/lib/mongo/dbs/config --nojournal --port 20000
Mon Aug 12 13:36:25.999 [initandlisten] MongoDB starting : pid=22951 port=20000 dbpath=/var/lib/mongo/dbs/config 32-bit host=h3
Mon Aug 12 13:36:26.001 [initandlisten]
Mon Aug 12 13:36:26.002 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary .
Mon Aug 12 13:36:26.003 [initandlisten] ** 32 bit builds are limited to le ss than 2GB of data (or less with --journal).
Mon Aug 12 13:36:26.004 [initandlisten] ** Note that journaling defaults t o off for 32 bit and is currently off.
Mon Aug 12 13:36:26.005 [initandlisten] ** See http://dochub.mongodb.org/c ore/32bit
Mon Aug 12 13:36:26.005 [initandlisten]
Mon Aug 12 13:36:26.006 [initandlisten] db version v2.4.5
Mon Aug 12 13:36:26.007 [initandlisten] git version: a2ddc68ba7c9cee17bfe69ed840 383ec3506602b
Mon Aug 12 13:36:26.007 [initandlisten] build info: Linux bs-linux32.10gen.cc 2. 6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49
Mon Aug 12 13:36:26.008 [initandlisten] allocator: system
Mon Aug 12 13:36:26.009 [initandlisten] options: { dbpath: "/var/lib/mongo/dbs/c onfig", nojournal: true, port: 20000 }
Mon Aug 12 13:36:26.071 [FileAllocator] allocating new datafile /var/lib/mongo/d bs/config/local.ns, filling with zeroes...
Mon Aug 12 13:36:26.072 [FileAllocator] creating directory /var/lib/mongo/dbs/co nfig/_tmp
Mon Aug 12 13:36:26.582 [FileAllocator] done allocating datafile /var/lib/mongo/ dbs/config/local.ns, size: 16MB, took 0.504 secs
Mon Aug 12 13:36:26.585 [FileAllocator] allocating new datafile /var/lib/mongo/d bs/config/local.0, filling with zeroes...
Mon Aug 12 13:36:27.725 [FileAllocator] done allocating datafile /var/lib/mongo/ dbs/config/local.0, size: 16MB, took 1.138 secs
Mon Aug 12 13:36:27.757 [initandlisten] command local.$cmd command: { create: "s tartup_log", size: 10485760, capped: true } ntoreturn:1 keyUpdates:0 reslen:37 1714ms
Mon Aug 12 13:36:27.764 [websvr] admin web console waiting for connections on po rt 21000
Mon Aug 12 13:36:27.771 [initandlisten] waiting for connections on port 20000
（2）启动mongos：
[root@h3 ~]# mongos --port 30000 --configdb localhost:20000
Mon Aug 12 13:40:13.042 warning: running with 1 config server should be done only for testing purposes and is not recommended for production
Mon Aug 12 13:40:13.075 [mongosMain] MongoS version 2.4.5 starting: pid=22968 port=30000 32-bit host=h3 (--help for usage)
Mon Aug 12 13:40:13.076 [mongosMain] git version: a2ddc68ba7c9cee17bfe69ed840383ec3506602b
Mon Aug 12 13:40:13.076 [mongosMain] build info: Linux bs-linux32.10gen.cc 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49
Mon Aug 12 13:40:13.076 [mongosMain] options: { configdb: "localhost:20000", port: 30000 }
Mon Aug 12 13:40:13.250 [LockPinger] creating distributed lock ping thread for localhost:20000 and process h3:30000:1376286013:1804289383 (sleeping for 30000ms)
Mon Aug 12 13:40:17.105 [mongosMain] distributed lock 'configUpgrade/h3:30000:1376286013:1804289383' acquired, ts : 52087541ff5991dd77979f2c
Mon Aug 12 13:40:17.126 [mongosMain] starting upgrade of config server from v0 to v4
Mon Aug 12 13:40:17.128 [mongosMain] starting next upgrade step from v0 to v4
Mon Aug 12 13:40:17.130 [mongosMain] about to log new metadata event: { _id: "h3-2013-08-12T05:40:17-52087541ff5991dd77979f2d", server: "h3", clientAddr: "N/A", time: new Date(1376286017130), what: "starting upgrade of config database", ns: "config.version", details: { from: 0, to: 4 } }
Mon Aug 12 13:40:17.138 [mongosMain] writing initial config version at v4
Mon Aug 12 13:40:17.232 [mongosMain] about to log new metadata event: { _id: "h3-2013-08-12T05:40:17-52087541ff5991dd77979f2f", server: "h3", clientAddr: "N/A", time: new Date(1376286017232), what: "finished upgrade of config database", ns: "config.version", details: { from: 0, to: 4 } }
Mon Aug 12 13:40:17.237 [mongosMain] upgrade of config server to v4 successful
Mon Aug 12 13:40:17.244 [mongosMain] distributed lock 'configUpgrade/h3:30000:1376286013:1804289383' unlocked.
Mon Aug 12 13:40:17.474 [websvr] admin web console waiting for connections on port 31000
Mon Aug 12 13:40:17.476 [Balancer] about to contact config servers and shards
Mon Aug 12 13:40:17.492 [Balancer] config servers and shards contacted successfully
Mon Aug 12 13:40:17.494 [Balancer] balancer id: h3:30000 started at Aug 12 13:40:17
Mon Aug 12 13:40:17.527 [mongosMain] waiting for connections on port 30000
（3）添加分片1：
[root@h3 ~]# cd /var/lib/mongo/dbs/
[root@h3 dbs]# mkdir shard1
[root@h3 dbs]# mongod --dbpath /var/lib/mongo/dbs/shard1 --port 10000 --nojournal
Mon Aug 12 13:43:10.560 [initandlisten] MongoDB starting : pid=23016 port=10000
dbpath=/var/lib/mongo/dbs/shard1 32-bit host=h3
Mon Aug 12 13:43:10.562 [initandlisten]
Mon Aug 12 13:43:10.562 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary.
Mon Aug 12 13:43:10.563 [initandlisten] ** 32 bit builds are limited to less than 2GB of data (or less with --journal).
Mon Aug 12 13:43:10.564 [initandlisten] ** Note that journaling defaults to off for 32 bit and is currently off.
Mon Aug 12 13:43:10.565 [initandlisten] ** See http://dochub.mongodb.org/core/32bit
Mon Aug 12 13:43:10.566 [initandlisten]
Mon Aug 12 13:43:10.566 [initandlisten] db version v2.4.5
Mon Aug 12 13:43:10.567 [initandlisten] git version: a2ddc68ba7c9cee17bfe69ed840383ec3506602b
Mon Aug 12 13:43:10.568 [initandlisten] build info: Linux bs-linux32.10gen.cc 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49
Mon Aug 12 13:43:10.568 [initandlisten] allocator: system
Mon Aug 12 13:43:10.569 [initandlisten] options: { dbpath: "/var/lib/mongo/dbs/shard1", nojournal: true, port: 10000 }
Mon Aug 12 13:43:10.626 [FileAllocator] allocating new datafile /var/lib/mongo/dbs/shard1/local.ns, filling with zeroes...
Mon Aug 12 13:43:10.628 [FileAllocator] creating directory /var/lib/mongo/dbs/shard1/_tmp
Mon Aug 12 13:43:11.547 [FileAllocator] done allocating datafile /var/lib/mongo/dbs/shard1/local.ns, size: 16MB, took 0.907 secs
Mon Aug 12 13:43:11.549 [FileAllocator] allocating new datafile /var/lib/mongo/dbs/shard1/local.0, filling with zeroes...
Mon Aug 12 13:43:13.339 [FileAllocator] done allocating datafile /var/lib/mongo/dbs/shard1/local.0, size: 16MB, took 1.787 secs
Mon Aug 12 13:43:13.427 [initandlisten] command local.$cmd command: { create: "startup_log", size: 10485760, capped: true } ntoreturn:1 keyUpdates:0 reslen:372801ms
Mon Aug 12 13:43:13.472 [initandlisten] waiting for connections on port 10000
Mon Aug 12 13:43:13.497 [websvr] admin web console waiting for connections on port 11000
（4）添加分片2：
[root@h3 dbs]# cd /var/lib/mongo/dbs/
[root@h3 dbs]# mkdir shard2
[root@h3 dbs]# mongod --dbpath /var/lib/mongo/dbs/shard2 --port 10001 --nojournal
Mon Aug 12 13:48:25.549 [initandlisten] MongoDB starting : pid=23129 port=10001 dbpath=/var/lib/mongo/dbs/shard2 32-bit host=h3
Mon Aug 12 13:48:25.550 [initandlisten]
Mon Aug 12 13:48:25.550 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary.
Mon Aug 12 13:48:25.550 [initandlisten] ** 32 bit builds are limited to less than 2GB of data (or less with --journal).
Mon Aug 12 13:48:25.550 [initandlisten] ** Note that journaling defaults to off for 32 bit and is currently off.
Mon Aug 12 13:48:25.550 [initandlisten] ** See http://dochub.mongodb.org/core/32bit
Mon Aug 12 13:48:25.551 [initandlisten]
Mon Aug 12 13:48:25.551 [initandlisten] db version v2.4.5
Mon Aug 12 13:48:25.552 [initandlisten] git version: a2ddc68ba7c9cee17bfe69ed840383ec3506602b
Mon Aug 12 13:48:25.552 [initandlisten] build info: Linux bs-linux32.10gen.cc 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49
Mon Aug 12 13:48:25.552 [initandlisten] allocator: system
Mon Aug 12 13:48:25.552 [initandlisten] options: { dbpath: "/var/lib/mongo/dbs/shard2", nojournal: true, port: 10001 }
Mon Aug 12 13:48:25.570 [FileAllocator] allocating new datafile /var/lib/mongo/dbs/shard2/local.ns, filling with zeroes...
Mon Aug 12 13:48:25.572 [FileAllocator] creating directory /var/lib/mongo/dbs/shard2/_tmp
Mon Aug 12 13:48:26.956 [FileAllocator] done allocating datafile /var/lib/mongo/dbs/shard2/local.ns, size: 16MB, took 1.378 secs
Mon Aug 12 13:48:26.957 [FileAllocator] allocating new datafile /var/lib/mongo/dbs/shard2/local.0, filling with zeroes...
Mon Aug 12 13:48:28.023 [FileAllocator] done allocating datafile /var/lib/mongo/dbs/shard2/local.0, size: 16MB, took 1.065 secs
Mon Aug 12 13:48:28.038 [initandlisten] command local.$cmd command: { create: "startup_log", size: 10485760, capped: true } ntoreturn:1 keyUpdates:0 reslen:37 2469ms
Mon Aug 12 13:48:28.041 [websvr] admin web console waiting for connections on port 11001
Mon Aug 12 13:48:28.042 [initandlisten] waiting for connections on port 10001
（5）添加分片3：
[root@h3 ~]# cd /var/lib/mongo/dbs/
[root@h3 dbs]# mkdir shard3
[root@h3 dbs]# mongod --dbpath /var/lib/mongo/dbs/shard3 --port 10002 --nojournal
Mon Aug 12 13:50:43.735 [initandlisten] MongoDB starting : pid=23142 port=10002 dbpath=/var/lib/mongo/dbs/shard3 32-bit host=h3
Mon Aug 12 13:50:43.737 [initandlisten]
Mon Aug 12 13:50:43.737 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary.
Mon Aug 12 13:50:43.738 [initandlisten] ** 32 bit builds are limited to less than 2GB of data (or less with --journal).
Mon Aug 12 13:50:43.739 [initandlisten] ** Note that journaling defaults to off for 32 bit and is currently off.
Mon Aug 12 13:50:43.739 [initandlisten] ** See http://dochub.mongodb.org/core/32bit
Mon Aug 12 13:50:43.740 [initandlisten]
Mon Aug 12 13:50:43.740 [initandlisten] db version v2.4.5
Mon Aug 12 13:50:43.740 [initandlisten] git version: a2ddc68ba7c9cee17bfe69ed840383ec3506602b
Mon Aug 12 13:50:43.740 [initandlisten] build info: Linux bs-linux32.10gen.cc 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49
Mon Aug 12 13:50:43.740 [initandlisten] allocator: system
Mon Aug 12 13:50:43.741 [initandlisten] options: { dbpath: "/var/lib/mongo/dbs/shard3", nojournal: true, port: 10002 }
Mon Aug 12 13:50:43.756 [FileAllocator] allocating new datafile /var/lib/mongo/dbs/shard3/local.ns, filling with zeroes...
Mon Aug 12 13:50:43.756 [FileAllocator] creating directory /var/lib/mongo/dbs/shard3/_tmp
Mon Aug 12 13:50:45.079 [FileAllocator] done allocating datafile /var/lib/mongo/dbs/shard3/local.ns, size: 16MB, took 1.32 secs
Mon Aug 12 13:50:45.099 [FileAllocator] allocating new datafile /var/lib/mongo/dbs/shard3/local.0, filling with zeroes...
Mon Aug 12 13:50:45.948 [FileAllocator] done allocating datafile /var/lib/mongo/dbs/shard3/local.0, size: 16MB, took 0.848 secs
Mon Aug 12 13:50:45.957 [initandlisten] command local.$cmd command: { create: "startup_log", size: 10485760, capped: true } ntoreturn:1 keyUpdates:0 reslen:37 2202ms
Mon Aug 12 13:50:45.960 [initandlisten] waiting for connections on port 10002
Mon Aug 12 13:50:45.961 [websvr] admin web console waiting for connections on port 11002
（6）添加刚刚启动的三个片：
[root@h3 dbs]# mongo localhost:30000/admin
MongoDB shell version: 2.4.5
connecting to: localhost:30000/admin
mongos> db.runCommand({addshard : "localhost:10000", allowLocal : true});
{ "shardAdded" : "shard0000", "ok" : 1 }
mongos> db.runCommand({addshard : "localhost:10001", allowLocal : true});
{ "shardAdded" : "shard0001", "ok" : 1 }
mongos> db.runCommand({addshard : "localhost:10002", allowLocal : true});
{ "shardAdded" : "shard0002", "ok" : 1 }
mongos>
（7）打开数据库foo的分片功能：
mongos> db.runCommand({"enablesharding" : "foo"});
{ "ok" : 1 }
mongos>
（8）打开数据库foo中数据集合bar的分片功能：
mongos> db.runCommand({"shardcollection" : "foo.bar", "key" : {"_id" : 1}});
{ "collectionsharded" : "foo.bar", "ok" : 1 }
mongos>
（9）查看所有的分片：
mongos> use config
switched to db config
mongos> db.shards.find();
{ "_id" : "shard0000", "host" : "localhost:10000" }
{ "_id" : "shard0001", "host" : "localhost:10001" }
{ "_id" : "shard0002", "host" : "localhost:10002" }
mongos>
（10）查看databases集合：
mongos> db.databases.find();
{ "_id" : "admin", "partitioned" : false, "primary" : "config" }
{ "_id" : "foo", "partitioned" : true, "primary" : "shard0000" }
mongos>
（11） 查看chucks集合：
mongos> db.chunks.find();
{ "_id" : "foo.bar-_id_MinKey", "lastmod" : Timestamp(1, 0), "lastmodEpoch" : ObjectId("52087d0a1072e2912d4242ea"), "ns" : "foo.bar", "min" : { "_id" : { "$minKey" : 1 } }, "max" : { "_id" : { "$maxKey" : 1 } }, "shard" : "shard0000" }
mongos>
（12）获得概要信息：
mongos> db.printShardingStatus();
--- Sharding Status ---
sharding version: {
"_id" : 1,
"version" : 3,
"minCompatibleVersion" : 3,
"currentVersion" : 4,
"clusterId" : ObjectId("52087bf41072e2912d4242b8")
}
shards:
{ "_id" : "shard0000", "host" : "localhost:10000" }
{ "_id" : "shard0001", "host" : "localhost:10001" }
{ "_id" : "shard0002", "host" : "localhost:10002" }
databases:
{ "_id" : "admin", "partitioned" : false, "primary" : "config" }
{ "_id" : "foo", "partitioned" : true, "primary" : "shard0000" }
foo.bar
shard key: { "_id" : 1 }
chunks:
shard0000 1
{ "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard0000 Timestamp(1, 0) mongos>
（13）移除分片：
mongos> use admin
switched to db admin
mongos> db.runCommand({"removeshard" : "localhost:10000"});
{
"msg" : "draining started successfully",
"state" : "started",
"shard" : "shard0000",
"note" : "you need to drop or movePrimary these databases",
"dbsToMove" : [
"foo"
],
"ok" : 1
}
mongos>
总结：对MongoDB的分片操作执行如下顺序：
（1）启动配置服务器；
（2）启动mongos进程；
（3）添加mongo实例（片）；
（4）对数据库或者数据集使用分片操作。
[置顶] MongoDB 分布式操作——分片操作的更多相关文章
- [置顶] SPL讲解(4)--Criteria操作篇
概念 以前一篇文章中,描述了实体Entity的操作,很明显,仅仅实体的操作是远远不够的.如:我们经常会根据查询条件从数据库中获取记录集并绑定到DataGrid上,会根据条件进行批量的Update和De ...
- [置顶] c#对于文件的操作
在system.iO:命名空间中提供了两种遍历枚举文件夹的类,Directory 和DirectoryInfo, 他们都有一个方法GetDirectories():来便利枚举文件夹, /// < ...
- [置顶] Android中使用sqlite3操作SQLite
SQLite库包含一个名字叫做sqlite3的命令行,它可以让用户手工输入并执行面向SQLite数据库的SQL命令.本文档提供一个样使用sqlite3的简要说明. 一.创建数据库: 1.将sqlit ...
- vue 通过自定义指令实现 置顶操作；
项目需求:要求当前项目每个页面滑到超出一屏的距离时,出现 backTop 按钮,点击则回到最顶端:俗称置顶操作: 因为涉及到的页面较多,每个页面都加肯定显得重复累赘,最终想到了 Vue 的自定义指令 ...
- jQuery操作table数据上移、下移和置顶
jQuery 操作table中的tr换行的步骤如下: 1.获取当前tr var $tr = $(this).parents("tr"); 2.移动tr //上移 $tr.prev( ...
- Mongodb 分片操作 介绍
为什么需要分片操作?由于数据量太大,使得CPU,内存,磁盘I/O等压力过大.当MongoDB存储海量的数据时,一台机器可能不足以存储数据,也可能不足以提供可接受的读写吞吐量.这时,我们就可以通过在多台 ...
- 006PHP文件处理—— 目录操作 删除目录 删除置顶类型文件
<?php /** * 目录操作 删除目录 删除置顶类型文件 */ //echo rmdir('61') or die('目录删除失败'); //删除一个目录中有其他文件的内容的方法: //第1 ...
- 表格中上移下移置顶的js操作
<script> $(function(){ //上移 var $up = $(".up") $up.click(function() { var $tr = ...
- Qt 窗口操作函数（置顶、全屏，最大化最小化按钮设置等）
一.窗口置顶 与 取消置顶 void MainWindow::on_windowTopButton_clicked() { if (m_flags == NULL) { m_flags = windo ...
随机推荐
- Duff 装置中case情况越多性能越好
猜想:Duff装置再循环里面,直接调用的方法越多(也就是case的数量比较多),性能相对越好 ???!!! 我们基于Duff装置来做进一步的测试. 然后分别添加两个新的函数,一个函数式 case有4种 ...
- sqlserver字符串拆分(split)方法汇总
--方法0:动态SQL法declare @s varchar(100),@sql varchar(1000)set @s='1,2,3,4,5,6,7,8,9,10'set @sql='select ...
- 原生js封装table表格操作，获取任意行列td，任意单行单列方法
V1.001更新增加findTable-min.js 本次更新,优化了代码性能方面,增加了部分新功能,可以获取多个table表格批量操作. 考虑到本人后面的项目中可能涉及到大量的表格操作,提前先封了 ...
- HDU2955-Robberies
描述: The aspiring Roy the Robber has seen a lot of American movies, and knows that the bad guys usual ...
- linux杂记（十）what is BASH Shell
first,what is shell?其实只要是碰过计算机的,对于OS(Operation System操作系统,不管是linux.unix.windows)有点概念的人大多都听过这个名词,因为只要 ...
- SnappyDB—Android上的NoSQL数据库简介
参考:http://www.open-open.com/lib/view/open1420816891937.html 参考:http://android-arsenal.com/details/1/ ...
- 风行一时瀑布流网页布局，实现无限加载（jquery）
今天跟大家分享一个瀑布流网页布局,先跟大家分析下实现的思路吧 主要思路:一.根据屏幕宽度和单个浮动层的宽度来确定浮动层列数 var $boxs = $("#main>div" ...
- C++中类成员函数作为回调函数
注:与tr1::function对象结合使用,能获得更好的效果,详情见http://blog.csdn.net/this_capslock/article/details/38564719 回调函数是 ...
- 让.net程序自动运行在管理员权限下
原文:让.net程序自动运行在管理员权限下 如何让.net程序自动运行在管理员权限下 VS2010 c# 编译的WINFORM程序 在Win7 以管理员身份运行 windows 7和vista提高的系 ...
- 利用 Android Studio 和 Gradle 打包多版本APK
在项目开发过程中,经常会有需要打包不同版本的 APK 的需求. 比如 debug版,release版,dev版等等. 有时候不同的版本中使用到的不同的服务端api域名也不相同. 比如 debug_ap ...