http://www.captaincodeman.com/2011/02/27/limit-mongodb-memory-use-windows/

Limit MongoDB memory use on Windows without Virtualization

I’ve seen the question of how to control MongoDB’s memory usage on Windows come up several times and the stock answer always seemed to be “you can’t – it uses memory-mapped files and if you want to limit resources you need to use some form of virtualization to do it (HyperV, VMWare, Virtuozzo etc…)”.

If you are using MongoDB on a dedicated server then you generally want it to use all the memory it can but if you want to use it on a server shared with other processes (e.g. an IIS website using MongoDB for storage, maybe with SQL Server as well) then you will want to put a cap on how much it uses to ensure memory is kept available for the other processes.

So is it possible if you are not on a virtualized environment? Yes (otherwise this would be a very short blog post!) and we’ll explore how …

The standard behaviour described above is actually a result of the default resource manager used by Windows but both Windows 2003 and Windows 2008 have a separate installable option called the “Windows System Resource Manager” (WSRM) that allows greater control over the CPU and Memory available to a process.

First of all, lets look at what we’re trying to solve. Here we have a low-memory server (only 2Gb) running MongoDB on Windows 2008 R2 x64. There are a few databases of a few Gb each so the mongod.exe process quickly starts consuming as much memory as it can (rightly so) to keep as much of it’s indexes in memory for the fast performance we know and love:

What we’d like to do is save some memory for other processes by limiting the mongod.exe process to 1Gb in this case (I know this is ridiculously low but the only thing that will change for you are the actual limits you want to use).

To do this we first need to install Windows System Resource Manager which on Windows 2008 is available under the Features section of the Server Manager.

Once that’s installed fire it up and you’ll see the default resource management policies. By default the standard Windows “memory is given to whoever shouts loudest” policy is used but other pre-configured alternatives are available. WSRM also provides a calendar / event system where the policy can be changed at certain times (a typical scenario is giving critical business apps priority during the day but then batch processes greater priority overnight). We’re not going to go into the calendar features here but it’s interesting to know about.

Let’s create a new policy to control the resources that MongoDB can consume. To do this, right click on the “Resource Allocation Policies” container and chose “New Resource Allocation Policy …”. This will present us with the New Resource Allocation Policy dialog below:

First of all, we need to add a new resource allocation entry so click the ‘Add…’ button and we get to another “Add or Edit Resource Allocation” dialog:

We don’t have a Process matching criteria for MongoDB yet so choose <New…> to get … yes, you guessed – another dialog, this time “New Process Matching Criteria”. We’ll call it “mongod_process” and click the Add… button to get another death-by-dialog to define it.

There are a few ways to do this – if MongoDB is installed as a service then you can choose “Registered Service” in the drop-down, click “Select” and choose it from the list or you can select from a list of running processes or you can just enter the full path and filename to mongod.exe. Here is the entry after selecting an installed MongoDB Windows Service:

After clicking OK we get back to the Process Matching Criteria dialog showing our new rule:

After clicking OK we’re now back at the Resource Allocation dialog with the new “mongod_process” Process matching criteria selected and can now decide what resources we want to allocate to the process. Lets limit the CPU to 50% (not that MongoDB seems to consume much CPU):

The Memory tab allows us to limit the memory and here there are two options. The maximum committed memory limit is more to control apps that may have a memory leak and can be setup to stop or alert someone when the process goes above the configured limit. We don’t want this one … instead we’ll set a maximum working set limit which will control how much memory is allocated to MongoDB. In this case, we’ll set the limit to 1Gb but the actual value to use will depend on your circumstances:

After clicking OK we should then be at the Resource Allocation Policy dialog with our process matching criteria, CPU and memory limits shown. We could include more limits in the policy but we’ll leave it as it is for now – any remaining resources will be allocated to other processes as normal after the limits have been imposed.

The final piece is to make this policy active which is done by clicking on the “Selected Policy” link on the main ‘page’ or right-clicking on the new entry under the “Resource Allocation Policies” and choosing “Set as Managing Policy”. You can also right-click on the “Windows System Resource Manager (local)” entry and choose “Properties …” to display the dialog below which allows you to select the Current resource allocation policy:

So, we’ve created a new policy that has a criteria to match the mongod.exe process which will limit the CPU usage to 50% and memory to 1Gb … does it work? Here’s the result after it’s enabled showing the memory used immediately dropping:

… and the MongoDB / mongod.exe process using the 1Gb limit we specified (1Gb = 1024Mb = 1,048,576Kb).

So, we’ve successfully limited the CPU and memory that MongoDB can consume without having to resort to any form of server-virtualization and while MongoDB will probably not run as fast as it did when it had free-reign to consume as much as it wanted (or rather, when the default windows resource manager gave it what it asked for) we will probably have a faster overall system as our other processes are allocated the memory and CPU that they need for a better balanced system.

Please let me know what you think of the above technique and if you find it useful.



27 February 2011

© 2015 Simon Green with help from Jekyll Bootstrap and Twitter Bootstrap

系统windows进程的资源分配的更多相关文章

  1. Windows进程崩溃问题定位方法

    Linux上进程崩溃通常会生成core文件,用gdb打开后执行bt命令即可查看堆栈.而在Windows平台上,我们通常会采用MiniDumpWriteDump来进行堆栈转储,而这需要对系统Api有一定 ...

  2. windows进程详解

    1:系统必要进程system process    进程文件: [system process] or [system process]进程名称: Windows内存处理系统进程描述: Windows ...

  3. windows进程/线程创建过程 --- windows操作系统学习

    有了之前的对进程和线程对象的学习的铺垫后,我们现在可以开始学习windows下的进程创建过程了,我将尝试着从源代码的层次来分析在windows下创建一个进程都要涉及到哪些步骤,都要涉及到哪些数据结构. ...

  4. Linux系统编程@进程通信(一)

    进程间通信概述 需要进程通信的原因: 数据传输 资源共享 通知事件 进程控制 Linux进程间通信(IPC)发展由来 Unix进程间通信 基于System V进程间通信(System V:UNIX系统 ...

  5. windows 进程间通讯方法

    Windows平台为我们提供了多种进程间通信的机制,主要包括:注册表方式.共享文件方式.共享内存方式.共享数据段.映射文件方式.管道方式. 剪贴板方式.消息方式.其中注册表方式需要增加注册表表项,而注 ...

  6. 走进windows编程的世界-----windows进程

    Windows进程  1 Windows进程    进程是一个容器,包括了一个应用程序实例的各种资源.Windows多任务的操作系统,因此能够同一时候运行多个进程.      2 Windows进程的 ...

  7. Windows进程单实例运行

    场景         Windows进程单实例运行,如果有进程没有退出,继续等待,直到进程完全退出,才会进入下一个实例 HANDLE pHandle = NULL; do  {  pHandle = ...

  8. linux进程cpu资源分配命令nice,renice,taskset

    进程cpu资源分配就是指进程的优先权(priority).优先权高的进程有优先执行权利.配置进程优先权对多任务环境的linux很有用,可以改善系统性能.还可以把进程运行到指定的CPU上,这样一来,把不 ...

  9. Windows进程间共享内存通信实例

    Windows进程间共享内存通信实例 抄抄补补整出来 采用内存映射文件实现WIN32进程间的通讯:Windows中的内存映射文件的机制为我们高效地操作文件提供了一种途径,它允许我们在WIN32进程中保 ...

随机推荐

  1. <孤独者生存(小小辛巴投资手记)>读书笔记

    书在这里 让投机客梦醒的办法就是让资产损失至少一半 天不会塌下来,世界末日也不会这么快就降临,经济也许会萧条但不会崩溃,人们还得穿衣吃饭.休息劳作 每个笨蛋都会从自己的错误中吸取教训,聪明的人则从别人 ...

  2. Spring WebSocket教程(二)

    实现目标 这一篇文章,就要直接实现聊天的功能,并且,在聊天功能的基础上,再实现缓存一定聊天记录的功能. 第一步:聊天实现原理 首先,需要明确我们的需求.通常,网页上的聊天,都是聊天室的形式,所以,这个 ...

  3. Dubbo相关博文整理

    configServer配置中心在dubbo client和 dubbo server之间的作用 http://www.cnblogs.com/dengzy/p/5677531.html dubbo ...

  4. hbase源码系列(五)Trie单词查找树

    在上一章中提到了编码压缩,讲了一个简单的DataBlockEncoding.PREFIX算法,它用的是前序编码压缩的算法,它搜索到时候,是全扫描的方式搜索的,如此一来,搜索效率实在是不敢恭维,所以在h ...

  5. Spring Cloud Config 配置高可用集群

    详细参考:<Sprin Cloud 与 Docker 微服务架构实战>p163-9.10节 spring cloud config 与 eureka 配合使用 我就不写了,请参见本书章节.

  6. PCL中分割_欧式分割(1)

    基于欧式距离的分割和基于区域生长的分割本质上都是用区分邻里关系远近来完成的.由于点云数据提供了更高维度的数据,故有很多信息可以提取获得.欧几里得算法使用邻居之间距离作为判定标准,而区域生长算法则利用了 ...

  7. 彻底去除Win10“快速访问”

    windows10的“快速访问”功能对于我个人用处不大,作为一个爱折腾的人决定尝试彻底除去“快速访问”这个侧边栏. 注意:此操作需要确保你已经设置了[让点击Win10任务栏“文件资源管理器”图标打开“ ...

  8. Android Studio生成keystore签名文件步骤讲解

    Android App打包时要用到签名文件,Android Studio生成签名文件步骤如下: Build---Generate Signed Apk...如图: 如果你的project中有2个或者2 ...

  9. JUnit教程

    测试是检查应用程序是否是工作按照要求,并确保在开发者水平,单元测试进入功能性的处理.单元测试是单一实体(类或方法)的测试. 单元测试在每一个软件公司开发高品质的产品给他们的客户是十分必要的. 单元测试 ...

  10. Java的图形用户界面的基本工具

    AWT(Abstract Window Toolkit),中文译为抽象窗口工具包,该包提供了一套与本地图形界面进行交互的接口,是Java提供的用来建立和设置Java的图形用户界面的基本工具. AWT中 ...