http://www.captaincodeman.com/2011/02/27/limit-mongodb-memory-use-windows/

Limit MongoDB memory use on Windows without Virtualization

I’ve seen the question of how to control MongoDB’s memory usage on Windows come up several times and the stock answer always seemed to be “you can’t – it uses memory-mapped files and if you want to limit resources you need to use some form of virtualization to do it (HyperV, VMWare, Virtuozzo etc…)”.

If you are using MongoDB on a dedicated server then you generally want it to use all the memory it can but if you want to use it on a server shared with other processes (e.g. an IIS website using MongoDB for storage, maybe with SQL Server as well) then you will want to put a cap on how much it uses to ensure memory is kept available for the other processes.

So is it possible if you are not on a virtualized environment? Yes (otherwise this would be a very short blog post!) and we’ll explore how …

The standard behaviour described above is actually a result of the default resource manager used by Windows but both Windows 2003 and Windows 2008 have a separate installable option called the “Windows System Resource Manager” (WSRM) that allows greater control over the CPU and Memory available to a process.

First of all, lets look at what we’re trying to solve. Here we have a low-memory server (only 2Gb) running MongoDB on Windows 2008 R2 x64. There are a few databases of a few Gb each so the mongod.exe process quickly starts consuming as much memory as it can (rightly so) to keep as much of it’s indexes in memory for the fast performance we know and love:

What we’d like to do is save some memory for other processes by limiting the mongod.exe process to 1Gb in this case (I know this is ridiculously low but the only thing that will change for you are the actual limits you want to use).

To do this we first need to install Windows System Resource Manager which on Windows 2008 is available under the Features section of the Server Manager.

Once that’s installed fire it up and you’ll see the default resource management policies. By default the standard Windows “memory is given to whoever shouts loudest” policy is used but other pre-configured alternatives are available. WSRM also provides a calendar / event system where the policy can be changed at certain times (a typical scenario is giving critical business apps priority during the day but then batch processes greater priority overnight). We’re not going to go into the calendar features here but it’s interesting to know about.

Let’s create a new policy to control the resources that MongoDB can consume. To do this, right click on the “Resource Allocation Policies” container and chose “New Resource Allocation Policy …”. This will present us with the New Resource Allocation Policy dialog below:

First of all, we need to add a new resource allocation entry so click the ‘Add…’ button and we get to another “Add or Edit Resource Allocation” dialog:

We don’t have a Process matching criteria for MongoDB yet so choose <New…> to get … yes, you guessed – another dialog, this time “New Process Matching Criteria”. We’ll call it “mongod_process” and click the Add… button to get another death-by-dialog to define it.

There are a few ways to do this – if MongoDB is installed as a service then you can choose “Registered Service” in the drop-down, click “Select” and choose it from the list or you can select from a list of running processes or you can just enter the full path and filename to mongod.exe. Here is the entry after selecting an installed MongoDB Windows Service:

After clicking OK we get back to the Process Matching Criteria dialog showing our new rule:

After clicking OK we’re now back at the Resource Allocation dialog with the new “mongod_process” Process matching criteria selected and can now decide what resources we want to allocate to the process. Lets limit the CPU to 50% (not that MongoDB seems to consume much CPU):

The Memory tab allows us to limit the memory and here there are two options. The maximum committed memory limit is more to control apps that may have a memory leak and can be setup to stop or alert someone when the process goes above the configured limit. We don’t want this one … instead we’ll set a maximum working set limit which will control how much memory is allocated to MongoDB. In this case, we’ll set the limit to 1Gb but the actual value to use will depend on your circumstances:

After clicking OK we should then be at the Resource Allocation Policy dialog with our process matching criteria, CPU and memory limits shown. We could include more limits in the policy but we’ll leave it as it is for now – any remaining resources will be allocated to other processes as normal after the limits have been imposed.

The final piece is to make this policy active which is done by clicking on the “Selected Policy” link on the main ‘page’ or right-clicking on the new entry under the “Resource Allocation Policies” and choosing “Set as Managing Policy”. You can also right-click on the “Windows System Resource Manager (local)” entry and choose “Properties …” to display the dialog below which allows you to select the Current resource allocation policy:

So, we’ve created a new policy that has a criteria to match the mongod.exe process which will limit the CPU usage to 50% and memory to 1Gb … does it work? Here’s the result after it’s enabled showing the memory used immediately dropping:

… and the MongoDB / mongod.exe process using the 1Gb limit we specified (1Gb = 1024Mb = 1,048,576Kb).

So, we’ve successfully limited the CPU and memory that MongoDB can consume without having to resort to any form of server-virtualization and while MongoDB will probably not run as fast as it did when it had free-reign to consume as much as it wanted (or rather, when the default windows resource manager gave it what it asked for) we will probably have a faster overall system as our other processes are allocated the memory and CPU that they need for a better balanced system.

Please let me know what you think of the above technique and if you find it useful.



27 February 2011

© 2015 Simon Green with help from Jekyll Bootstrap and Twitter Bootstrap

系统windows进程的资源分配的更多相关文章

  1. Windows进程崩溃问题定位方法

    Linux上进程崩溃通常会生成core文件,用gdb打开后执行bt命令即可查看堆栈.而在Windows平台上,我们通常会采用MiniDumpWriteDump来进行堆栈转储,而这需要对系统Api有一定 ...

  2. windows进程详解

    1:系统必要进程system process    进程文件: [system process] or [system process]进程名称: Windows内存处理系统进程描述: Windows ...

  3. windows进程/线程创建过程 --- windows操作系统学习

    有了之前的对进程和线程对象的学习的铺垫后,我们现在可以开始学习windows下的进程创建过程了,我将尝试着从源代码的层次来分析在windows下创建一个进程都要涉及到哪些步骤,都要涉及到哪些数据结构. ...

  4. Linux系统编程@进程通信(一)

    进程间通信概述 需要进程通信的原因: 数据传输 资源共享 通知事件 进程控制 Linux进程间通信(IPC)发展由来 Unix进程间通信 基于System V进程间通信(System V:UNIX系统 ...

  5. windows 进程间通讯方法

    Windows平台为我们提供了多种进程间通信的机制,主要包括:注册表方式.共享文件方式.共享内存方式.共享数据段.映射文件方式.管道方式. 剪贴板方式.消息方式.其中注册表方式需要增加注册表表项,而注 ...

  6. 走进windows编程的世界-----windows进程

    Windows进程  1 Windows进程    进程是一个容器,包括了一个应用程序实例的各种资源.Windows多任务的操作系统,因此能够同一时候运行多个进程.      2 Windows进程的 ...

  7. Windows进程单实例运行

    场景         Windows进程单实例运行,如果有进程没有退出,继续等待,直到进程完全退出,才会进入下一个实例 HANDLE pHandle = NULL; do  {  pHandle = ...

  8. linux进程cpu资源分配命令nice,renice,taskset

    进程cpu资源分配就是指进程的优先权(priority).优先权高的进程有优先执行权利.配置进程优先权对多任务环境的linux很有用,可以改善系统性能.还可以把进程运行到指定的CPU上,这样一来,把不 ...

  9. Windows进程间共享内存通信实例

    Windows进程间共享内存通信实例 抄抄补补整出来 采用内存映射文件实现WIN32进程间的通讯:Windows中的内存映射文件的机制为我们高效地操作文件提供了一种途径,它允许我们在WIN32进程中保 ...

随机推荐

  1. C语言 · Quadratic Equation

    算法提高 Quadratic Equation   时间限制:1.0s   内存限制:512.0MB      问题描述 求解方程ax2+bx+c=0的根.要求a, b, c由用户输入,并且可以为任意 ...

  2. C++生成全局唯一Guid

    在软件开发中,我们经常需要一个唯一标识来标识一个对象,在PHP中,可以用UUID来获得全局唯一的ID,其实C++也内置了简单的GUID获得办法,猜想GUID应该是本机的特征以及运行的时间的一个哈希值, ...

  3. Android ——Toolbar开发实践总结(转)

    过年前发了一篇介绍 Translucent System Bar 特性的文章 Translucent System Bar 的最佳实践 ,收到很多开发者的关注和反馈.今天开始写第二篇,全面的介绍一下  ...

  4. Hibernate 查询说明

    1.Criteria setFetchSize实际上与分页操作并没有关系,它的作用是在查询中进行分批数据返回,其中的值就是每次分批查询的记录数,主要是为了提高查询性能的. 举个例子来说:如果你这次查询 ...

  5. hbase源码系列(九)StoreFile存储格式

    从这一章开始要讲Region Server这块的了,但是在讲Region Server这块之前得讲一下StoreFile,否则后面的不好讲下去,这块是基础,Region Sever上面的操作,大部分都 ...

  6. Java知多少(17)强调一下编程风格

    讲完了Java的基础语法,大家就可以编写简单的程序代码了,这里有必要强调一下编程风格. 代码风格虽然不影响程序的运行,但对程序的可读性却非常重要.自己编写的程序要让别人看懂,首先在排版方面要非常注意. ...

  7. 在ASP.NET CORE下生成PDF文档

    原文链接:https://www.c-sharpcorner.com/article/creating-pdf-in-asp-net-core-mvc-using-rotativa-aspnetcor ...

  8. HwPointEventFilter: do not support AFT because of no config华为手机进入工程菜单

    在调试时应用报出HwPointEventFilter: do not support AFT because of no config 是因为华为系统里设置了不打印log 解决方法是在拨号界面输入*# ...

  9. HBase Rowkey的散列与预分区设计

    转自:http://www.cnblogs.com/bdifn/p/3801737.html 问题导读:1.如何防止热点?2.如何预分区?扩展:为什么会产生热点存储? HBase中,表会被划分为1.. ...

  10. 自然语言交流系统 phxnet团队 创新实训 项目博客 (十)

      关于本项目中使用到的庖丁分词的总结:   Paoding 详细介绍 庖丁中文分词库是一个使用Java开发的,可结合到Lucene应用中的,为互联网.企业内部网使用的中文搜索引擎分词组件.Paodi ...