一种确定过程“真实"的方法内存使用，即私人脏RSS? [英] A way to determine a process's "real" memory usage, i.e. private dirty RSS?

问题描述

ps"和top"等工具报告各种内存使用情况，例如 VM 大小和驻留集大小.但是，这些都不是真正的"内存使用:

Tools like 'ps' and 'top' report various kinds of memory usages, such as the VM size and the Resident Set Size. However, none of those are the "real" memory usage:

• 程序代码在同一程序的多个实例之间共享.
• 共享库程序代码在使用该库的所有进程之间共享.
• 某些应用分叉进程并与它们共享内存(例如，通过共享内存段).
• 虚拟内存系统使虚拟机大小报告几乎毫无用处.
• 当进程被换出时，RSS 为 0，这使得它不是很有用.
• 等等

我发现 Linux 报告的私有脏 RSS 是最接近真实"内存使用情况的东西.这可以通过对 /proc/somepid/smaps 中的所有 Private_Dirty 值求和来获得.

I've found that the private dirty RSS, as reported by Linux, is the closest thing to the "real" memory usage. This can be obtained by summing all Private_Dirty values in /proc/somepid/smaps.

但是，其他操作系统是否提供类似的功能?如果不是，有哪些替代方案?特别是，我对 FreeBSD 和 OS X 感兴趣.

However, do other operating systems provide similar functionality? If not, what are the alternatives? In particular, I'm interested in FreeBSD and OS X.

推荐答案

在 OSX 上，活动监视器实际上给了你一个很好的猜测.

On OSX the Activity Monitor gives you actually a very good guess.

私有内存肯定是仅由您的应用程序使用的内存.例如.堆栈内存和所有使用 malloc() 和类似函数/方法(Objective-C 的 alloc 方法)动态保留的内存都是私有内存.如果你分叉，私人内存将与你的孩子共享，但标记为写时复制.这意味着只要页面没有被任何一个进程(父进程或子进程)修改，它就会在它们之间共享.只要任一进程修改任何页面，就会在修改之前复制该页面.即使此内存与 fork 子进程共享(并且它只能与 fork 子进程共享)，它仍然显示为私有".内存，因为在最坏的情况下，它的每一页都会被修改(迟早)，然后它又对每个进程再次私有.

Private memory is for sure memory that is only used by your application. E.g. stack memory and all memory dynamically reserved using malloc() and comparable functions/methods (alloc method for Objective-C) is private memory. If you fork, private memory will be shared with you child, but marked copy-on-write. That means as long as a page is not modified by either process (parent or child) it is shared between them. As soon as either process modifies any page, this page is copied before it is modified. Even while this memory is shared with fork children (and it can only be shared with fork children), it is still shown as "private" memory, because in the worst case, every page of it will get modified (sooner or later) and then it is again private to each process again.

共享内存要么是当前共享的内存(相同页面在不同进程的虚拟进程空间中可见)，要么是将来可能共享的内存(例如只读内存，因为没有理由不共享只读内存).至少我是这样阅读 Apple 的一些命令行工具的源代码的.因此，如果您使用 mmap(或将同一内存映射到多个进程的类似调用)在进程之间共享内存，这将是共享内存.然而，可执行代码本身也是共享内存，因为如果你的应用程序的另一个实例被启动，它没有理由不共享已经加载到内存中的代码(可执行代码页默认是只读的，除非你正在运行你的应用程序)调试器中的应用程序).因此，共享内存实际上是您的应用程序使用的内存，就像私有内存一样，但它可能会另外与另一个进程共享(或者可能不会，但如果共享，为什么不计入您的应用程序?)

Shared memory is either memory that is currently shared (the same pages are visible in the virtual process space of different processes) or that is likely to become shared in the future (e.g. read-only memory, since there is no reason for not sharing read-only memory). At least that's how I read the source code of some command line tools from Apple. So if you share memory between processes using mmap (or a comparable call that maps the same memory into multiple processes), this would be shared memory. However the executable code itself is also shared memory, since if another instance of your application is started there is no reason why it may not share the code already loaded in memory (executable code pages are read-only by default, unless you are running your app in a debugger). Thus shared memory is really memory used by your application, just like private one, but it might additionally be shared with another process (or it might not, but why would it not count towards your application if it was shared?)

实际内存是当前分配"的 RAM 量；到您的流程，无论是私有的还是共享的.这可以正好是私有和共享的总和，但通常不是.您的进程分配给它的内存可能比当前需要的多(这会加快将来对更多内存的请求)，但这对系统没有任何损失.如果另一个进程需要内存而没有可用内存，则在系统开始交换之前，它会从您的进程中取出多余的内存并将其分配给另一个进程(这是一个快速且轻松的操作)；因此，您的下一个 malloc 调用可能会慢一些.实内存也可以小于私有和物理内存；这是因为如果您的进程从系统请求内存，它只会接收虚拟内存".只要您不使用它，这个虚拟内存就不会链接到任何实际内存页面(因此 malloc 10 MB 内存，只使用其中的一个字节，您的进程将只获得一个页面，4096 字节，分配的内存- 其余的仅在您确实需要时才分配).被交换的更多内存可能也不计入实内存(对此不确定)，但会计入共享内存和私有内存.

Real memory is the amount of RAM currently "assigned" to your process, no matter if private or shared. This can be exactly the sum of private and shared, but usually it is not. Your process might have more memory assigned to it than it currently needs (this speeds up requests for more memory in the future), but that is no loss to the system. If another process needs memory and no free memory is available, before the system starts swapping, it will take that extra memory away from your process and assign it another process (which is a fast and painless operation); therefor your next malloc call might be somewhat slower. Real memory can also be smaller than private and physical memory; this is because if your process requests memory from the system, it will only receive "virtual memory". This virtual memory is not linked to any real memory pages as long as you don't use it (so malloc 10 MB of memory, use only one byte of it, your process will get only a single page, 4096 byte, of memory assigned - the rest is only assigned if you actually ever need it). Further memory that is swapped may not count towards real memory either (not sure about this), but it will count towards shared and private memory.

虚拟内存是在您的应用程序进程空间中被认为有效的所有地址块的总和.这些地址可能会链接到物理内存(同样是私有或共享的)，也可能不会，但在这种情况下，一旦您使用该地址，它们就会链接到物理内存.访问已知地址之外的内存地址将导致 SIGBUS 并且您的应用程序将崩溃.当内存被交换时，该内存的虚拟地址空间仍然有效，访问这些地址会导致内存被换回.

Virtual memory is the sum of all address blocks that are consider valid in your apps process space. These addresses might be linked to physical memory (that is again private or shared), or they might not, but in that case they will be linked to physical memory as soon as you use the address. Accessing memory addresses outside of the known addresses will cause a SIGBUS and your app will crash. When memory is swapped, the virtual address space for this memory remains valid and accessing those addresses causes memory to be swapped back in.

结论:
如果您的应用程序没有显式或隐式地使用共享内存，则私有内存是您的应用程序需要的内存量，因为堆栈大小(或多线程时的大小)以及您为动态内存进行的 malloc() 调用.在这种情况下，您不必非常关心共享内存或真实内存.

Conclusion:
If your app does not explicitly or implicitly use shared memory, private memory is the amount of memory your app needs because of the stack size (or sizes if multithreaded) and because of the malloc() calls you made for dynamic memory. You don't have to care a lot for shared or real memory in that case.

如果您的应用程序使用共享内存，并且这包括图形用户界面，其中内存在您的应用程序和 WindowServer 之间共享，例如，那么您也可以查看共享内存.非常高的共享内存数量可能意味着您目前在内存中加载了过多的图形资源.

If your app uses shared memory, and this includes a graphical UI, where memory is shared between your application and the WindowServer for example, then you might have a look at shared memory as well. A very high shared memory number may mean you have too many graphical resources loaded in memory at the moment.

真正的内存对应用程序开发没什么兴趣.如果它大于共享和私有的总和，那么这仅意味着系统懒惰地从您的进程中取出内存.如果它更小，那么你的进程请求的内存比实际需要的多，这也不错，因为只要你不使用所有请求的内存，你就不会窃取"内存.来自系统的内存.如果它远小于 shared 和 private 的总和，你可能只考虑在可能的情况下请求更少的内存，因为你有点过度请求内存(同样，这还不错，但它告诉我你的代码不是针对最小内存使用进行了优化，如果它是跨平台的，其他平台可能没有如此复杂的内存处理，因此您可能更喜欢分配许多小块而不是一些大块，或者更快地释放内存，等等上).

Real memory is of little interest for app development. If it is bigger than the sum of shared and private, then this means nothing other than that the system is lazy at taken memory away from your process. If it is smaller, then your process has requested more memory than it actually needed, which is not bad either, since as long as you don't use all of the requested memory, you are not "stealing" memory from the system. If it is much smaller than the sum of shared and private, you may only consider to request less memory where possible, as you are a bit over-requesting memory (again, this is not bad, but it tells me that your code is not optimized for minimal memory usage and if it is cross platform, other platforms may not have such a sophisticated memory handling, so you may prefer to alloc many small blocks instead of a few big ones for example, or free memory a lot sooner, and so on).

如果您对所有这些信息仍然不满意，您可以获得更多信息.打开终端并运行:

If you are still not happy with all that information, you can get even more information. Open a terminal and run:

sudo vmmap <pid>

您的进程的进程 ID 在哪里.这将向您显示进程空间中每个 内存块的统计信息，包括起始地址和结束地址.它还会告诉你这个内存来自哪里(映射文件?堆栈内存?Malloc 内存?可执行文件的 __DATA 或 __TEXT 部分?)，它有多大，以 KB 为单位，访问权限以及它是否是私有的，共享或写时复制.如果它是从文件映射的，它甚至会给你文件的路径.

where is the process ID of your process. This will show you statistics for EVERY block of memory in your process space with start and end address. It will also tell you where this memory came from (A mapped file? Stack memory? Malloc'ed memory? A __DATA or __TEXT section of your executable?), how big it is in KB, the access rights and whether it is private, shared or copy-on-write. If it is mapped from a file, it will even give you the path to the file.

如果您只想要实际"RAM使用情况，使用

If you want only "actual" RAM usage, use

sudo vmmap -resident <pid>

现在它将显示每个内存块的虚拟内存块有多大，以及当前实际存在于物理内存中的内存块有多少.

Now it will show for every memory block how big the memory block is virtually and how much of it is really currently present in physical memory.

在每个转储的末尾还有一个概述表，其中包含不同内存类型的总和.这个表格现在在我的系统上对于 Firefox 来说是这样的:

At the end of each dump is also an overview table with the sums of different memory types. This table looks like this for Firefox right now on my system:

REGION TYPE             [ VIRTUAL/RESIDENT]
===========             [ =======/========]
ATS (font support)      [   33.8M/   2496K]
CG backing stores       [   5588K/   5460K]
CG image                [     20K/     20K]
CG raster data          [    576K/    576K]
CG shared images        [   2572K/   2404K]
Carbon                  [   1516K/   1516K]
CoreGraphics            [      8K/      8K]
IOKit                   [  256.0M/      0K]
MALLOC                  [  256.9M/  247.2M]
Memory tag=240          [      4K/      4K]
Memory tag=242          [     12K/     12K]
Memory tag=243          [      8K/      8K]
Memory tag=249          [    156K/     76K]
STACK GUARD             [  101.2M/   9908K]
Stack                   [   14.0M/    248K]
VM_ALLOCATE             [   25.9M/   25.6M]
__DATA                  [   6752K/   3808K]
__DATA/__OBJC           [     28K/     28K]
__IMAGE                 [   1240K/    112K]
__IMPORT                [    104K/    104K]
__LINKEDIT              [   30.7M/   3184K]
__OBJC                  [   1388K/   1336K]
__OBJC/__DATA           [     72K/     72K]
__PAGEZERO              [      4K/      0K]
__TEXT                  [  108.6M/   63.5M]
__UNICODE               [    536K/    512K]
mapped file             [  118.8M/   50.8M]
shared memory           [    300K/    276K]
shared pmap             [   6396K/   3120K]

这告诉我们什么?例如.Firefox 二进制文件和它加载的所有库在它们的 __TEXT 部分中总共有 108 MB 数据，但目前只有 63 MB 驻留在内存中.字体支持 (ATS) 需要 33 MB，但实际内存中只有 2.5 MB 左右.它使用 5 MB 多一点的 CG 后备存储，CG = Core Graphics，这些很可能是窗口内容、按钮、图像和其他缓存数据以用于快速绘制.它通过 malloc 调用请求了 256 MB，目前 247 MB​​ 确实映射到内存页面.它为堆栈保留了 14 MB 的空间，但现在只有 248 KB 的堆栈空间真正在使用.

What does this tell us? E.g. the Firefox binary and all library it loads have 108 MB data together in their __TEXT sections, but currently only 63 MB of those are currently resident in memory. The font support (ATS) needs 33 MB, but only about 2.5 MB are really in memory. It uses a bit over 5 MB CG backing stores, CG = Core Graphics, those are most likely window contents, buttons, images and other data that is cached for fast drawing. It has requested 256 MB via malloc calls and currently 247 MB are really in mapped to memory pages. It has 14 MB space reserved for stacks, but only 248 KB stack space is really in use right now.

vmmap 在表格上方也有很好的总结

vmmap also has a good summary above the table

ReadOnly portion of Libraries: Total=139.3M resident=66.6M(48%) swapped_out_or_unallocated=72.7M(52%)
Writable regions: Total=595.4M written=201.8M(34%) resident=283.1M(48%) swapped_out=0K(0%) unallocated=312.3M(52%)

这显示了 OS X 一个有趣的方面:​​对于来自库的只读内存，如果它被换出或只是未分配，它就不起作用；只有常驻，没有常驻.对于可写内存，这会有所不同(在我的情况下，所有请求的内存中有 52% 从未使用过并且未分配，0% 的内存已换出到磁盘).

And this shows an interesting aspect of the OS X: For read only memory that comes from libraries, it plays no role if it is swapped out or simply unallocated; there is only resident and not resident. For writable memory this makes a difference (in my case 52% of all requested memory has never been used and is such unallocated, 0% of memory has been swapped out to disk).

原因很简单:映射文件中的只读内存不会被交换.如果系统需要内存，则简单地从进程中删除当前页面，因为内存已经交换".它仅由直接从文件映射的内容组成，并且可以在需要时重新映射此内容，因为文件仍然存在.这样这个内存也不会浪费交换文件中的空间.只有可写内存必须在删除之前先交换到文件，因为它的内容之前没有存储在磁盘上.

The reason for that is simple: Read-only memory from mapped files is not swapped. If the memory is needed by the system, the current pages are simply dropped from the process, as the memory is already "swapped". It consisted only of content mapped directly from files and this content can be remapped whenever needed, as the files are still there. That way this memory won't waste space in the swap file either. Only writable memory must first be swapped to file before it is dropped, as its content wasn't stored on disk before.

这篇关于一种确定过程“真实"的方法内存使用，即私人脏RSS?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！