是否可以监视派生的linux进程在写时的副本?(特别是python) [英] Is it possible to monitor copy on write for forked linux processes? (specifically python)

问题描述

我有一组共享一个大对象的python进程(共享是通过在初始化该对象后分叉这些进程来完成的)

I have a set of python processes that share a large object (sharing is done by forking the processes after initializing the object)

我注意到一个奇怪的内存泄漏:

I notice a strange memory leak:

• 进程内存(VSZ和RSS)几乎不变
• 系统总内存稳步增长

我的猜测是共享对象确实发生了更改(它在逻辑上是只读的，但是即使从其读取时某些内部私有变量也可能会发生更改)-这会导致内存页被复制

My guess is the shared object does change (it's 'logically' readonly, but it's possible that some internal private variables change even when just reading from it) - which causes memory pages to be copied

有没有办法验证这一点?

is there a way to verify this?

推荐答案

要回答您的特定问题是否可以验证此问题?"，如果我正确理解的话，如果要查看是否可以进行以下操作与包含大对象的页面相关联的任何更改.

To answer your specific question "is there a way to verify this?", if I understand it correctly, you can do the following if you want to see whether there are any changes associated with the pages that contain the large object.

1)确定大型共享对象"的地址以及该对象结束的地址.

1) Determine the address of your "large shared object" and the address of where that object ends.

2)如果起始地址不在4K页面边界上，则将起始地址向下舍入到对象开始之前的页面边界.

2) If the start address is not on a 4K page boundary, round the start address down to the page boundary before where the object starts.

3)如果结束地址不在4K边界上，则将结束地址向上舍入到对象结束后的页面边界.

3) If the end address is not on a 4K boundary, round the end address up to the page boundary after where the object ends.

4)将进程及其所有子进程的内存范围转储到单独的文件中并进行比较.

4) Dump that memory range for the process and all its children to separate files and compare them.

但是，我认为

However, I think that Will os.fork() use copy on write or do a full copy of the parent-process in Python? may already provide an explanation for at least some of the writing that is requiring copying. Specifically, python objects are reference counted and your child processes will be altering reference counts.

您是否考虑过使用python的线程而不是创建子进程?

Have you considered using python's threading rather than creating child processes?

这篇关于是否可以监视派生的linux进程在写时的副本?(特别是python)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！