Apache模块内存映射文件同步过度 [英] Excessive synchronizing of memory mapped file in Apache module

问题描述

我目前正在开发一个使用大型 mmap 文件在进程之间共享数据的 Apache 模块.这是在启动时创建并在服务器关闭时删除(可以选择在稍后阶段保留它).我已经使用 Apache APR 库实现了这一点，它运行良好，至少对于较小的文件是这样.

I am currently working on an Apache module that uses a large mmap file to share data between processes. This is created on start-up and removed when the server shuts down (May choose to keep it at a later stage). I have implemented this using the Apache APR libraries, and it works well, at least for smaller files.

然而，当内存映射文件的大小增加时(服务器运行时仍有足够的 RAM 来缓存它)，系统有时实际上会停止，因为它似乎操作系统(在我的情况下为 Linux)消耗大量资源将文件与磁盘同步.有没有办法控制/减少这种同步?

When the size of the memory mapped file however increases (there is still enough RAM to cache it when the server is running) the system at times virtually grinds to a halt as it seems the operating system (Linux in my case) consumes a lot of resources synchronizing the file with the disk. Is there any way to control/reduce this synchronization?

由于我目前不需要将内容同步到磁盘，所以我可能应该使用共享内存段来代替，并且会尝试使用它.然而，我仍然对控制内存映射文件同步的方法感兴趣.

Since I at the moment do not need the contents synchronized to disk I should probably have used a shared memory segment instead, and will try that instead. I am however still interested in ways to control synchronization of memory mapped files.

推荐答案

写入映射文件会创建脏页 - 在某些时候需要写出到其支持的内存页存储(在本例中为您的磁盘文件).

Writing to your mapped file is creating dirty pages - memory pages that need, at some point, to be written out to their backing store (in this case, your on-disk file).

脏页的写出可以通过/proc/sys/vm/中的一些旋钮来调整.特别是，如果映射文件中的脏数据量与系统总内存相比通常很大，您可能希望显着增加 dirty_ratio(例如，增加到 60)和 dirty_background_ratio 一点点(比如，到 30).

The writeout of dirty pages is tuneable by some knobs in /proc/sys/vm/. In particular, if the amount of dirty data in your mapped file is typically large compared to your total system memory, you may want to increase dirty_ratio significantly (say, to 60), and dirty_background_ratio a little (say, to 30).

您可能还想增加 dirty_expire_centisecs，但默认值已经很长(5 分钟)(这是脏数据在符合写出条件之前必须达到的年龄).

You may also want to increase dirty_expire_centisecs, but the default is already quite long (5 minutes) (this is the age that dirty data must reach before it becomes eligible for writeout).

值得指出的是，切换到共享内存只是在 /dev/shm/ 中创建映射文件的问题，假设您的发行版在那里安装了 tmpfs.

It is worth pointing out that switching to shared memory is simply a matter of creating your mapped file in /dev/shm/, assuming your distribution mounts a tmpfs there.

这篇关于Apache模块内存映射文件同步过度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！