如何为共享内存映射选择一个固定地址 [英] How to choose a fixed address for shared memory mapping

问题描述

我想在多个进程之间使用共享内存，并希望能够继续使用原始指针(和stl容器).

为此，我正在使用映射到固定地址的共享内存:

segment = new boost::interprocess::managed_shared_memory(
    boost::interprocess::open_or_create,
    "MySegmentName",
    1048576, // alloc size
    (void *)0x400000000LL // fixed address
);

选择此固定地址的最佳策略是什么?例如，是否应该使用一个较高的数字来减少堆空间用完的机会?

解决方案

这是一个难题.如果您要分叉一个程序来创建子级，而只有父级和子级会使用该内存段，则请确保在进行分叉之前先将其映射.子代将自动从其父代继承映射，而无需使用固定地址.

如果不是，那么首先要考虑的是您是否真的需要使用原始STL容器而不是boost进程间容器.您已经在使用boost进程间分配共享内存段，这表明使用boost没有任何问题，因此我能想到的使用STL容器的唯一优势就是不必移植现有代码.请记住，要使其使用固定地址，容器及其包含的指针(假设您使用的是指针容器)将需要保留在共享内存空间中./p>

如果确定是您想要的，则必须找出一些方法让他们协商地址.请记住，允许操作系统拒绝您所需的固定内存地址.如果该地址的页面已被映射到内存或已分配，它将拒绝该地址.由于不同的程序会在不同的时间分配不同的内存量，因此哪些页面可用和哪些页面不可用会因您的程序而异.

因此，您需要程序在内存地址上达成共识.这意味着可能必须尝试并拒绝多个地址.如果有可能在启动新程序后的某个时间对新程序感兴趣，那么寻求共识的搜索将不得不重新开始.该算法将如下所示:

1. 程序A向所有其他程序提出内存地址X.
2. 其他程序以true或false进行响应，以指示地址X处的内存映射是否成功.
3. 如果程序A收到任何错误响应，请转到#1.
4. 程序A向其他程序发送一条消息，让他们知道该地址已被验证并且可以使用.
5. 如果新应用对数据感兴趣，它必须通知程序A它想要一个地址.
6. 然后程序A必须告诉所有其他程序停止使用数据并转到#1.

要想出A应该建议的地址，您可以让A映射一个非固定的内存段，查看它映射到的地址，然后提出该地址.如果不满意，请映射另一个线段并提出.您将需要在某个时候取消映射这些段，但是您无法立即取消映射它们，因为如果您取消映射然后重新映射具有相同大小的段，则操作系统将一遍又一遍地为您提供相同的地址. 请记住，您可能永远无法达成共识；不能保证在所有流程的同一位置有足够大的细分.如果您的程序全部独立地使用了几乎所有内存，例如如果它们由大量的交换进行备份，则可能会发生这种情况(尽管如果您对性能的关注足够大，希望可以避免使用共享内存).

以上所有条件均假设您处于相对受限的地址空间中. 如果您使用的是64位，则可以使用.大多数计算机的RAM +交换空间将远远小于64位所允许的空间，因此您可以将内存映射到一个非常遥远的固定地址，该地址不可能已经映射了所有进程.我建议至少使用2 ^ 48，因为当前的64位x86处理器都不会超出该范围(尽管指针是64位，所以您只能插入48位所允许的尽可能多的RAM，但仍然需要大量内存).撰写本文时).尽管没有理由使智能堆分配器无法利用地址空间的巨大空间来减少其簿记工作，但要真正强大起来，您仍然需要建立共识.请记住，您至少希望该地址是可配置的-即使我们很快就没有这么多的存储空间，从现在到现在，其他人可能会有相同的想法并选择您的地址.

要进行双向通信，可以使用套接字，管道或其他共享内存段中的任何一个.您的操作系统可能提供其他形式的IPC.但是请坚决认为，您现在可能引入的复杂性要比仅使用boost进程间容器要解决的复杂性;)

I would like to use shared memory between several processes, and would like to be able to keep using raw pointers (and stl containers).

For this purpose, I am using shared memory mapped at a fixed address:

segment = new boost::interprocess::managed_shared_memory(
    boost::interprocess::open_or_create,
    "MySegmentName",
    1048576, // alloc size
    (void *)0x400000000LL // fixed address
);

What is a good strategy for choosing this fixed address? For example, should I just use a pretty high number to reduce the chance that I run out of heap space?

解决方案

This is a hard problem. If you are forking a single program to create children, and only the parent and the children will use the memory segment, just be sure to map it before you fork. The children will automatically inherit the mapping from their parent and there's no need to use a fixed address.

If you aren't, then the first thing to consider is whether you really need to use raw STL containers instead of the boost interprocess containers. That you're already using boost interprocess to allocate the shared memory segment suggests you don't have any problem using boost, so the only advantage I can think of to using STL containers would be so you don't have to port existing code. Keep in mind that for it to work with fixed addresses, the containers and what they contain pointers to (assuming you're working with containers of pointers) will need to be kept in the shared memory space.

If you're certain that it's what you want, you'll have to figure out some method for them to negotiate an address. Keep in mind that the OS is allowed to reject your desired fixed memory address. It will reject an address if the page at that address has already been mapped into memory or allocated. Because different programs will have allocated different amounts of memory at different times, which pages are available and which are unavailable will vary across your programs.

So you need for the programs to gain consensus on a memory address. This means that several addresses might have to be tried and rejected. If it's possible that sometime after startup a new program will become interested, the search for consensus will have to start over again. The algorithm would look something like this:

1. Program A proposes memory address X to all other programs.
2. The other programs respond with true or false to indicate whether the memory mapping at address X succeeded.
3. If program A receives any false responses, goto #1.
4. Program A sends a message to the other programs letting them know the address has been validated and maybe used.
5. If a new app becomes interested in the data, it must notify program A it would like an address.
6. Program A then has to tell all the other programs to stop using the data and goto #1.

To come up with what addresses A should propose, you could have A map a non-fixed memory segment, see what address it's mapped at, and propose that address. If it's unsatisfactory, map another segment and propose it instead. You will need to unmap the segments at some point, but you can't unmap them right away because if you unmap then remap a segment of the same size chances are the OS will give you the same address back over and over. Keep in mind that you may never reach consensus; there's no guarantee that there's a large enough segment at a common location across all the processes. This could happen if your programs all independently use almost all memory, say if they are backed up by a ton of swap (though if you care enough about performance to use shared memory hopefully you are avoiding swap).

All of the above assumes you're in a relatively constrained address space. If you're on 64-bit, this could work. Most computers' RAM + swap will be far less than what's allowed by 64-bits, so you could put map the memory at a very far out fixed address that all processes are unlikely to have mapped already. I suggest at least 2^48, since current 64-bit x86 processors don't each beyond that range (despite pointers being 64-bits, you can only plug in as much RAM as allowed by 48-bits, still a ton at the time of this writing). Although there's no reason a smart heap allocator couldn't take advantage of the vastness of the address space to reduce its bookkeeping work, so to be truly robust you would still need to build consensus. Keep in mind that you will at least want the address to be configurable -- even if we don't have that much memory anytime soon, between now and then someone else might have the same idea and pick your address.

To do the bidirectional communication you could use any of sockets, pipes, or another shared memory segment. Your OS may provide other forms of IPC. But strongly consider that you are probably now introducing more complexity than you would have to deal with if you just used the boost interprocess containers ;)

这篇关于如何为共享内存映射选择一个固定地址的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！