无法在共享内存中使用运行时大小参数构造运行时 boost spsc_queue [英] unable to construct runtime boost spsc_queue with runtime size parameter in shared memory
问题描述
我想使用运行时指定的最大条目数在共享内存中创建一个无锁环形缓冲区.我的代码基于在 GitHub 中找到的示例.我使用此代码在共享内存中成功创建了一个无锁环形缓冲区.
就我而言，我需要指定环形缓冲区在运行时构建时可以接受的最大条目数，而不是在每个示例的编译时.示例中构造 shm::ring_buffer 的调用如下所示.
I want to make a lock free ring buffer in shared memory using runtime specified maximum number of entries. I am basing my code off an example I found in GitHub. I successfully created a lock free ring buffer in shared memory using this code.
In my case, I need to specify the maximum number of entries that the ring buffer can accept at runtime construction and not at compile time per the example. The call to construct the shm::ring_buffer in the example is shown below.
namespace bip = boost::interprocess;
namespace shm
{
using char_alloc = bip::allocator<char, bip::managed_shared_memory::segment_manager>;
using shared_string = bip::basic_string<char, std::char_traits<char>, char_alloc>;
using ring_buffer = boost::lockfree::spsc_queue<shared_string, boost::lockfree::capacity<200>>;
}
共享内存段分配如下:
mQueuingSharedMem = std::make_unique<bip::managed_shared_memory>(
bip::open_or_create, (mSharedMemoryName + "Queuing").c_str(), rSHMSize);
根据 GitHub 示例，当我通过可选的 boost::lockfree::capacity<> 模板参数构造在编译时指定的最大大小的环形缓冲区时，一切正常(注意:共享内存段 construct 方法采用 [] 中的 # ring_buffers，并在后面的括号中指定构造函数参数.
Per the GitHub example, when I construct the ring buffer with the maximum size specified at compile time via the optional boost::lockfree::capacity<> template parameter, everything works (note: that the shared memory segment construct method takes the # ring_buffers in the [] and the constructor parameters are specified in the parenthesis that follow.
auto pSharedMemAddr = mQueuingSharedMem->construct<
shm::ring_buffer>(rQueuingPortName.c_str())[1](/*aMaxNumMessages*/);
我认为为了在运行时构建上述 shm::ring_buffer，我需要从 boost::lockfree::capacity<200> 硬编码大小参数中删除第二个 boost::lockfree::capacity<200>>shm::spsc_queue，而是传递 shm::ring_buffer 的最大大小和 shm::shm_string 的共享内存分配器.我在here找到了类似的答案，但我无法适应它我的代码.
I thought that in order to construct the above shm::ring_buffer at runtime, I needed to remove the 2nd boost::lockfree::capacity<200> hard coded size parameter from the shm::spsc_queue and instead pass a maximum size for the shm::ring_buffer and the shared memory allocator for the shm::shm_string. I found a similar answer here but I was unable to adapt it to work with my code.
为了在运行时指定环形缓冲区的大小，我对上面工作的代码进行了以下更改:
I made the following changes to the code that worked above in an attempt to specify the size of the ring buffer at runtime:
namespace bip = boost::interprocess;
namespace shm
{
using char_alloc = bip::allocator<char, bip::managed_shared_memory::segment_manager>;
using shared_string = bip::basic_string<char, std::char_traits<char>, char_alloc>;
using ring_buffer = boost::lockfree::spsc_queue<shared_string/*, boost::lockfree::capacity<200>*/>;
}
shm::char_alloc char_alloc(mQueuingSharedMem->get_segment_manager());
auto pSharedMemAddr = mQueuingSharedMem->construct<
shm::ring_buffer>(rQueuingPortName.c_str())[1](aMaxNumMessages);
我遇到了大量无法理解的编译器错误，我不知道如何解决:
I get a plethora of unintelligible compiler errors that I do not quite know how to fix:
1>------ Build started: Project: apex, Configuration: Debug x64 ------
1>APEXManager.cpp
1>C:\Users\johnc\main\extlibs\boost_1_65_1\boost/container/string.hpp(216): error C2512: 'boost::interprocess::allocator<char,boost::interprocess::segment_manager<CharType,MemoryAlgorithm,IndexType>>::allocator': no appropriate default constructor available
1> with
1> [
1> CharType=char,
1> MemoryAlgorithm=boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,boost::interprocess::offset_ptr<void,ptrdiff_t,uintptr_t,0>,0>,
1> IndexType=boost::interprocess::iset_index
1> ]
1>C:\Users\johnc\main\extlibs\boost_1_65_1\boost/container/string.hpp(214): note: while compiling class template member function 'boost::container::container_detail::basic_string_base<Allocator>::members_holder::members_holder(void)'
1> with
1> [
1> Allocator=shm::char_alloc
1> ]
1>C:\Users\johnc\main\extlibs\boost_1_65_1\boost/container/string.hpp(100): note: see reference to function template instantiation 'boost::container::container_detail::basic_string_base<Allocator>::members_holder::members_holder(void)' being compiled
1> with
1> [
1> Allocator=shm::char_alloc
1> ]
1>C:\Users\johnc\main\extlibs\boost_1_65_1\boost/container/string.hpp(224): note: see reference to class template instantiation 'boost::container::container_detail::basic_string_base<Allocator>::members_holder' being compiled
1> with
1> [
1> Allocator=shm::char_alloc
1> ]
1>C:\Users\johnc\main\extlibs\boost_1_65_1\boost/container/string.hpp(506): note: see reference to class template instantiation 'boost::container::container_detail::basic_string_base<Allocator>' being compiled
1> with
1> [
1> Allocator=shm::char_alloc
1> ]
1>C:\Users\johnc\main\extlibs\boost_1_65_1\boost/lockfree/spsc_queue.hpp(557): note: see reference to class template instantiation 'boost::container::basic_string<char,std::char_traits<char>,shm::char_alloc>' being compiled
1>C:\Users\johnc\main\extlibs\boost_1_65_1\boost/lockfree/spsc_queue.hpp(555): note: while compiling class template member function 'boost::lockfree::detail::runtime_sized_ringbuffer<T,std::allocator<T>>::~runtime_sized_ringbuffer(void)'
1> with
1> [
1> T=shm::shared_string
1> ]
1>..\..\src\apex\APEXManager.cpp(660): note: see reference to function template instantiation 'boost::lockfree::detail::runtime_sized_ringbuffer<T,std::allocator<T>>::~runtime_sized_ringbuffer(void)' being compiled
1> with
1> [
1> T=shm::shared_string
1> ]
1>C:\Users\johnc\main\extlibs\boost_1_65_1\boost/lockfree/spsc_queue.hpp(693): note: see reference to class template instantiation 'boost::lockfree::detail::runtime_sized_ringbuffer<T,std::allocator<T>>' being compiled
1> with
1> [
1> T=shm::shared_string
1> ]
1>C:\Users\johnc\main\extlibs\boost_1_65_1\boost/interprocess/detail/in_place_interface.hpp(61): note: see reference to class template instantiation 'boost::lockfree::spsc_queue<shm::shared_string>' being compiled
1>C:\Users\johnc\main\extlibs\boost_1_65_1\boost/interprocess/detail/in_place_interface.hpp(58): note: while compiling class template member function 'void boost::interprocess::ipcdetail::placement_destroy<T>::destroy_n(void *,::size_t,size_t &)'
1> with
1> [
1> T=shm::ring_buffer
1> ]
1>C:\Users\johnc\main\extlibs\boost_1_65_1\boost/interprocess/detail/named_proxy.hpp(50): note: see reference to class template instantiation 'boost::interprocess::ipcdetail::placement_destroy<T>' being compiled
1> with
1> [
1> T=shm::ring_buffer
1> ]
1>C:\Users\johnc\main\extlibs\boost_1_65_1\boost/interprocess/detail/named_proxy.hpp(130): note: see reference to class template instantiation 'boost::interprocess::ipcdetail::CtorArgN<T,false,const MESSAGE_RANGE_TYPE &>' being compiled
1> with
1> [
1> T=shm::ring_buffer
1> ]
1>..\..\src\apex\APEXManager.cpp(365): note: see reference to function template instantiation 'T *boost::interprocess::ipcdetail::named_proxy<boost::interprocess::segment_manager<CharType,MemoryAlgorithm,IndexType>,T,false>::operator ()<const MESSAGE_RANGE_TYPE&>(const MESSAGE_RANGE_TYPE &) const' being compiled
1> with
1> [
1> T=shm::ring_buffer,
1> CharType=char,
1> MemoryAlgorithm=boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,boost::interprocess::offset_ptr<void,ptrdiff_t,uintptr_t,0>,0>,
1> IndexType=boost::interprocess::iset_index
1> ]
1>..\..\src\apex\APEXManager.cpp(365): note: see reference to function template instantiation 'T *boost::interprocess::ipcdetail::named_proxy<boost::interprocess::segment_manager<CharType,MemoryAlgorithm,IndexType>,T,false>::operator ()<const MESSAGE_RANGE_TYPE&>(const MESSAGE_RANGE_TYPE &) const' being compiled
1> with
1> [
1> T=shm::ring_buffer,
1> CharType=char,
1> MemoryAlgorithm=boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family,boost::interprocess::offset_ptr<void,ptrdiff_t,uintptr_t,0>,0>,
1> IndexType=boost::interprocess::iset_index
1> ]
1>Done building project "apex.vcxproj" -- FAILED.
========== Build: 0 succeeded, 1 failed, 2 up-to-date, 0 skipped ==========
推荐答案
动态大小仅在您指定进程间内存分配器时才有效:http://www.boost.org/doc/libs/1_65_1/doc/html/boost/lockfree/allocator.html.
Dynamic size will only work if you specify the interprocess-memory allocator: http://www.boost.org/doc/libs/1_65_1/doc/html/boost/lockfree/allocator.html.
遗憾的是，虽然 spsc_queue 确实支持有状态分配器:
Sadly, though spsc_queue does support stateful allocators:
定义分配器.boost.lockfree 支持有状态分配器并兼容 Boost.Interprocess 分配器
Defines the allocator. boost.lockfree supports stateful allocator and is compatible with Boost.Interprocess allocators
它似乎不支持 uses_allocator<> 协议将分配器向下传递给它的元素 (shared_string)，即使在使用 scoped_allocator_adaptor 时也不支持¹.
It does NOT seem to support the uses_allocator<> protocol required to pass the allocator down to it's element (shared_string), not even when using scoped_allocator_adaptor¹.
我之前遇到过这个:
共享内存 IPC 同步(无锁)(使用编译时大小的spsc_queue)
所以我的建议是去除其中一种成分:
So my suggestion would be to remove one of the ingredients:
- 使元素成为非动态容器(容器的容器总是需要作用域分配器感知或显式元素构造(
- 使队列固定大小(对于共享内存情况，这通常是一个好主意，IYAM)
- 添加一个间接层...
对于后者，您可以将队列存储为托管 bip::shared_ptr<shared_string>:
On the latter, you could make the queue store a manged bip::shared_ptr<shared_string> instead:
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/containers/string.hpp>
#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/managed_mapped_file.hpp> // for Coliru
#include <boost/interprocess/smart_ptr/shared_ptr.hpp>
#include <boost/lockfree/spsc_queue.hpp>
#include <iostream>
/// noisy - traces special members
struct noisy {
noisy& operator=(noisy&&) noexcept { std::cout << "operator=(noisy&&)\n"; return *this; }
noisy& operator=(const noisy&) { std::cout << "operator=(const noisy&)\n"; return *this; }
noisy(const noisy&) { std::cout << "noisy(const noisy&)\n"; }
noisy(noisy&&) noexcept { std::cout << "noisy(noisy&&)\n"; }
~noisy() { std::cout << "~noisy()\n"; }
noisy() { std::cout << "noisy()\n"; }
};
namespace bip = boost::interprocess;
namespace blf = boost::lockfree;
namespace Shared {
using Segment = bip::managed_mapped_file; // Coliru unsupported: managed_shared_memory;
using Manager = Segment::segment_manager;
template <typename T> using Alloc = bip::allocator<T, Manager>;
using String = bip::basic_string<char, std::char_traits<char>, Alloc<char> >;
// using Element = String;
// For debug/demonstration
struct Element : String, noisy { using String::String; }; // inherit constructors
using Ptr = bip::managed_shared_ptr<Element, Segment>::type;
using Buffer = blf::spsc_queue<Ptr, blf::allocator<Alloc<Ptr> > >;
}
static std::string unique_id_gen() {
static std::atomic_size_t s_gen { 0 };
return "buffer_element" + std::to_string(++s_gen);
}
int main() {
struct shm_remove {
shm_remove() { bip::shared_memory_object::remove("MySharedMemory"); }
~shm_remove() { bip::shared_memory_object::remove("MySharedMemory"); }
} remover;
Shared::Segment segment(bip::create_only, "MySharedMemory", 4 << 20);
auto& buffer = *segment.construct<Shared::Buffer>(bip::unique_instance)[1](20, segment.get_segment_manager());
auto create = [&segment](auto&&... args) {
return make_managed_shared_ptr(segment.construct<Shared::Element>(unique_id_gen().c_str())
(
std::forward<decltype(args)>(args)...,
segment.get_segment_manager()
), segment);
};
std::cout << "Pushing\n";
for (auto msg : { "hello", "world", "bye", "cruel", "world" })
buffer.push(create(msg));
std::cout << "Popping\n";
{
Shared::Ptr into;
while (buffer.pop(into)) {
std::cout << "Popped: '" << *into << "'\n";
}
std::cout << "Going out of scope\n";
} // RAII
std::cout << "Out of scope\n";
{
// make sure any other owned queue elements are freed if the queue is destroyed before it's empty:
for (auto msg : { "HELLO", "WORLD", "BYE", "CRUEL", "WORLD" })
buffer.push(create(msg));
std::cout << "Destroying buffer containing 5 elements\n";
segment.destroy<Shared::Buffer>(bip::unique_instance);
}
}
打印:
Pushing
noisy()
noisy()
noisy()
noisy()
noisy()
Popping
Popped: 'hello'
~noisy()
Popped: 'world'
~noisy()
Popped: 'bye'
~noisy()
Popped: 'cruel'
~noisy()
Popped: 'world'
Going out of scope
~noisy()
Out of scope
noisy()
noisy()
noisy()
noisy()
noisy()
Destroying buffer containing 5 elements
~noisy()
~noisy()
~noisy()
~noisy()
~noisy()
<小时>
¹ 搜索我的答案，了解如何将这些与共享内存中的其他容器容器一起使用
¹ search my answers on how to use these with other containers of containers in shared memory
这篇关于无法在共享内存中使用运行时大小参数构造运行时 boost spsc_queue的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
