如何规避dlopen()缓存? [英] How to circumvent dlopen() caching?

问题描述

根据其手册页，dlopen()不会两次加载同一个库:

According to its man page, dlopen() will not load the same library twice:

如果使用dlopen()再次加载相同的共享对象，则相同返回对象句柄.动态链接器维护参考计数对象句柄，因此动态加载的共享对象是在多次调用dlclose()之前，不会释放它因为dlopen()已成功完成.任何初始化返回(请参阅下面)仅被调用一次.但是，随后的dlopen()调用使用RTLD_NOW加载同一共享库的对象可能会强制符号早期加载RTLD_LAZY的共享库的分辨率.

If the same shared object is loaded again with dlopen(), the same object handle is returned. The dynamic linker maintains reference counts for object handles, so a dynamically loaded shared object is not deallocated until dlclose() has been called on it as many times as dlopen() has succeeded on it. Any initialization returns (see below) are called just once. However, a subsequent dlopen() call that loads the same shared object with RTLD_NOW may force symbol resolution for a shared object earlier loaded with RTLD_LAZY.

(重点是我的).

但是究竟是什么决定了共享对象的身份呢?我试图研究代码，但没有走太远.是吗:

But what actually determines the identity of shared objects? I tried to look into the code, but did not come very far. Is it:

• 某种形式的规范化路径名(例如realpath?)
• inode吗?
• 藏书的内容?

我很确定我可以排除最后一点，因为实际的文件系统副本会产生两个不同的句柄.

I am pretty sure that I can rule out this last point, since an actual filesystem copy yields two different handles.

解释这个问题的动机:我正在使用一些具有静态全局变量的代码.我需要该代码的多个实例以线程安全的方式运行.我当前的方法是将所述代码编译并链接到动态库中，并多次加载该库.借助一些链接器魔术，它似乎可以创建全局变量的多个副本，并将每个库中的访问权限解析为其自己的副本.唯一的问题是，我的原型将生成的库复制了n次，同时使用了n次.这不仅有点丑陋，而且我还怀疑它可能会在其他平台上中断.

To explain the motivation behind this question: I am working with some code that has static global variables. I need multiple instances of that code to run in a thread-safe manner. My current approach is to compile and link said code into a dynamic library and load that library multiple times. With some linker magic, it appears to create several copies of the globals and resolve access in each library to its own copies. The only problem is that my prototype copies the generated library n times for n concurrent uses. This is not only somewhat ugly but I also suspect that it might break on a different platform.

那么根据POSIX标准， dlopen()的确切行为是什么?

So what is the exact behaviour of dlopen() according to the POSIX standard?

因为它出现在评论和答案中，所以不重构代码绝对不是一种选择.这将涉及数月甚至数年的工作，并且有可能一开始就牺牲使用代码的所有好处.存在一个正在进行的研究项目可能会以更简洁的方式解决此问题，但它是实际研究，可能会失败.我现在需要解决方案.

edit: Because it came up in a comment and an answer, no refactoring the code is definitely not an option. It would involve months or even years of work and potentially sacrifice all benefits of using the code in the first place. There exists an ongoing research project that might solve this problem in a much cleaner way, but it is actual research and might fail. I need a solution now.

edit2 :因为人们似乎仍然不相信用例实际上是有效的.我正在研究一种纯函数式语言，它将被嵌入到更大的C/C ++应用程序中.因为我需要一个带有垃圾收集器的原型，一个经过验证的类型检查器以及尽快的合理性能，所以我使用OCaml作为中间代码.现在，我将源模块编译为OCaml模块，将生成的目标代码(包括启动等)链接到具有OCaml运行时和dlopen() that 的共享库中.强>共享库.每个.so都有其自己的运行时副本，包括多个全局变量(例如，指向年轻一代的指针)，即应该完全正常.该库恰好公开了两个功能:一个初始化程序和一个单独的导出，可完成原始模块打算执行的任何操作.OCaml运行时的任何符号都不会导出/共享.当我加载库时，它的内部符号将按预期方式重定位，我现在唯一的问题是我实际上需要在运行时为作业的每个实例复制.so文件.

edit2: Because people still seem to not believe the usecase is actually valid. I am working on a pure functional language, that shall be embedded into a larger C/C++ application. Because I need a prototype with a garbage collector, a proven typechecker, and reasonable performance ASAP, I used OCaml as intermediate code. Right now, I am compiling a source module into an OCaml module, link the generated object code (including startup etc.) into a shared library with the OCaml runtime and dlopen() that shared library. Every .so has its own copy of the runtime, including several global variabels (e.g. the pointer to the young generation) and that is, or rather should be, totally fine. The library exposes exactly two functions: An initializer and a single export that does whatever the original module is intended to do. No symbols of the OCaml runtime are exported/shared. when I load the library, its internal symbols are relocated as expected, the only issue I have right now is that I actually need to copy the .so file for each instance of the job at runtime.

关于线程本地存储:这实际上是一个有趣的想法，因为对运行时的修改确实相当简单.但是问题在于OCaml编译器生成的机器代码，因为它无法发出tls符号的加载指令(还可以吗?).

Regarding thread-local-storage: That is actually an interesting idea, as the modification to the runtime is indeed rather simple. But the problem is the machine code generated by the OCaml compiler, as it cannot emit loading instructions for tls symbols (yet?).

推荐答案

POSIX说:

即使引用该文件多次调用了dlopen()，即使使用了不同的路径名来引用该文件，也仅将目标文件的单个副本带入地址空间.

Only a single copy of an object file is brought into the address space, even if dlopen() is invoked multiple times in reference to the file, and even if different pathnames are used to reference the file.

因此答案是"inode".复制库文件应该可以"，但硬链接不会.除了.由于它们将公开相同的全局符号，并且在发生这种情况时，所有(可移植性)的赌注都将关闭.您正处于错误定义行为的中间，这种行为是通过错误修复而不是好的设计而演变的.

So the answer is "inode". Copying the library file "should work", but hard links won't. Except. Since they will expose the same global symbols and when that happens all (portability) bets are off. You're in the middle of weakly defined behavior that has evolved through bug fixes rather than good design.

当您陷入困境时，请不要更深入.添加其他可怕的技巧以从根本上破坏图书馆的工作的方法只会导致额外的破坏.只需花费几个小时就可以修复该库，使其不使用全局变量，而不是花几天时间来破解动态链接(这将是不可移植的).

Don't dig deeper when you're in a hole. The approach to add additional horrible hacks to make a fundamentally broken library work just leads to additional breakage. Just spend a few hours to fix the library to not use globals instead of spending days to hack around dynamic linking (which will be unportable at best).

这篇关于如何规避dlopen()缓存?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！