如何在 Linux 上根据 CPU 能力进行运行时绑定 [英] How to do runtime binding based on CPU capabilities on linux

问题描述

是否可以让 linux 库(例如libloader.so")加载另一个库来解析任何外部符号?

Is it possible to have a linux library (e.g. "libloader.so") load another library to resolve any external symbols?

我有一大堆代码有条件地编译以支持 SIMD 级别(SSE2、AVX、AVX2).如果构建平台与运行时平台相同，这将正常工作.但它阻碍了跨代处理器的重用.

I've got a whole bunch of code that gets conditionally compiled for the SIMD level to be supported ( SSE2, AVX, AVX2 ). This works fine if the build platform is the same as the runtime platform. But it hinders reuse across different processor generations.

一个想法是让 executable 调用 function 链接到不直接实现 function 的 libloader.so>.相反，它从另一个加载的库中解析(绑定?)该符号，例如libimpl_sse2.so、libimpl_avx2.so 等取决于 cpuflags.

One thought is to have executable which calls function link to libloader.so that does not directly implement function. Rather, it resolves(binds?) that symbol from another loaded library e.g. libimpl_sse2.so, libimpl_avx2.so or so on depending on cpuflags.

有数百个函数需要以这种方式动态绑定，因此更改声明或调用代码是不切实际的.程序链接很容易改变.运行时环境变量也可以更改，但我不想这样做.

There are hundreds of functions that need to be dynamically bound in this way, so changing the declarations or calling code is not practical. The program linkage is fairly easy to change. The runtime environment variables could also be changed, but I'd prefer not to.

我已经通过 ld 标志 --unresolved-symbols=ignore-all 制作了一个可执行文件，该可执行文件以未解析的外部符号 (UES) 构建和启动.但是随后加载 impl lib 并不会将 UES 函数的值从 NULL 更改.

I've gotten as far as making an executable that builds and starts with unresolved external symbols (UES) via the ld flag --unresolved-symbols=ignore-all. But subsequent loading of the impl lib does not change the value of the UES function from NULL.

推荐答案

后来我发现下面描述的技术只能在有限的情况下工作.具体来说，您的共享库必须仅包含函数，而不能包含任何全局变量.如果要分派到的库中有全局变量，那么最终会出现运行时动态链接器错误.发生这种情况因为在调用共享库构造函数之前重新定位全局变量.因此，链接器需要在此处描述的调度方案有机会运行之前尽早解析这些引用.

I found out later on that the technique described below will only work under limited circumstances. Specifically, your shared libraries must contain functions only, without any global variables. If there are globals inside the libraries that you want to dispatch to, then you will end up with a runtime dynamic linker error. This occurs because global variables are relocated before shared library constructors are invoked. Thus, the linker needs to resolve those references early, before the dispatching scheme described here has a chance to run.

实现您想要的一种方法是(ab)使用共享库的 ELF 标头中的 DT_SONAME 字段.这可用于更改动态加载程序 (ld-linux-so*) 在运行时加载的文件的名称，以解析共享库依赖项.这最好用一个例子来解释.假设我使用以下命令行编译共享库 libtest.so:

One way of accomplishing what you want is to (ab)use the DT_SONAME field in your shared library's ELF header. This can be used to alter the name of the file that the dynamic loader (ld-linux-so*) loads at runtime in order to resolve the shared library dependency. This is best explained with an example. Say I compile a shared library libtest.so with the following command line:

g++ test.cc -shared -o libtest.so -Wl,-soname,libtest_dispatch.so

这将创建一个共享库，其文件名为libtest.so，但其DT_SONAME 字段设置为libtest_dispatch.so.让我们看看当我们将程序链接到它时会发生什么:

This will create a shared library whose filename is libtest.so, but its DT_SONAME field is set to libtest_dispatch.so. Let's see what happens when we link a program against it:

g++ testprog.cc -o test -ltest

让我们检查生成的应用程序二进制文件 test 的运行时库依赖项:

Let's examine the runtime library dependencies for the resulting application binary test:

> ldd test
linux-vdso.so.1 =>  (0x00007fffcc5fe000)
libtest_dispatch.so => not found
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd1e4a55000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd1e4e4f000)

请注意，动态加载器不是查找 libtest.so，而是希望加载 libtest_dispatch.so.您可以利用它来实现您想要的调度功能.这是我的做法:

Note that instead of looking for libtest.so, the dynamic loader instead wants to load libtest_dispatch.so instead. You can exploit this to implement the dispatching functionality that you want. Here's how I would do it:

• 创建不同版本的共享库.我假设有一些始终可以使用的通用"版本，并在运行时适当地使用其他优化版本.我会用普通"库名称 libtest.so 命名通用版本，并根据您的选择命名其他版本(例如 libtest_sse2.so、libtest_avx.so 等).

• Create the various versions of your shared library. I assume that there is some "generic" version that can always be used, with other optimized versions utilized at runtime as appropriate. I would name the generic version with the "plain" library name libtest.so, and name the others however you choose (e.g. libtest_sse2.so, libtest_avx.so, etc.).

在链接库的通用版本时，将其 DT_SONAME 覆盖为其他内容，例如 libtest_dispatch.so.

When linking the generic version of the library, override its DT_SONAME to something else, like libtest_dispatch.so.

创建一个名为 libtest_dispatch.so 的调度程序库.在应用程序启动时加载调度程序时，它负责加载库的适当实现.下面是 libtest_dispatch.so 实现的伪代码:

Create a dispatcher library called libtest_dispatch.so. When the dispatcher is loaded at application startup, it is responsible for loading the appropriate implementation of the library. Here's pseudocode for what the implementation of libtest_dispatch.so might look like:

#include <dlfcn.h>
#include <stdlib.h>

// the __attribute__ ensures that this function is called when the library is loaded
__attribute__((constructor)) void init()
{
    // manually load the appropriate shared library based upon what the CPU supports
    // at runtime
    if (avx_is_available) dlopen("libtest_avx.so", RTLD_NOW | RTLD_GLOBAL);
    else if (sse2_is_available) dlopen("libtest_sse2.so", RTLD_NOW | RTLD_GLOBAL);
    else dlopen("libtest.so", RTLD_NOW | RTLD_GLOBAL);
    // NOTE: this is just an example; you should check the return values from 
    // dlopen() above and handle errors accordingly
}

在将应用程序链接到您的库时，将其链接到vanilla"libtest.so，即覆盖了其 DT_SONAME 以指向调度程序库.这使得分派对于使用您的库的任何应用程序作者来说基本上是透明的.

When linking an application against your library, link it against the "vanilla" libtest.so, the one that has its DT_SONAME overridden to point to the dispatcher library. This makes the dispatching essentially transparent to any application authors that use your library.

这应该在 Linux 上按上述方式工作.在 Mac OS 上，共享库具有类似于 ELF 共享库中使用的 DT_SONAME 的安装名称"，因此可以使用与上述非常相似的过程来代替.我不确定是否可以在 Windows 上使用类似的东西.

This should work as described above on Linux. On Mac OS, shared libraries have an "install name" that is analogous to the DT_SONAME used in ELF shared libraries, so a process very similar to the above could be used instead. I'm not sure about whether something similar could be used on Windows.

注意:上面做出了一个重要假设:库的各种实现之间的 ABI 兼容性.也就是说，您的库应该设计为在链接时链接到最通用的版本是安全的，同时在运行时使用优化版本(例如 libtest_avx.so).

Note: There is one important assumption made in the above: ABI compatibility between the various implementations of the library. That is, your library should be designed such that it is safe to link against the most generic version at link time while using an optimized version (e.g. libtest_avx.so) at runtime.

这篇关于如何在 Linux 上根据 CPU 能力进行运行时绑定的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！