在没有线程支持的程序加载的共享库中使用C ++ 11多线程 [英] Using C++11 multithreading in shared library loaded by program without thread support
问题描述
我目前正在尝试在共享库中使用C ++ 11多线程,该共享库已加载到Linux上的主程序(用C编写)中.这是大型仿真程序的一部分,我无法更改库加载或更改主程序的所有内容.
主程序是用gcc 4.1.2编译的,我没有源代码(我不能用gcc 4.8.2重新编译它).
共享库是使用gcc 4.8.2编译的,以便使用C ++ 11多线程.我正在传递编译器命令
<代码> -pthread -lpthread -std = c ++ 11
,如中所述在Linux下的GCC中使用std :: thread的正确链接选项是什么?
使用此配置(" -pthread -std = c ++ 11
"和gcc 4.8)编译独立的测试程序可以在我的系统上正常工作.但是,当我启动加载共享库的程序时,出现异常:
捕获到std :: exception!异常消息:启用多线程以使用std :: thread:不允许的操作正在终止...
使用 -pthread
和 -lpthread
(,并且仅使用 -pthread
而不使用-lpthread
)编译参数不起作用.编译器参数为(我正在使用Cook构建系统):
<代码> -pthread -std = c ++ 11 -fmessage-length = 0 -fPIC -Wchar-subscripts ...(此处有很多-W *)...-无用变量-m64 -D__64BIT__ -pthread -lpthread
和链接器参数(由于构建系统而有重复的参数):
<代码> -pthread -lpthread -std = c ++ 11 -pthread -lpthread -std = c ++ 11 -shared -fPIC -Wl,-Bsymbolic -Wl,--allow-shlib-undefined -pthread -lpthread
在我的库上调用ldd会得到以下输出
<代码> $ $ ldd calc3/build/amd64_linux26_RH5/library.solinux-vdso.so.1 =>(0x00007fff4d1fd000)libpthread.so.0 =>/lib64/libpthread.so.0(0x00002ae6ec124000)libstdc ++.so.6 =>/afs/bb/data/d6833/util/gcc_482/lib64/libstdc++.so.6(0x00002ae6ec340000)libm.so.6 =>/lib64/libm.so.6(0x00002ae6ec655000)libgcc_s.so.1 =>/afs/bb/data/d6833/util/gcc_482/lib64/libgcc_s.so.1(0x00002ae6ec8d8000)libc.so.6 =>/lib64/libc.so.6(0x00002ae6ecaef000)/lib64/ld-linux-x86-64.so.2(0x00000032cb400000)
并在主程序上
<代码> $ ldd .../bin-64/main_programlinux-vdso.so.1 =>(0x00007fff64595000)libdl.so.2 =>/lib64/libdl.so.2(0x00000032cc000000)libz.so.1 =>/usr/lib64/libz.so.1(0x00000032cc800000)libc.so.6 =>/lib64/libc.so.6(0x00000032cb800000)/lib64/ld-linux-x86-64.so.2(0x00000032cb400000)
pthread库链接到我的共享库,但没有链接到主程序.该答案指出您必须将pthread链接到主程序,但是对此答案的第二条评论(@R.)表示没有必要(从逻辑上讲).
不幸的是,除了我的库正在使用另一个C ++库作为API之外,我对整个系统的加载机制一无所知.
请注意,其他C ++ 11功能也可以工作(并且libstdc ++.so在我的库的依赖项中),但C ++ 11多线程处理不起作用(尽管libpthread.so也在我的库的依赖项中)./p>
使用程序本身包含的库中的线程类是可行的(并且该线程类似乎也使用pthreads).
我还尝试使用 -fabi-version = 0
或 -fabi-version = 2
,因为主程序是在我的库中使用gcc 4.1.2编译的但它并没有改变任何东西.
是否有我忽略的东西或可以用来使其工作的编译器选项?还是我的程序环境有问题?任何想法都欢迎.
我尝试使用 -Wl,-no-ne-aseded
(如注释中所建议),但不幸的是它没有任何改变.
使用clang 3.5代替gcc 4.8也不起作用.
只要我对主应用程序和共享库都使用gcc 4.8或clang 3.5,创建一个加载共享库的小型测试应用程序(就像在@chill中的答案)就可以工作(即使没有编译器标志).但是,当将gcc 4.1用于主程序时,主程序甚至无法加载该库(该库在我的实际"应用程序中运行).我认为编译器的不同ABI可能有问题.
直接从 pthread.h
使用pthread似乎可行(尽管该程序当前在 pthread_join
上终止,但没有错误消息,但我仍在此处进行测试...)
使用 LD_LIBRARY_PATH = $(pwd):$ LD_LIBRARY_PATH
运行测试程序"(因为gcc 4.8库路径也必须位于其中,谢谢@MvG)确实运行了程序但崩溃了再次使用启用多线程以使用std :: thread:不允许操作
异常.
我检查了所有其他已加载的库(使用 strace ./main_program 2>& 1 | grep'^ open(.* \.so"'
找到了它们[请参阅此处]),并使用 ldd
对其全部进行检查.它们都依赖于相同的库(具有相同的路径)). ldd
输出(全部输出):
linux-vdso.so.1 =>(0x00007fff4d3fd000)libstdc ++.so.6 =>/afs/bb/data/d6833/util/gcc_482/lib64/libstdc++.so.6(0x00002ade28774000)libm.so.6 =>/lib64/libm.so.6(0x00002ade28ab0000)libgcc_s.so.1 =>/afs/bb/data/d6833/util/gcc_482/lib64/libgcc_s.so.1(0x00002ade28d33000)libc.so.6 =>/lib64/libc.so.6(0x00002ade28f49000)/lib64/ld-linux-x86-64.so.2(0x00000032ea200000)
(除了我的库和其他库之外,它们都不依赖libpthread.so.0(但它是相同的/lib64/libpthread.so.0
))
某些库确实有更多的依赖项(似乎与线程无关),但是似乎没有任何冲突"依赖项(在任何这些库中都没有对同一个库的不同版本/路径的依赖项)库).
在 __ gthread_active_p
返回false.该调用仅检查给定符号是否可用.有问题的符号是一个弱符号:不必存在,但是要检查它的存在以决定是否支持线程.
但是符号的存在是什么意思?在这种情况下,这意味着该符号位于符号表的列表中,所涉及的库(在我的情况下为 libgcc_s.so.1
)将在其中搜索符号定义.其中包括应用程序本身导出的符号,还包括之前加载的所有库导出的符号.但是,它不包括之后加载的库.不幸的是,如果在之前 libpthread
之前加载了 libgcc
,则该符号在其搜索域中 not 不可用.因此,它将线程报告为不受支持.我猜您在多线程模块之前已经加载了其他一些C ++模块,所以您会遇到此问题.
一种在我的复制示例中起作用的解决方案是设置 LD_PRELOAD =/lib64/用于调用二进制文件的环境中的libpthread.so.0
.这会预先加载 libpthread
,因此可以使用其符号来满足弱符号链接.这不适用于setuid/setgid二进制文件,在其他情况下也可能被认为是丑陋的,所以我对更清洁的解决方案感兴趣.尽管如此,这大部分时间都能完成工作.
I am currently trying to use C++11 multithreading in a shared library that is loaded into the main program (written in C) on Linux. This is part of a big simulation program and I cannot change anything about the library loading or change the main program in general.
The main program is compiled with gcc 4.1.2 and I don't have the sources for it (I cannot recompile it with gcc 4.8.2).
The shared library is compiled with gcc 4.8.2 in order to use C++11 multithreading. I am passing the compiler commands
-pthread -lpthread -std=c++11
as explained in What is the correct link options to use std::thread in GCC under linux?
Compiling a standalone test program with this configuration ("-pthread -std=c++11
" and gcc 4.8) works correctly on my system. But when I start the program loading the shared library I get an exception:
Caught std::exception!
Exception Message: Enable multithreading to use std::thread: Operation not permitted
Terminating...
Using the -pthread
and -lpthread
(Edit: and also only -pthread
without -lpthread
) compile parameter doesn't work.
The compiler arguments are (I am using the cook build system):
-pthread -std=c++11 -fmessage-length=0 -fPIC -Wchar-subscripts ...(lots of -W* here)
... -Wunused-variable -m64 -D__64BIT__ -pthread -lpthread
and the linker arguments (duplicate parameters due to the build system):
-pthread -lpthread -std=c++11 -pthread -lpthread -std=c++11 -shared -fPIC -Wl,-Bsymbolic -Wl,--allow-shlib-undefined -pthread -lpthread
calling ldd on my library gives the following output
$ ldd calc3/build/amd64_linux26_RH5/library.so
linux-vdso.so.1 => (0x00007fff4d1fd000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00002ae6ec124000)
libstdc++.so.6 => /afs/bb/data/d6833/util/gcc_482/lib64/libstdc++.so.6 (0x00002ae6ec340000)
libm.so.6 => /lib64/libm.so.6 (0x00002ae6ec655000)
libgcc_s.so.1 => /afs/bb/data/d6833/util/gcc_482/lib64/libgcc_s.so.1 (0x00002ae6ec8d8000)
libc.so.6 => /lib64/libc.so.6 (0x00002ae6ecaef000)
/lib64/ld-linux-x86-64.so.2 (0x00000032cb400000)
and on the main program
$ ldd .../bin-64/main_program
linux-vdso.so.1 => (0x00007fff64595000)
libdl.so.2 => /lib64/libdl.so.2 (0x00000032cc000000)
libz.so.1 => /usr/lib64/libz.so.1 (0x00000032cc800000)
libc.so.6 => /lib64/libc.so.6 (0x00000032cb800000)
/lib64/ld-linux-x86-64.so.2 (0x00000032cb400000)
The pthread library is linked to my shared library but not to the main program. This answer states that you have to link pthreads to the main program but the 2nd comment on this answer (by @R..) says it isn't necessary (which sounds logically).
Unfortunately I don't know anything about the loading mechanics of the whole system, except that my library is using another C++ library as API.
Note that other C++11 features do work (and libstdc++.so is in the dependencies of my library) but C++11 multithreading is not (although libpthread.so is also in the dependencies of my library).
Using a threading class from a library contained in the program itself is working (and this thread class seems to use pthreads too).
I have also tried to use -fabi-version=0
or -fabi-version=2
because the main program is compiled with gcc 4.1.2 with my library but it didn't change anything.
Is there anything I have overlooked or a compiler option I can use to make it work? Or does it seem to be a problem of my program environment? Any ideas are welcome.
Edit:
I tried using -Wl,-no-as-needed
(as suggested in the comments) but it unfortunately didn't change anything.
Using clang 3.5 instead of gcc 4.8 also didn't work.
Creating a small test application which loads a shared library (like in the answer below by @chill) works (even without the compiler flag) as long as I use gcc 4.8 or clang 3.5 for both main application and shared library. When using gcc 4.1 for the main program however the main program even fails to load the library (which is working in my 'real' application). I think there might be a problem with the differen ABIs of the compilers.
Using pthreads directly from pthread.h
seems to work (although the program currently terminates on pthread_join
without error message, but I'm still testing there...)
Edit 2:
Running the 'test program' with LD_LIBRARY_PATH=$(pwd):$LD_LIBRARY_PATH
(because the gcc 4.8 library paths need to be in there too, thanks @MvG) did run the program but crashed again with the Enable multithreading to use std::thread: Operation not permitted
exception.
I checked all other libraries that are loaded (found them with strace ./main_program 2>&1 | grep '^open(".*\.so"'
[see here]) and checked them all with ldd
. They all depend on the same libraries (with the same paths). ldd
outputs (on all of them):
linux-vdso.so.1 => (0x00007fff4d3fd000)
libstdc++.so.6 => /afs/bb/data/d6833/util/gcc_482/lib64/libstdc++.so.6 (0x00002ade28774000)
libm.so.6 => /lib64/libm.so.6 (0x00002ade28ab0000)
libgcc_s.so.1 => /afs/bb/data/d6833/util/gcc_482/lib64/libgcc_s.so.1 (0x00002ade28d33000)
libc.so.6 => /lib64/libc.so.6 (0x00002ade28f49000)
/lib64/ld-linux-x86-64.so.2 (0x00000032ea200000)
(that they all don't depend on libpthread.so.0 except my library and one other (but it's the same /lib64/libpthread.so.0
))
Some libraries do have more dependencies (which don't seem thread-related) but there don't seem to be any 'conflicting' dependencies (there are no dependencies to different versions/paths of the same library in any of these libraries).
In thread.cc
you can read that this exception is generated if __gthread_active_p
returns false. That call simply checks whether or not a given symbol is available. The symbol in question is a weak symbol: it does not have to be present, but its presence is checked to decide whether or not threading is supported.
But what does presence of a symbol mean? In this case it means that the symbol is in the list of symbol tables which the library in question (libgcc_s.so.1
in my case) searches for symbol definitions. That includes symbols exported by the application itself, but also symbols exported by all the libraries loaded before it. However, it does not include libraries loaded afterwards. Unfortunately, if libgcc
is loaded before libpthread
, then the symbol is not available on its search domain. So it reports threading as unsupported. I guess you have some other C++ module loaded before the multi-threaded one, so you encounter this problem.
One solution which works in my reproducing example is setting LD_PRELOAD=/lib64/libpthread.so.0
in the environment used to call the binary. That loads libpthread
up front, so its symbols are available to satisfy the weak symbol linkage. This won't work for setuid/setgid binaries, and might be considered an ugly hack in other cases as well, so I'm interested in cleaner solutions. Nevertheless, this gets the job done most of the time.
这篇关于在没有线程支持的程序加载的共享库中使用C ++ 11多线程的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!