gfortran将c库与conda链接 [英] gfortran linking c libraries with conda

问题描述

我试图在没有超级用户权限的Ubuntu 18.04服务器上使用conda编译C ++/Fortran程序.

I am trying to compile a C++/Fortran program using conda on an Ubuntu 18.04 server where I do not have superuser rights.

我能够在Ubuntu 18.04 PC上使用相同的源代码正确编译程序(也使用conda)，但是在服务器上却遇到了很多错误.此刻，我陷入了找不到库-未定义参考"的困境.错误:

I am able to compile correctly the program with the same source code on my Ubuntu 18.04 PC (using conda too), but on the server I get a bunch of errors. At the moment, I am stuck with a "library not found - undefined reference" error:

gfortran -o glm -Wl,--export-dynamic obj/glm_globals.o obj/glm_util.o obj/glm_csv.o obj/glm_mobl.o obj/glm_mixu.o obj/glm_wqual.o obj/glm_layers.o obj/glm_surface.o obj/glm_input.o obj/glm_plot.o obj/glm_output.o obj/glm_ncdf.o obj/glm_lnum.o obj/glm_init.o obj/glm_flow.o obj/glm_mixer.o obj/glm_deep.o obj/glm_stress.o obj/glm_bird.o obj/glm_model.o obj/glm_types.o obj/glm_const.o obj/glm_debug.o obj/glm_main.o obj/glm_zones.o -L/local/XXX/my_name/PycharmProjects/glm/glm_source/GLM/../libutil/lib -lutil -L/local/XXX/my_name/anaconda3/lib -lnetcdf -L/usr/lib -L/local/XXX/my_name/PycharmProjects/glm/glm_source/GLM/../libplot/lib -lplot -lgd -lpng -ljpeg -lm -lX11    -lgfortran

/local/XXX/my_name/anaconda3/bin/ld: warning: libpthread.so.0, needed by /local/XXX/my_name/anaconda3/lib/libnetcdf.so, not found (try using -rpath or -rpath-link)
/local/XXX/my_name/anaconda3/bin/ld: warning: libxcb.so.1, needed by /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libX11.so, not found (try using -rpath or -rpath-link)
/local/XXX/my_name/anaconda3/bin/ld: warning: libdl.so.2, needed by /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libX11.so, not found (try using -rpath or -rpath-link)
/local/XXX/my_name/anaconda3/bin/ld: warning: librt.so.1, needed by /local/XXX/my_name/anaconda3/lib/./libhdf5_hl.so.100, not found (try using -rpath or -rpath-link)
/local/XXX/my_name/anaconda3/bin/ld: warning: libresolv.so.2, needed by /local/XXX/my_name/anaconda3/lib/././libgssapi_krb5.so.2, not found (try using -rpath or -rpath-link)
/local/XXX/my_name/anaconda3/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libX11.so: undefined reference to `dlopen@GLIBC_2.2.5'
/local/XXX/my_name/anaconda3/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libX11.so: undefined reference to `xcb_wait_for_event'
/local/XXX/my_name/anaconda3/bin/ld: /local/XXX/my_name/anaconda3/lib/././libkrb5.so.3: undefined reference to `__res_nsearch@GLIBC_2.2.5'
...

collect2: error: ld returned 1 exit status
Makefile:277: recipe for target 'glm' failed
make: *** [glm] Error 1

我可以看到，链接器找不到的共享库libpthread.so.0，librt.so.1，libdl.so.2，libresolv.so.2是glibc的一部分.因此，我认为链接到该库时会出现某种问题.

I can see the shared libraries libpthread.so.0, librt.so.1, libdl.so.2, libresolv.so.2 not found by the linker are part of glibc. So I assume there is some kind of problem linking to that library.

在先前的尝试中，我尝试使用conda安装glibc.这导致conda崩溃，并显示分段错误(核心已转储)".当我尝试编译时.我必须重新安装anaconda才能使conda再次工作.

In a previous attempt I tried installing glibc with conda. This resulted in conda crashing with a "segmentation fault (core dumped)" when I tried to compile. I had to reinstall anaconda to make conda work again.

在最后几天，我还尝试按照警告中的建议添加-rpath，添加LD_LIBRARY_PATH，使用-L添加包含共享库的目录.什么都没用.

In the last days I also have tried to add -rpath as suggested in the warnings, to add an LD_LIBRARY_PATH, to add the directory containing the shared libraries with -L. Nothing worked.

此刻我感到很失落.知道可能出什么问题了吗?

I feel quite lost at the moment. Any idea of what could have gone wrong?

PS:在计算机和服务器上进行编译之间的区别在于，我在计算机上的系统中安装了缺少的库，但是在服务器的conda上安装了丢失的库.因此，在服务器上进行编译时，我必须使用-L添加这些库的位置.

PS: The difference between compiling on my computer and on the server is that I installed missing libraries in the system on my computer, but on conda on the server. Thus, I had to add the location of those libraries with -L when compiling on the server.

推荐答案

此答案是建议的替代工作流程，旨在避免出现问题，而不是准确诊断OP中的问题.

This answer is a suggested alternative workflow that aims at avoiding the problem rather than an exact diagnosis of the issue in OP.

根据我的经验，我发现Conda Forge编译器软件包有助于简化自定义环境的创建和使用.例如，这是我用来构建的环境的YAML定义.kallisto 软件.对于您的情况，我希望类似

In my experience, I have found the Conda Forge compiler packages help simplify the creation and use of custom environments for compilation. As an example, here is the YAML definition for the environment I use to build the kallisto software. For your case, I would expect something like

fortran-compiler.yaml

name: fortran-compiler  # name it what you want
channels:
  - conda-forge
  - defaults
dependencies:
  - fortran-compiler
  - cxx-compiler
  - libpng
  - libgd
  - jpeg
  - libnetcdf
  - openlibm
  - xorg-libx11
# include the other required libraries

创建环境

conda env create -f fortran-compiler.yaml

然后在激活此环境的情况下运行编译.激活环境应自动管理链接器的外观，以便可以找到环境中的库(部分是Conda Forge *-compiler 软件包提供的内容).想法是让尽可能多的库来自环境本身.

then run your compilation with this environment activated. Activating the environment should automatically manage where the linker will look so that the libraries in the environment will be found (this is, in part, what the Conda Forge *-compiler packages provide). The idea is to have as many libraries as possible come from the environment itself.

我发现这种方法使手动定位和包括库路径的数量降至最低.但是，它确实需要跟踪Conda Forge中的库(例如，搜索libx11 )，但不幸的是，它并不总是从库名到包名的一对一映射.优势在于，以这种方式定义的编译环境可以促进跨平台的传输-例如，我为Kallisto编写的示例YAML在 osx-64 和 linux-64 上均可工作无需任何更改-因为它明确定义了Conda将提供共享库.

I find this approach keeps the amount of manual locating and including of library paths to a minimum. However, it does require having to track down the libraries in Conda Forge (e.g., search for libx11), which unfortunately is not always a one-to-one mapping from library name to package name. The advantage is that a compilation environment defined in this way can facilitate transferring across platforms - e.g., the example YAML I have for kallisto works on both osx-64 and linux-64 without any changes - because it explicitly defines that Conda will provide the shared libraries.

也许有更简单的方法可以做到这一点，但这至少是我在尝试使用 gcc 或 clang Anaconda遇到了许多不良经验之后发现的直接打包.

Perhaps there are simpler ways to do this, but this is at least what I've found works after having multiple bad experiences trying to use the gcc or clang Anaconda packages directly.

这篇关于gfortran将c库与conda链接的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！