将ctypes与由intel fortran编译器编译的fortran dll一起使用并链接到intel的mkl库时，缺少dll依赖项 [英] Dll dependencies missing when using ctypes with fortran dll compiled with intel fortran compiler and linking to intel's mkl library

问题描述

我在Windows下. mkl_example.f 中的fortran代码:

I am under windows. The fortran code in mkl_example.f :

    subroutine matmultmkl(M1, M2, M3, M, N, K) bind(c, name='matmultmkl')
        !DEC$ ATTRIBUTES DLLEXPORT :: matmultmkl
        use iso_c_binding, only: c_float, c_int
    
        integer(c_int),intent(in) :: M, N, K
        real(c_float), intent(in) :: M1(M, N), M2(N, K)
        real(c_float), intent(out):: M3(M, K)

        CALL DGEMM('N','N',M,K,N,1.,M1,M,M2,N,0.,M3,M)
    
    end subroutine 

我使用以下批处理文件通过命令行(在之前运行 compilervars.bat 的cmd窗口中)进行编译:

I compile it with command line (in a cmd windows where I ran compilervars.bat before) using the following batch file :

@Echo off

setlocal ENABLEDELAYEDEXPANSION
SET "IFORT_INITIAL_FLAGS=-c -fpp"
SET "IFORT_OPTIMIZATION_FLAGS=/O3"

ifort %IFORT_OPTIMIZATION_FLAGS% %IFORT_INITIAL_FLAGS% /I"C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2020.4.311\windows\mkl\include" -o mkl_example.obj mkl_example.f
ifort -dll -o mylib.dll mkl_example.obj /link /LIBPATH:"C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2020.4.311\windows\mkl\lib\intel64_win" mkl_intel_lp64.lib mkl_intel_thread.lib mkl_core.lib libiomp5md.lib 

然后我运行以下python脚本 mkl_example.py :

Then I run the follow python script mkl_example.py :

from ctypes import *
import time

import os
os.add_dll_directory("C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2020.4.311/windows/mkl/lib/intel64_win")
os.add_dll_directory("C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2020.4.311/windows/compiler/lib/intel64_win")

import numpy as np

mylib = CDLL('./mylib.dll')

mylib.matmultmkl.argtypes = [ POINTER(c_float), 
                                POINTER(c_float), 
                                POINTER(c_float),
                                POINTER(c_int),
                                POINTER(c_int),
                                POINTER(c_int) ]
mylib.matmultmkl.restype = None
M=1000
N=1000
K=1000

a = np.empty((M,N), dtype=c_float)
b = np.empty((N,K), dtype=c_float)
c = np.empty((M,K), dtype=c_float)

a[:] = np.random.rand(M,N)
b[:] = np.random.rand(N,K)

# Fortran mkl call
start = time.time()
mylib.matmultmkl( a.ctypes.data_as(POINTER(c_float)), 
                    b.ctypes.data_as(POINTER(c_float)), 
                    c.ctypes.data_as(POINTER(c_float)), 
                    c_int(M), c_int(N), c_int(K) )
stop = time.time()
print(f"Fortran mkl \t {stop - start}s")

输出为:

Traceback (most recent call last):
  File "path\to\mkl_example.py", line 14, in <module>
    mylib = CDLL('./mylib.dll')
  File "C:\PYTHON\3\3.8.2_64bits\lib\ctypes\__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: path\to\mylib.dll' (or one of its dependencies). Try using 
the full path with constructor syntax.

如果y删除了所有的mkl内容并自己编写了一个函数，则python在运行时会找到 mylib.dll .因此，在我链接到mkl的情况下，该消息并不表示即使 mylib.dll 位于正确的位置，由于某些奇怪的原因也无法在运行时找到该消息，但实际上找不到它的依赖项.

If y remove all the mkl stuff and code a function myself, the mylib.dll is found at runtime by the python. Hence in the case where I link to the mkl, the message doesn't mean that even if mylib.dll is at the right place it is not found at runtime for some weird reason, but thet indeed one of it's dependencies is not found.

除了将 all 依赖项放在fortran dll和python文件旁边的同一文件夹中之外，我什么也看不到.但是，如果这是 only 的方式，也许我宁愿绝对退出.

I don't see what I can do except finding all dependencies at putting them in the same folder next to the fortran dll and the python file. But if this is the only way, maybe I prefer quitting definitely.

推荐答案

在python脚本中添加到DLL搜索路径的路径用于包含编译器和MKL的静态和导入库的目录，而不是运行时DLL

The paths being added to the DLL search path in the python script are for directories that contain the static and import libraries for the compiler and MKL, not the runtime DLLs.

(安装Intel编译器通常会将编译器运行时安装在 Common files 目录中，并将其添加到系统搜索路径中-因此，该路径很可能会找到编译器运行时DLL-但这不适用于MKL运行时DLL.)

(Installation of the Intel compiler typically installs the compiler runtime in a Common files directory, and adds that to the system search path - so the compiler runtime DLLs are likely being found by that route - but this does not apply to the MKL runtime DLLs.)

在 os.add_dll_directory 调用中使用正确的目录.类似于(您需要检查安装) C:\ Program Files(x86)\ IntelSWTools \ compilers_and_libraries_xxxx \ windows \ redist \ intel64_win \ mkl .与版本无关的符号链接目录也可能更适合，具体取决于您的需求.

Use the correct directory in the os.add_dll_directory call. It will be something like (you need to check your installation) C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_xxxx\windows\redist\intel64_win\mkl. Version agnostic symlinked directories may also be more suitable, depending on your needs.

(要在没有安装编译器的情况下部署到另一台计算机上，将需要有关如何部署依赖项的策略，但这是一个更大的话题.)

(Deployment to another machine without the compiler installed will require a strategy around how you deploy dependencies, but this is a much larger topic.)

这篇关于将ctypes与由intel fortran编译器编译的fortran dll一起使用并链接到intel的mkl库时，缺少dll依赖项的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！