f2py链接Quadmath库?使用ctypes作为fortran包装器? [英] f2py linking quadmath libraries? Use ctypes for fortran wrapper instead?
问题描述
更新11/23/2019: 最初的问题是，为什么我不能让f2py用于简单的fortran包装器.我的答案"(如下)是改用ctypes.
Update 11/23/2019: This started out as a question about why I could not get f2py to work for a simple fortran wrapper. My "answer" (below) is to use ctypes instead.
原始帖子: 最近三天，我一直在尝试使用f2py将fortran与python接口.我正在使用cygwin和mingw在Windows上工作.这篇文章是关于使用cygwin的，但是我担心两者之间的冲突. 这是源文件multxy.f90:
Original post: I have spent the last three days trying to use f2py to interface fortran to python. I am working on windows using both cygwin and mingw. This post is about using cygwin, but I'm concerned about conflicts between the two. Here is the source multxy.f90:
subroutine multxy(x,y,z)
integer, parameter :: flt = selected_real_kind(15)
real(flt), intent(in) :: x,y
real(flt), intent(out) :: z
write(*,'(a,3g12.5)')'multxy'
write(*,'(a,3g12.5)')'multxy',x,y,x*y
z = x*y
end subroutine multxy
如果我像看过的所有示例一样运行f2py:
f2py -m multxyC -c multxy.f90
产生链接错误.在成吨的产量中，我对strtoflt128有所抱怨.这是gnu gcc quad数学库中的函数.我的flt类型等效于C中的double.代码中没有四边形数学，为什么要使用它?我试图创建一个扩展文件以查看它是否可以告诉我任何内容，但它仍尝试链接，因此未生成(或保存)任何代码.
至此，我使用strtofl128构建了一个小型c程序.如果我包含Quadmath库，它将编译并链接:gcc main.c -lquadmath
它无法正常运行，但是稍后我会对此进行研究.
我相信实际的四方库文件是…./cwin/lib/gcc/x86_64-pc-cygwin/8.3.0/libquadmath.a.运行f2py的输出显示此目录，因此您会认为它将链接库.我尝试使用-l选项的各种变体运行f2py.通常它会抱怨找不到库.我可以使用一种变体，但是链接仍然失败.
我还使用了整数函数:
If I run f2py like all the examples I’ve seen:
f2py -m multxyC -c multxy.f90
It produces a link error. Amongst the tons of output, I get complaints about strtoflt128. This is a function in the gnu gcc quad math library. My flt kind is equivalent to a double in C. There is no quad math in the code, so why is it used? I tried to create an extension file to see if it would tell me anything, but it still tried to link, so no code was produced (or saved).
At this point, I built a small c program using strtofl128. It compiles and links if I include the quadmath library: gcc main.c -lquadmath
It doesn’t run correctly, but that is something I'll look into later.
I believe the actual quadmath library file is …./cwin/lib/gcc/x86_64-pc-cygwin/8.3.0/libquadmath.a. The output from running f2py shows this directory, so you would think it would link the library. I have tried running f2py with all sorts of variations of the -l option. Usually it would complain about no library found. I got one variant to work, but the link still failed.
I’ve also worked with an integer function:
integer function intfunc(n)
integer, intent(in) :: n
intfunc = n*n
end function intfunc
我已经能够创建一个dll并从python调用它，而无需使用f2py.它似乎通过f2py运行就好了.它创建一个文件intfunc.cp37-win_amd64.pyd和一个目录，该目录包含一个带有巨大名称的.dll.我还无法从python调用它，但我会继续努力.
I have been able to create a dll and call it from python without using f2py. It seem to run through f2py just fine. It creates a file intfunc.cp37-win_amd64.pyd and a directory which contains a .dll with a huge name. I haven’t been able to call it from python yet, but I’ll keep working on that.
问题:
- 为什么f2py使用四精度?有避免使用它的选项吗?
- 如何包含正确的库?
- 如何在不调用链接程序的情况下创建C扩展代码?
- Cygwin和MinGW之间可能存在冲突吗?如何避免?
- f2py准备好迎接黄金时段了吗?使用C扩展名进行手动界面会更简单吗?
- f2py是否可以处理fortran函数?建议在几个地方使用子例程?
f2py被吹捧为易于使用.也许是这样，但是只有在您使其正常工作之后才可以.例如，我从事不同语言的交流已有25年了. c/c ++，fortran，excel/vba，matlab/octave，几乎没有这么多困难.这似乎是一个程序，它隐藏了很多功能，因此很难进行故障排除.
f2py is touted as easy to use. Perhaps it is, but only after you get it to work properly. I have been interfacing different languages for 25 years, eg. c/c++, fortran, excel/vba, matlab/octave, without nearly this much difficulty. This seems to be a program which does a lot which is hidden under the covers and that makes it difficult to troubleshoot.
一个问题是，运行f2py的输出与核心转储一样容易读取.运行时的输出是:
One issue is that the output from running f2py is about as easy to read as a core dump. The output when I run it is:
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building extension "multxyC" sources
f2py options: []
f2py:> C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\multxyCmodule.c
creating C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7
Reading fortran codes...
Reading file 'multxy.f90' (format:free)
Post-processing...
Block: multxyC
Block: multxy
Post-processing (stage 2)...
Building modules...
Building module "multxyC"...
Constructing wrapper function "multxy"...
z = multxy(x,y)
Wrote C/API module "multxyC" to file "C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\multxyCmodule.c"
adding 'C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\fortranobject.c' to sources.
adding 'C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7' to include_dirs.
copying c:\program files\python37\lib\site-packages\numpy\f2py\src\fortranobject.c -> C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7
copying c:\program files\python37\lib\site-packages\numpy\f2py\src\fortranobject.h -> C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7
build_src: building npy-pkg config files
running build_ext
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
customize MSVCCompiler using build_ext
get_default_fcompiler: matching types: '['gnu', 'intelv', 'absoft', 'compaqv', 'intelev', 'gnu95', 'g95', 'intelvem', 'intelem', 'flang']'
customize GnuFCompiler
Could not locate executable g77
Could not locate executable f77
customize IntelVisualFCompiler
Could not locate executable ifort
Could not locate executable ifl
customize AbsoftFCompiler
Could not locate executable f90
customize CompaqVisualFCompiler
Found executable C:\cwin\bin\DF.exe
customize IntelItaniumVisualFCompiler
Could not locate executable efl
customize Gnu95FCompiler
Found executable C:\cwin\bin\gfortran.exe
Using built-in specs.
COLLECT_GCC=/usr/bin/gfortran
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-pc-cygwin/8.3.0/lto-wrapper.exe
Target: x86_64-pc-cygwin
Configured with: /cygdrive/i/szsz/tmpp/gcc/gcc-8.3.0-1.x86_64/src/gcc-8.3.0/configure --srcdir=/cygdrive/i/szsz/tmpp/gcc/gcc-8.3.0-1.x86_64/src/gcc-8.3.0 --prefix=/usr --exec-prefix=/usr --localstatedir=/var --sysconfdir=/etc --docdir=/usr/share/doc/gcc --htmldir=/usr/share/doc/gcc/html -C --build=x86_64-pc-cygwin --host=x86_64-pc-cygwin --target=x86_64-pc-cygwin --without-libiconv-prefix --without-libintl-prefix --libexecdir=/usr/lib --enable-shared --enable-shared-libgcc --enable-static --enable-version-specific-runtime-libs --enable-bootstrap --enable-__cxa_atexit --with-dwarf2 --with-tune=generic --enable-languages=ada,c,c++,fortran,lto,objc,obj-c++ --enable-graphite --enable-threads=posix --enable-libatomic --enable-libgomp --enable-libitm --enable-libquadmath --enable-libquadmath-support --disable-libssp --enable-libada --disable-symvers --with-gnu-ld --with-gnu-as --with-cloog-include=/usr/include/cloog-isl --without-libiconv-prefix --without-libintl-prefix --with-system-zlib --enable-linker-build-id --with-default-libstdcxx-abi=gcc4-compatible --enable-libstdcxx-filesystem-ts
Thread model: posix
gcc version 8.3.0 (GCC)
customize Gnu95FCompiler
Using built-in specs.
COLLECT_GCC=/usr/bin/gfortran
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-pc-cygwin/8.3.0/lto-wrapper.exe
Target: x86_64-pc-cygwin
Configured with: /cygdrive/i/szsz/tmpp/gcc/gcc-8.3.0-1.x86_64/src/gcc-8.3.0/configure --srcdir=/cygdrive/i/szsz/tmpp/gcc/gcc-8.3.0-1.x86_64/src/gcc-8.3.0 --prefix=/usr --exec-prefix=/usr --localstatedir=/var --sysconfdir=/etc --docdir=/usr/share/doc/gcc --htmldir=/usr/share/doc/gcc/html -C --build=x86_64-pc-cygwin --host=x86_64-pc-cygwin --target=x86_64-pc-cygwin --without-libiconv-prefix --without-libintl-prefix --libexecdir=/usr/lib --enable-shared --enable-shared-libgcc --enable-static --enable-version-specific-runtime-libs --enable-bootstrap --enable-__cxa_atexit --with-dwarf2 --with-tune=generic --enable-languages=ada,c,c++,fortran,lto,objc,obj-c++ --enable-graphite --enable-threads=posix --enable-libatomic --enable-libgomp --enable-libitm --enable-libquadmath --enable-libquadmath-support --disable-libssp --enable-libada --disable-symvers --with-gnu-ld --with-gnu-as --with-cloog-include=/usr/include/cloog-isl --without-libiconv-prefix --without-libintl-prefix --with-system-zlib --enable-linker-build-id --with-default-libstdcxx-abi=gcc4-compatible --enable-libstdcxx-filesystem-ts
Thread model: posix
gcc version 8.3.0 (GCC)
customize Gnu95FCompiler using build_ext
building 'multxyC' extension
compiling C sources
creating C:\cwin\tmp\tmprix0z7i7\Release\cwin
creating C:\cwin\tmp\tmprix0z7i7\Release\cwin\tmp
creating C:\cwin\tmp\tmprix0z7i7\Release\cwin\tmp\tmprix0z7i7
creating C:\cwin\tmp\tmprix0z7i7\Release\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MT -IC:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7 -Ic:\program files\python37\lib\site-packages\numpy\core\include -Ic:\program files\python37\include -Ic:\program files\python37\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\include -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt /TcC:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\multxyCmodule.c /FoC:\cwin\tmp\tmprix0z7i7\Release\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\multxyCmodule.obj
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MT -IC:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7 -Ic:\program files\python37\lib\site-packages\numpy\core\include -Ic:\program files\python37\include -Ic:\program files\python37\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\include -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt /TcC:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\fortranobject.c /FoC:\cwin\tmp\tmprix0z7i7\Release\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\fortranobject.obj
compiling Fortran sources
Fortran f77 compiler: C:\cwin\bin\gfortran.exe -Wall -g -ffixed-form -fno-second-underscore -O3 -funroll-loops
Fortran f90 compiler: C:\cwin\bin\gfortran.exe -Wall -g -fno-second-underscore -O3 -funroll-loops
Fortran fix compiler: C:\cwin\bin\gfortran.exe -Wall -g -ffixed-form -fno-second-underscore -Wall -g -fno-second-underscore -O3 -funroll-loops
compile options: '-IC:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7 -Ic:\program files\python37\lib\site-packages\numpy\core\include -Ic:\program files\python37\include -Ic:\program files\python37\include -c'
gfortran.exe:f90: multxy.f90
C:\cwin\bin\gfortran.exe -Wall -g -Wall -g -shared ..\..\..\..\cwin\tmp\tmprix0z7i7\Release\multxy.o -L/usr/lib/gcc/x86_64-pc-cygwin/8.3.0 -Lc:\program files\python37\libs -Lc:\program files\python37\PCbuild\amd64 -o C:\cwin\tmp\tmprix0z7i7\Release\.libs\libmultxy.J7YCD6VUIR3DWPQ3PSLGVWZTCQ2KMJO6.gfortran-win_amd64.dll -Wl,--allow-multiple-definition -Wl,--output-def,C:\cwin\tmp\tmprix0z7i7\Release\libmultxy.J7YCD6VUIR3DWPQ3PSLGVWZTCQ2KMJO6.gfortran-win_amd64.def -Wl,--export-all-symbols -Wl,--enable-auto-import -static -mlong-double-64
/usr/lib/gcc/x86_64-pc-cygwin/8.3.0/../../../../x86_64-pc-cygwin/bin/ld: /usr/lib/gcc/x86_64-pc-cygwin/8.3.0/libgfortran.a(read.o): in function `_gfortrani_convert_real':
/usr/src/debug/gcc-8.3.0-1/libgfortran/io/read.c:173:(.text$_gfortrani_convert_real+0x85): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `strtoflt128'
/usr/lib/gcc/x86_64-pc-cygwin/8.3.0/../../../../x86_64-pc-cygwin/bin/ld: /usr/lib/gcc/x86_64-pc-cygwin/8.3.0/libgfortran.a(read.o): in function `_gfortrani_convert_infnan':
/usr/src/debug/gcc-8.3.0-1/libgfortran/io/read.c:249:(.text$_gfortrani_convert_infnan+0x5c): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `strtoflt128'
collect2: error: ld returned 1 exit status
error: Command "C:\cwin\bin\gfortran.exe -Wall -g -Wall -g -shared ..\..\..\..\cwin\tmp\tmprix0z7i7\Release\multxy.o -L/usr/lib/gcc/x86_64-pc-cygwin/8.3.0 -Lc:\program files\python37\libs -Lc:\program files\python37\PCbuild\amd64 -o C:\cwin\tmp\tmprix0z7i7\Release\.libs\libmultxy.J7YCD6VUIR3DWPQ3PSLGVWZTCQ2KMJO6.gfortran-win_amd64.dll -Wl,--allow-multiple-definition -Wl,--output-def,C:\cwin\tmp\tmprix0z7i7\Release\libmultxy.J7YCD6VUIR3DWPQ3PSLGVWZTCQ2KMJO6.gfortran-win_amd64.def -Wl,--export-all-symbols -Wl,--enable-auto-import -static -mlong-double-64" failed with exit status 1
推荐答案
我没有完全弄清 f2py 的问题，所以请不要为这个非答案而烦恼.我想要做的只是在一些fortran周围放一个简单的包装.我虽然解决了这个问题，所以这篇文章是我的答案.我将其发布，希望对其他有类似问题的人有所帮助.
I did not exactly figure out the problem with f2py, so please don't ding me for this nonanswer. All I wanted was to put a simple wrapper around some fortran. I solved that problem though, so this post is my answer. I am posting it in hopes it may help someone else with a similar problem.
发布后，我认为我需要降低到较低的水平以弄清楚发生了什么.由于 f2py 是基于 C-api 构建的，因此我决定直接使用它. C-fortran互操作性已得到很好的定义.我有一些简单的代码可以工作，但是当我开始查看传递数组并阅读有关引用计数的信息时，我被吓到了.因此，我认为 C-api 的水平太低，无法满足我的需求. Cython 似乎是针对编写可编译的类似Python的代码.我最终决定使用 ctypes ，因为我真正需要的只是转换内部python类型，然后调用(它认为是)c代码.周围有一些示例，但似乎没有一个完全合适，因此我将发布一些测试代码，以希望对其他人有帮助.该示例显示了通过函数和子例程来回传递int，double和array的情况.如果将.so替换为.dll，则在Windows中也可以使用相同的代码.此信息很有帮助-
After the post, I decided I needed to drop down to a lower level to figure out what was going on. Since f2py is built on the C-api, I decided to work with it directly. C - fortran interoperability is well defined. I got some simple code to work, but I was scared off when I started looking at passing arrays and read about reference counting. So, I decided the C-api it was too low level for my needs. Cython appears oriented toward writing Python-like code that can be compiled. I finally settled on ctypes, since all I really need is to convert the internal python types and then call (what it thinks is) c code. There are examples around but none seemed to fit exactly, so I will post some test code in hopes it may help others. The example shows passing ints, doubles and arrays back and forth by both functions and subroutines. The same code works in windows if .so is replaced with .dll.This information was helpful - How to dereference a memory location from python ctypes?
fortran是:
Integer Function intfunc(n)
integer, intent(in) :: n
intfunc = n*n
end Function intfunc
! ---------------------------------------------------------------------------------------
subroutine Asub(acf,af)
include 'defs.fi' ! integer, parameter :: float = selected_real_kind(12)
Real(float), intent(in) :: acf
Real(float), intent(out) :: af
if(acf < 0.49)then
af = 0.37464 + acf*(1.54226 - 0.26992*acf)
else
af = 0.379642 + acf*(1.48503 + acf*(-0.164423 + 0.016666*acf))
endif
end subroutine Asub
! ---------------------------------------------------------------------------------------
Function Afactor(acf) result(af)
include 'defs.fi'
Real(float), intent(in) :: acf
Real(float) :: af
if(acf < 0.49)then
af = 0.37464 + acf*(1.54226 - 0.26992*acf)
else
af = 0.379642 + acf*(1.48503 + acf*(-0.164423 + 0.016666*acf))
endif
End Function Afactor
! ---------------------------------------------------------------------------------------
Function MultXY(x,y) result(z)
include 'defs.fi'
Real(float), intent(in) :: x,y
Real(float)::z
z = x*y
End Function MultXY
! ---------------------------------------------------------------------------------------
subroutine arrays(n,m,a)
include 'defs.fi'
integer, intent(in) :: n,m
real(float), intent(out) :: a(n,m)
integer :: i,j
do j=1,m
do i=1,n
a(i,j) = real(100*i+j,float)
enddo
enddo
end subroutine arrays
通过以下方式将代码编译到共享库中:
The code is compiled into a shared library by:
gfortran -shared -o code.so code.f90 -fPIC -Xlinker -Map=code.map
它可以与以下python代码一起运行.可以将这段代码中的许多步骤组合在一起，但是我认为将它们分解更能说明问题.我不完全了解函数原型(argtypes和amps restypes).它们似乎允许一些快捷方式，例如Python执行适当的强制转换.有人可以详细说明吗?
It can be run with the following python code. A number of the steps in this code could be combined, but I thought it was more illustrative to break them down. I don't completely understand the function prototypes (argtypes & restypes). They seem to allow some shortcuts, such that Python performs the appropriate cast. Can someone elaborate?
import ctypes as ct
import numpy as np
from numpy.ctypeslib import ndpointer
flib = ct.CDLL('code.so')
flib.asub_.argtypes = [ct.POINTER(ct.c_double),ct.POINTER(ct.c_double)]
flib.afactor_.argtypes = [ct.POINTER(ct.c_double)]
flib.afactor_.restype = ct.c_double
flib.multxy_.argtypes = [ct.POINTER(ct.c_double),ct.POINTER(ct.c_double)]
flib.multxy_.restype = ct.c_double
flib.arrays_.argtypes = [ct.POINTER(ct.c_int),ct.POINTER(ct.c_int),ndpointer(ct.c_double)]
def main():
nval = 4
nptr = ct.pointer(ct.c_int(nval))
# integer function intfunc(n)
nx = flib.intfunc_(nptr)
print('intfunc(',nval,') =',nx)
x = 0.35
xc = ct.c_double(x)
yc = ct.c_double(0.50)
zc = ct.c_double(0.0)
# subroutine Asub(acf,af)
flib.asub_(ct.byref(xc),ct.byref(zc))
z = zc.value
x = xc.value
y = yc.value
print(' x,z =',x,z)
# Function Afactor(acf) result(af)
z = flib.afactor_(ct.byref(xc))
print(' x,z =',x,z)
# Function MultXY(x,y) result(z)
z = flib.multxy_(ct.byref(xc),ct.byref(yc))
print('multxy(',x,y,') =',z)
n = 3
m = 4
nc = ct.c_int(n)
mc = ct.c_int(m)
a = np.empty((m,n),dtype=np.double) # m,n not n,m
# subroutine arrays(n,m,a)
flib.arrays_(ct.byref(nc),ct.byref(mc),a)
for i in range(n): # note reversal of indices
for j in range(m):
print('i,j,a(j,i) =',i+1,j+1,a[j,i])
main()
输出为:
intfunc( 4 ) = 16
x,z = 0.35 0.8813658067584037
x,z = 0.35 0.8813658067584037
multxy( 0.35 0.5 ) = 0.175
i,j,a(j,i) = 1 1 101.0
i,j,a(j,i) = 1 2 102.0
i,j,a(j,i) = 1 3 103.0
i,j,a(j,i) = 1 4 104.0
i,j,a(j,i) = 2 1 201.0
i,j,a(j,i) = 2 2 202.0
i,j,a(j,i) = 2 3 203.0
i,j,a(j,i) = 2 4 204.0
i,j,a(j,i) = 3 1 301.0
i,j,a(j,i) = 3 2 302.0
i,j,a(j,i) = 3 3 303.0
i,j,a(j,i) = 3 4 304.0
这篇关于f2py链接Quadmath库?使用ctypes作为fortran包装器?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
