阅读FORTRAN直接访问数据并写入格式化数据 - 与Python比FORTRAN快？ [英] Reading fortran direct access data and writing formatted data - faster with python than with fortran?

问题描述

晚上好，

我用Fortran产生无格式（直接访问）数据的大型文件的模拟。
从一些这些文件我想生产ASCII人类可读的文件。

I have a simulation written in Fortran that produces large files of unformatted (direct access) data. From some of these files I want to produce ascii human-readable files.

有关某种原因，这（在python）：

For some reason this (in python):

f = open(filename,'rb')
for i in xrange(0,N):
    pos = i * 64
    f.seek(pos)
    name = struct.unpack('ffff',f.read(16))
    print name[0],name[1],name[2],name[3]

仅需〜4个秒（管道输出到外壳上的文件），而这（Fortran中）

takes only ~4 seconds (piping the output into a file on the shell) while this (in Fortran)

 open (1,file=inputfile,access='direct',recl=64, action='read',status="OLD")
 open (2, file=outputfile, access="sequential", action="write",status="REPLACE")
 do i=1,(N)
     read(1, rec = i ) a,b,c,d
     write(2,*) a,b,c,d
 enddo

大约需要20秒。
我究竟做错了什么？有没有用Fortran这样做的更快的方法？

takes ~ 20 seconds. What am I doing wrong? Is there a faster way of doing this in Fortran?

最好的问候！
RER

Best regards! rer

推荐答案

免责声明：我不知道，如果这个问题解决了，但我知道，我可以得到一个时间差高达20我也没有只测试数据的输出，并没有阅读的因素。

DISCLAIMER: I don't know, if this solves the problem, but I know, that I can get time differences by a factor of up to 20. I also did only test the output of data and didn't read it.

我正在调查的Fortran与蟒蛇的相互作用，因此想知道，Fortran语言的二进制文件是如何构建。虽然这样做，我注意到，这两个 ifort 和 gfortran 要切换缓冲IO或关闭的选项。

I was investigating the interaction of Fortran with python and as such wanted to know, how Fortran's binary files are build. While doing this, I noticed, that both ifort and gfortran have an option to switch buffered IO on or off.

的 ifort 的：可以指定关键字 BUFFERED = ['YES'|'NO'] 打开文件

ifort: You can specify the keyword BUFFERED=['YES'|'NO'] while opening a file.

的 gfortran 的：你可以设置环境变量 GFORTRAN_UNBUFFERED_ALL 到 Y | Y | 1 或 N | N | 0 对于无缓冲缓冲的IO，分别为

gfortran: You can set the environmental variable GFORTRAN_UNBUFFERED_ALL to y|Y|1 or n|N|0 for unbuffered and buffered IO, respectively.

请注意，gfortran确实IO缓冲默认情况下，当ifort没有。

Please note, that gfortran does buffer IO by default, while ifort does not.

我的示例code底部结果时间如下：

My sample code at the bottom results in the following times:

        |buffered|unbuffered
--------+--------+----------
ifort   |   1.9s |  18.2s
gfortran|   2.4s |  37.5s

---

本示例code写直接访问二进制文件与每个12字节10M的数据集。

---

This sample code writes a direct access binary file with 10M datasets of 12 bytes each.

PROGRAM btest
IMPLICIT NONE

INTEGER :: i

! IFORT
OPEN(11,FILE="test_d.bin",ACCESS="DIRECT",FORM="UNFORMATTED",RECL=3, &
& STATUS="REPLACE",BUFFERED="NO") ! ifort defines RECL as words
! GFORTRAN
!OPEN(11,FILE="test_d.bin",ACCESS="DIRECT",FORM="UNFORMATTED",RECL=12, &
!& STATUS="REPLACE") ! gfortran defines RECL as bytes

DO i = 1, 10000000
    WRITE(11,REC=i) i,i*1._8
END DO

CLOSE(11)

END PROGRAM

这篇关于阅读FORTRAN直接访问数据并写入格式化数据 - 与Python比FORTRAN快？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！