阅读使用不同的编译器编写fortran直接访问未格式化的文件 [英] Reading writing fortran direct access unformatted files with different compilers

查看：338
本文介绍了阅读使用不同的编译器编写fortran直接访问未格式化的文件的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

  open（53，file）在程序中有一段代码，它写入一个直接访问二进制文件，如下所示： = filename，form ='unformatted'，status ='unknown'，
& access ='direct'，action ='write'，recl = 320 * 385 * 8）
 write（53，rec = 1）ulat 
 write（53，rec = 2）ulng 
 close（53）

这个程序是用ifort编译的。但是，如果我从用gfortran编译的其他程序读取数据文件，我无法正确重建数据。如果读取数据的程序也编译成ifort，那么我可以正确重建数据。这里是读取数据文件的代码： 

  OPEN（53，FILE = fname，form =unformatted，status =unknown ，access =direct，action =read，recl = 320 * 385 * 8）
 READ（53，REC = 2）DAT

我不明白为什么会发生这种情况？我可以用两种编译器正确读取第一条记录，如果我混合使用编译器，这是我无法正确重构的第二条记录。

解决方案

Ifort和gfortran默认情况下不使用相同的块大小来记录长度。在ifort中， open 语句中 recl 的值是4字节的块，所以您的记录长度为' t 985,600字节，长度为3,942,400字节。这意味着这些记录每隔390万字节写入一次。



gfortran使用1字节的 recl 块大小你的记录长度是985,600。当您读取第一条记录时，一切正常，但是当您读取第二条记录时，您会看到文件中的985,600字节，但数据位于文件中的3,942,400字节。这也意味着你正在浪费文件中的大量数据，因为你只使用其大小的1/4。



有几种方法可以解决这个问题：


  • ifif在4字节块中指定recl，例如 320 * 385 * 2 代替 * 8

  • 编译标志 -assume byterecl 将 recl 值解释为字节。
  • 在gfortran中补偿大小并使用 recl = 320 * 385 * 32 ，以便您的阅读正确定位。



然而，更好的方法是在 recl 单位大小中设计不可知论。您可以使用 inquire 来计算数组的recl。例如：

$ pre $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ :: reclen
allocate（recltest（320,385））
inquire（iolength = reclen）recltest
deallocate（recltest）
...
open（53，file =文件名，form ='unformatted'，status ='unknown'，
& access ='direct'，action ='write'，recl = reclen）
...
OPEN（53 ，FILE = fname，form =unformatted，status =unknown，&
access =direct，action =read，recl = reclen）

这会将 reclen 设置为存储 320x385 数组基于记录长度的编译器基本单位。如果您在写入和读取代码时都使用这两种编译器，而无需在 ifort 中使用编译时标志，或者使用编译器之间的硬编码recl差异进行补偿， p>



一个说明性示例



测试用例1 



 程序测试
使用iso_fortran_env 
隐式无
 
整数（kind = int64），维（5 ）:: array 
 integer :: io_output，reclen，i 
 reclen = 5 * 8！ 5个8字节整数的元素。 
 
 open（newunit = io_output，file ='output'，form ='unformatted'，status ='new'，& 
 access ='direct'，action ='write'， recl = reclen）
 $ b $ array = [（i，i = 1,5）] 
 write（io_output，rec = 1）array 
 array = [（i，i = 101,105）] 
 write（io_output，rec = 2）array 
 array = [（i，i = 1001,1005）] 
 write（io_output，rec = 3）array 
 
 close（io_output）
结束程序测试

该程序写入5个8字节整数3次，记录1,2和3中的文件。该数组是5 * 8个字节，并且我已将该数字硬编码为recl值。

gfortran 5.2的测试用例1



我用命令行编译了这个测试用例：

  gfortran -o write-gfortran write.f90

这产生输出文件（用 od -A d -t d8 ）解释： 

  0000000 1 2 
 0000016 3 4 
 0000032 5 101 
 0000048 102 103 
 0000064 104 105 
 0000080 1001 1002 
 0000096 1003 1004 
 0000112 1005 
 0000120

-bye元素被连续压缩到文件中，并且记录编号2（ 101 ... 105 ）开始于我们预期它在偏移40处，这是recl值文件 5 * 8 。



Testcase 1 with ifort 16



编译方式如下：

  ifort -o write-ifort write.f90

这对于完全相同的代码来说会产生输出文件（用 od -A d -t d8 ）解释： 

  0000000 1 2 
 0000016 3 4 
 0000032 5 0 
 0000048 0 0 
 * 
 0000160 101 102 
 0000176 103 104 
 0000192 105 0 
 0000208 0 0 
 * 
 0000320 1001 1002 
 0000336 1003 1004 
 0000352 1005 0 
 0000368 0 0 
 * 
 000 0480

数据全部存在，但文件充满了0个值元素。以 * 开头的行表示偏移量之间的每一行都是0.记录号2从偏移量160开始，而不是40。请注意，160是40 * 4，其中40是我们的指定recl 5 * 8 。默认情况下，ifort使用4字节块，因此recl为40意味着160字节的物理记录大小。

如果使用gfortran编译的代码要读取它，则记录2,3和4将包含全部0个元素，并且记录5的读取将正确读取写入记录的数组2 by ifort。 gfortran读取记录2位于文件中的替代方法是使用 recl = 160 （4 * 5 * 4），以便物理记录大小与由ifort编写。



另一个结果是浪费空间。过度指定recl意味着您使用4倍的必要磁盘空间来存储您的记录。

ifcase 16和 -assume byterecl



<这是编译为：

  ifort -assume byterecl -o write-ifort write.f90

并产生输出文件： 

  0000000 1 2 
 0000016 3 4 
 0000032 5 101 
 0000048 102 103 
 0000064 104 105 
 0000080 1001 1002 
 0000096 1003 1004 
 0000112 1005 
 0000120

这会按预期生成文件。命令行参数 -assume byterecl 告诉ifort将任何 recl 值解释为字节而不是双字（4字节块）。这将产生与gfortran编译的代码匹配的写入和读取。

测试用例2 



 程序测试
使用iso_fortran_env 
隐式无
 
 integer（kind = int64），dimension（5）:: array 
 integer :: io_output，reclen ，i 
 inquire（iolength = reclen）array 
 print *，'使用recl ='，reclen 
 
 open（newunit = io_output，file ='output'，form =' unformatted'，status ='new'，& 
 access ='direct'，action ='write'，recl = reclen）
 
 array = [（i，i = 1,5 ）] 
 write（io_output，rec = 1）array 
 array = [（i，i = 101,105）] 
 write（io_output，rec = 2）array 
 array = [ （i，i = 1001,1005）] 
写入（io_output，rec = 3）数组
 
关闭（io_output）
结束程序测试
  $ b $ p 
 $ b 
这个测试用例唯一的区别是我正在查询正确的recl来表示我的40字节数组（5个8字节整数）。  
 
 
输出
 
 
 
 gfortran 5.2： 
 
 
 使用recl = 40 
  
 ifort 16，no options：
 使用recl = 10 
  
 ifort 16， -assume byterecl ：
 使用recl = 40 
  
我们看到，对于gfortran和ifort使用的1字节块， byterecl 假设recl < 40 ，它等于我们的40字节数组。我们还可以看到，默认情况下，ifort使用10的recl，即10个4字节块或10个双字，这两个字都表示40个字节。所有这三个测试用例都产生相同的文件输出，并且来自任一编译器的读取/写入都能正常运行。 >要在ifort和gfortran之间创建基于记录的，未格式化的直接数据，最简单的选择是将 -assume byterecl 添加到ifort使用的标志中。你真的应该已经这样做了，因为你正在指定以字节为单位的记录长度，所以这将是一个直接的变化，可能对你没有任何影响。 
 
 
另一种方法是不用担心该选项，并使用 inquire 内部函数来查询 iolength 为你的数组。  
I have a section in a program that writes a direct-access binary file as follows:
open (53, file=filename, form='unformatted', status='unknown',
& access='direct',action='write',recl=320*385*8)
write (53,rec=1) ulat
write (53,rec=2) ulng
close(53)

This program is compiled with ifort. However, I cannot reconstruct the data correctly if I read the data file from a different program compiled with gfortran. If the program reading the data is also compiled in ifort, then I can correctly reconstruct the data. Here's the code reading the data file:
OPEN(53, FILE=fname, form="unformatted", status="unknown", access="direct", action="read", recl=320*385*8)
READ(53,REC=2) DAT

I do not understand why this is happening? I can read the first record correctly with both compilers, it's the second record that I cannot reconstruct properly if I mix the compilers. 
 解决方案 
Ifort and gfortran do not use the same block size for record length by default.  In ifort, the value of recl in your open statement is in 4-byte blocks, so your record length isn't 985,600 bytes, it is 3,942,400 bytes long.  That means the records are written at intervals of 3.9 million bytes.



gfortran uses a recl block size of 1 byte and your record length is 985,600 byes.  When you read the first record, everything works, but when you read the second record you look at 985,600 bytes into the file but the data is at 3,942,400 bytes into the file.  This also means you are wasting a ton of data in the file, as you are using only 1/4 of its size.



There are a couple ways to fix this:



    

  • In ifort specify recl in 4-byte blocks, e.g. 320*385*2 instead of *8
    • 

  • In ifort, use the compile flag -assume byterecl to have recl values interpreted as bytes.
    • 

  • In gfortran compensate for the size and use recl=320*385*32 so that your reads are correctly positioned.
    • 




A better way, however, is to engineer agnosticism in the recl unit size. You can use inquire to figure out the recl of an array.  For example:
real(kind=wp), allocatable, dimension(:,:) :: recltest
integer :: reclen
allocate(recltest(320,385))
inquire(iolength=reclen) recltest
deallocate(recltest)
...
open (53, file=filename, form='unformatted', status='unknown',
& access='direct',action='write',recl=reclen)
...
OPEN(53, FILE=fname, form="unformatted", status="unknown", &
access="direct", action="read", recl=reclen)

This will set reclen to the value needed to store a 320x385 array based on the that compilers base unit for record length.  If you use this when both writing and reading your code will work with both compilers without having to use compile-time flags in ifort or compensate with hardcoded recl differences between compilers.






An illustrative example



Testcase 1



program test
  use iso_fortran_env
  implicit none

  integer(kind=int64), dimension(5) :: array
  integer :: io_output, reclen, i
  reclen = 5*8 ! 5 elements of 8 byte integers.

  open(newunit=io_output, file='output', form='unformatted', status='new', &
       access='direct', action='write', recl=reclen)

  array = [(i,i=1,5)]  
  write (io_output, rec=1) array
  array = [(i,i=101,105)]
  write (io_output, rec=2) array
  array = [(i,i=1001,1005)]
  write (io_output, rec=3) array

  close(io_output)
end program test

This program writes an array of 5 8-byte integers 3 times to the file in records 1,2 and 3.  The array is 5*8 bytes and I have hardcoded that number as the recl value. 



Testcase 1 with gfortran 5.2



I compiled this testcase with the command line:
gfortran -o write-gfortran write.f90

This produces the output file (interpreted with od -A d -t d8):
0000000                    1                    2
0000016                    3                    4
0000032                    5                  101
0000048                  102                  103
0000064                  104                  105
0000080                 1001                 1002
0000096                 1003                 1004
0000112                 1005
0000120

The arrays of 5 8-bye elements are packed contiguously into the file and record number 2 (101 ... 105) starts where we would expect it to at offset 40, which is the recl value in the file 5*8.



Testcase 1 with ifort 16



This is compiled similarly:
ifort -o write-ifort write.f90

And this, for the exact same code, produces the output file (interpreted with od -A d -t d8): 
0000000                    1                    2
0000016                    3                    4
0000032                    5                    0
0000048                    0                    0
*
0000160                  101                  102
0000176                  103                  104
0000192                  105                    0
0000208                    0                    0
*
0000320                 1001                 1002
0000336                 1003                 1004
0000352                 1005                    0
0000368                    0                    0
*
0000480

The data is all there but the file is full of 0 valued elements.  The lines starting with * indicate every line between the offsets is 0.  Record number 2 starts at offset 160 instead of 40.  Notice that 160 is 40*4, where 40 is our specified recl of 5*8.  By default ifort uses 4-byte blocks, so a recl of 40 means a physical record size of 160 bytes.  



If code compiled with gfortran were to read this, records 2,3 and 4 would contain all 0 elements and a read of record 5 would correctly read the array written as record 2 by ifort.  An alternative to have gfortran read record 2 where it lies in the file would be to use recl=160 (4*5*4) so that the physical record size matches what was written by ifort.



Another consequence of this is wasted space.  Over-specifying the recl means you are using 4 times the necessary disk space to store your records.  



Testcase 1 with ifort 16 and -assume byterecl



This was compiled as:
ifort -assume byterecl -o write-ifort write.f90

And produces the output file:
0000000                    1                    2
0000016                    3                    4
0000032                    5                  101
0000048                  102                  103
0000064                  104                  105
0000080                 1001                 1002
0000096                 1003                 1004
0000112                 1005
0000120

This produces the file as expected.  The command line argument -assume byterecl tells ifort to interpret any recl values as bytes rather than double words (4-byte blocks).  This will produce writes and reads that match code compiled with gfortran.



Testcase 2



program test
  use iso_fortran_env
  implicit none

  integer(kind=int64), dimension(5) :: array
  integer :: io_output, reclen, i
  inquire(iolength=reclen) array
  print *,'Using recl=',reclen

  open(newunit=io_output, file='output', form='unformatted', status='new', &
       access='direct', action='write', recl=reclen)

  array = [(i,i=1,5)]  
  write (io_output, rec=1) array
  array = [(i,i=101,105)]
  write (io_output, rec=2) array
  array = [(i,i=1001,1005)]
  write (io_output, rec=3) array

  close(io_output)
end program test

The only difference in this testcase is that I am inquiring the proper recl to represent my 40-byte array (5 8-byte integers).



The output



gfortran 5.2:
 Using recl=          40

ifort 16, no options:
 Using recl=          10

ifort 16, -assume byterecl:
 Using recl=          40

We see that for the 1-byte blocks used by gfortran and ifort with the byterecl assumption that recl is 40, which equals our 40 byte array.  We also see that by default, ifort uses a recl of 10, which means 10 4-byte blocks or 10 double words, both of which mean 40 bytes.  All three of these testcases produce identical file output and read/writes from either compiler will function properly.



Summary



To have record-based, unformatted, direct data be portable between ifort and gfortran the easiest option is to just add -assume byterecl to the flags used by ifort.  You really should have been doing this already since you are specifying record lengths in bytes, so this would be a straightforward change that probably has no consequences for you.  



The other alternative is to not worry about the option and use the inquire intrinsic to query the iolength for your array.  


                        
这篇关于阅读使用不同的编译器编写fortran直接访问未格式化的文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
相关文章
其他开发最新文章
热门教程
热门工具
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆