写一个float数组在Java中的文件 [英] Write a float array to file in Java

问题描述

我读NetCDF文件，我想每个阵列作为一个float数组中读，然后写float数组到一个新文件。我可以做，如果我float数组中读取，然后遍历数组中的每个元素（使用DataOutputStream类）它的工作，但是这是非常，非常慢，我的NetCDF文件超过1GB。

I'm reading in a NetCDF file and I want to read in each array as a float array and then write the float array to a new file. I can make it work if I read in the float array and then iterate over each element in the array (using a DataOutputStream), but this is very, very slow, my NetCDF files are over 1GB.

我试着使用ObjectOutputStream，但这个写信息的额外的字节。

I tried using an ObjectOutputStream, but this writes extra bytes of information.

因此​​，要回顾一下。
1.打开netCDF文件
从netCDF文件2.读取浮动数组x
3.写浮动数组x的原始数据文件中的一个步骤
4.重复步骤2 X + 1

So, to recap. 1. Open NetCDF file 2. Read float array x from NetCDF file 3. Write float array x to raw data file in a single step 4. Repeat step 2 with x+1

推荐答案

好吧，你有1 GB的读取和1 GB写。根据您的硬盘上，你可能会得到大约100 MB / s的读取和60 MB / s的写入速度。这意味着将需要大约27秒内读写。

Ok, You have 1 GB to read and 1 GB to write. Depending on your hard drive, you might get about 100 MB/s read and 60 MB/s write speed. This means it will take about 27 seconds to read and write.

什么是您的驱动器，以及如何比这慢得多，你看到的速度？

What is the speed of your drive and how much slower than this are you seeing?

如果你想测试你的硬盘的速度，不经任何处理，时间需要多长时间来复制你最近没有访问的文件（即它不是在磁盘高速缓存），这会给你的想法最小延迟，你可以期望读然后写大多数数据从文件（即不处理或Java参与）

If you want to test the speed of your disk without any processing, time how long it takes to copy a file which you haven't accessed recently (i.e. it is not in disk cache) This will give you an idea of the minimum delay you can expect to read then write most of the data from the file (i.e. with no processing or Java involved)

对于任何想谁的利益，要懂得做一个循环较少的数据副本，即它不只是调用哪个循环为你的方法。

For the benefit of anyone who would like to know how to do a loop less copy of data i.e. it doesn't just call a method which loops for you.

FloatBuffer src = // readable memory mapped file.
FloatByffer dest = // writeable memory mapped file.
src.position(start);
src.limit(end);
dest.put(src);

如果您有混合类型的数据可以使用的ByteBuffer这名义上拷贝一个字节在同一时间但在现实中可以使用长或宽型8个或多个字节一次复制。即无论CPU可以做的。

If you have mixed types of data you can use ByteBuffer which notionally copies a byte at a time but in reality could use long or wider type to copy 8 or more bytes at a time. i.e. whatever the CPU can do.

有关小块，这将使用循环但对于大的块就可以在操作系统中使用页面的映射技巧。在任何情况下，它是怎么做的不是在Java定义的，但它可能是复制数据的最快方法。

For small blocks this will use a loop but for large blocks it can use page mapping tricks in the OS. In any case, how it does it is not defined in Java, but its likely to be the fastest way to copy data.

大部分这些花招只能有所作为，如果你是在内存中已经拷贝文件到缓存的文件。当你从磁盘读取文件或文件过大缓存的物理磁盘的IO带宽是真正重要的唯一的事情。

Most of these tricks only make a difference if you are copying file already in memory to a cached file. As soon as you read a file from disk or the file is too large to cache the IO bandwidth of the your physical disk is the only thing which really matters.

这是因为CPU可以在6 GB / s的主存储器复制数据，但只有60-100 MB / s到硬盘驱动器。如果CPU /内存副本是2倍，10倍或50倍的速度低于它可能是，它仍然会等待磁盘。注意：不带缓冲，这是完全有可能的，更糟糕​​的，但只要你有任何简单的缓冲CPU会比磁盘快

This is because a CPU can copy data at 6 GB/s to main memory but only 60-100 MB/s to a hard drive. If the copy in the CPU/memory is 2x, 10x or 50x slower than it could be, it will still be waiting for the disk. Note: with no buffering this is entirely possible and worse, but provided you have any simple buffering the CPU will be faster than the disk.

这篇关于写一个float数组在Java中的文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！