Java中的文件流 [英] File Streaming in Java

问题描述

我目前正在使用 JOGL （Java OpenGL绑定）开发3D图形应用程序。简而言之，我有一个巨大的横向二进制文件。由于它的大小，我必须在运行时流式传输地形块。因此，我们明确地看到随机访问问题。我已经完成了第一个（和脏:)）实现（也许它是多线程的），我正在使用一个愚蠢的方法......这是它的初始化：

I'm currently developing 3D graphics application using JOGL (Java OpenGL binding). In brief, I have a huge landscape binary file. Due to its size, I have to stream terrain chunks in the run-time. Therefore, we explicitly see the random access concern. I have already finished the first (and dirty :)) implementation (perhaps it is multi-threaded), where I'm using a foolish approach... Here is the initialization of it:

dataInputStream = new DataInputStream(new BufferedInputStream(fileInputStream,4 * 1024);
dataInputStream.mark(dataInputStream.available());

当我需要读取（流）特殊块（我已经知道它在文件中的偏移）我正在执行以下（对我来说很遗憾:)）：

And when I need to read (stream) special chunk (I already know its "offset" in the file) I'm performing the following (shame on me :)):

dataInputStream.reset();
dataInputStream.skipBytes(offset);
dataInputStream.read(whatever I need...);

因为我的经验很少，这是我能想到的第一件事:)
所以，到现在为止，我已经阅读了3篇有用而且非常有趣的文章（我建议你阅读它们，或许如果你对这个主题感兴趣的话）

Since I had little experience that was the first thing I could think about :) So, until now I have read 3 useful and quite interesting articles (I'm suggesting you to read them, perhaps if you are interested in this topic)

1. 字节缓冲区和非堆内存 - 格雷戈里先生似乎在Java NIO中识字。

1. Byte Buffers and Non-Heap Memory - Mr. Gregory seems to be literate in Java NIO.

Java提示：如何快速阅读文件 [http://nadeausoftware.com/articles/2008/02/java_tip_how_read_files_quickly] - 这是一个有趣的基准。

Java tip: How to read files quickly [http://nadeausoftware.com/articles/2008/02/java_tip_how_read_files_quickly] - That's an interesting benchmark.

文章：调优Java I / O Performance [http://java.sun.com/developer/technicalArticles/Programming/PerfTuning/] - 简单的Sun建议，但请向下滚动并查看那里的随机访问部分;他们展示了一个简单的RandomAccessFile（RAF）实现，具有自我缓冲的改进。

Articles: Tuning Java I/O Performance [http://java.sun.com/developer/technicalArticles/Programming/PerfTuning/] - Simple Sun recommendations, but please scroll down and have a look at "Random Access" section there; they show a simple implementation of RandomAccessFile (RAF) with self-buffering improvement.

Mr。 Gregory在他的文章末尾提供了几个* .java文件。其中之一是FileChannel + ByteBuffer + Mapping（FBM）和RAF之间的基准测试。他说，与RAF相比，他在使用FBM时注意到了4倍的加速。我在以下条件下运行此基准：

Mr. Gregory provides several *.java files in the end of his article. One of them is a benchmarking between FileChannel + ByteBuffer + Mapping (FBM) and RAF. He says that he noticed 4x speedup when using FBM compared to RAF. I have ran this benchmark in the following conditions:

1. 偏移量（例如访问地点）是随机生成的（在文件范围内，例如0 - file.length（））;


2. 文件大小为220MB;


3. 1 000 000次访问（75％读取和25％写入）

结果令人惊叹：

〜/ 28秒RAF！ FBM的
~0.2秒！

~ 28 sec for RAF! ~ 0.2 sec for FBM!

但是，他在此基准测试中实施的RAF没有自我缓冲（第3篇文章讲述了一个），所以我猜这是RandomAccessFile.seek方法调用，谁会如此努力地降低性能。

However, his implementation of RAF in this benchmark doesn't have self-buffering (the 3rd article tells about one), so I guess it is the "RandomAccessFile.seek" method calling, who drops performance so hard.

好的，现在在所有这些事情之后我已经知道有1个问题和1个困境：）

Ok, now after all those things I've learnt there is 1 question and 1 dilemma :)

问题：当我们使用FileChannel.map映射文件时，Java会不会将整个文件内容复制到MappedByteBuffer中？或者它只是模仿它？如果它复制，那么使用FBM方法不适合我的情况，是吗？

Question: When we are mapping a file using "FileChannel.map" does Java copy the whole file contents into the MappedByteBuffer? Or does it just emulate it? If it copies, then using FBM approach is not suitable for my situation, is it?

困境：取决于你对这个问题的答案。 ..

Dilemma: Depends on your answers on the question...

1. 如果映射复制文件，那么我似乎只有两种可能的解决方案： RAF +自我缓冲（第3篇文章中的那篇）或利用FileChannel中的位置（不使用映射）...哪一个会更好？

1. If mapping copies a file, then it seems like I have only 2 possible solutions to go: RAF + self-buffering (the one from the 3rd article) or make use of position in FileChannel (not with mapping)... Which one would be better?

如果映射不复制文件，那么我有3个选项：前两个选项和 FBM本身。

If mapping doesn't copy a file, then I have 3 options: two previous ones and FBM itself.

编辑：这是另外一个问题。你们有些人说，映射不会将文件复制到MappedByteBuffer中。那么，为什么我不能映射1GB文件呢？我收到无法映射的消息......

Edit: Here is one more question. Some of you here say that mapping doesn't copy file into MappedByteBuffer. Ok then, why can't I map 1GB file then, I'm getting "failed to map" message...

P. S。我希望收到有关建议的答案，因为我无法在互联网上找到关于此主题的一致信息。

P. S. I would like to receive a fulfilled answer with advices, since I'm not able to find the consistent information over this topic in the internet.

谢谢：）

推荐答案

不，数据没有缓冲。 MappedByteBuffer使用指针引用数据。换句话说，数据不会被复制，它只是映射到物理内存中。如果您还没有，请参阅 API文档已经。

No, the data is not buffered. A MappedByteBuffer references the data using a pointer. In other words, the data is not copied, it is simply mapped into physical memory. See the API docs if you haven't already.

内存映射文件是
虚拟内存的一段，已分配
a直接字节 - for-byte correlation
与文件的某些部分或
文件类资源。这个资源是
，通常是一个物理上
存在于磁盘上的文件，但也可以是
设备，共享内存对象或操作系统可以$
资源。 b $ b通过文件描述符引用。
一旦出现，
文件和内存空间之间的这种相关性允许
应用程序将映射的
部分视为主内存。

A memory-mapped file is a segment of virtual memory which has been assigned a direct byte-for-byte correlation with some portion of a file or file-like resource. This resource is typically a file that is physically present on-disk, but can also be a device, shared memory object, or other resource that the operating system can reference through a file descriptor. Once present, this correlation between the file and the memory space permits applications to treat the mapped portion as if it were primary memory.

来源：维基百科

如果您要经常阅读数据，最好至少缓存一些数据。

If you are going to be reading data quite frequently, it is a good idea to at least cache some of it.

这篇关于Java中的文件流的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！