Java NIO FileChannel 与 FileOutputstream 的性能/实用性 [英] Java NIO FileChannel versus FileOutputstream performance / usefulness
问题描述
我想弄清楚当我们使用 nio FileChannel 与普通的 FileInputStream/FileOuputStream 读取和写入文件时，性能(或优势)是否有任何差异文件系统.我观察到，在我的机器上，两者的性能都处于同一水平，而且很多时候 FileChannel 方式更慢.我能否了解比较这两种方法的更多细节.这是我使用的代码，我正在测试的文件大约是 350MB.如果我不考虑随机访问或其他此类高级功能，将基于 NIO 的类用于文件 I/O 是否是一个不错的选择?
I am trying to figure out if there is any difference in performance (or advantages) when we use nio FileChannel versus normal FileInputStream/FileOuputStream to read and write files to filesystem. I observed that on my machine both perform at the same level, also many times the FileChannel way is slower. Can I please know more details comparing these two methods. Here is the code I used, the file that I am testing with is around 350MB. Is it a good option to use NIO based classes for File I/O, if I am not looking at random access or other such advanced features?
package trialjavaprograms;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
public class JavaNIOTest {
public static void main(String[] args) throws Exception {
useNormalIO();
useFileChannel();
}
private static void useNormalIO() throws Exception {
File file = new File("/home/developer/test.iso");
File oFile = new File("/home/developer/test2");
long time1 = System.currentTimeMillis();
InputStream is = new FileInputStream(file);
FileOutputStream fos = new FileOutputStream(oFile);
byte[] buf = new byte[64 * 1024];
int len = 0;
while((len = is.read(buf)) != -1) {
fos.write(buf, 0, len);
}
fos.flush();
fos.close();
is.close();
long time2 = System.currentTimeMillis();
System.out.println("Time taken: "+(time2-time1)+" ms");
}
private static void useFileChannel() throws Exception {
File file = new File("/home/developer/test.iso");
File oFile = new File("/home/developer/test2");
long time1 = System.currentTimeMillis();
FileInputStream is = new FileInputStream(file);
FileOutputStream fos = new FileOutputStream(oFile);
FileChannel f = is.getChannel();
FileChannel f2 = fos.getChannel();
ByteBuffer buf = ByteBuffer.allocateDirect(64 * 1024);
long len = 0;
while((len = f.read(buf)) != -1) {
buf.flip();
f2.write(buf);
buf.clear();
}
f2.close();
f.close();
long time2 = System.currentTimeMillis();
System.out.println("Time taken: "+(time2-time1)+" ms");
}
}
推荐答案
我在处理较大文件时的经验是 java.nio 比 java.io 快.明显更快.就像在 >250% 的范围内一样.也就是说，我正在消除明显的瓶颈，我建议您的微基准测试可能会受到影响.潜在的调查领域:
My experience with larger files sizes has been that java.nio is faster than java.io. Solidly faster. Like in the >250% range. That said, I am eliminating obvious bottlenecks, which I suggest your micro-benchmark might suffer from. Potential areas for investigating:
缓冲区大小.您基本上拥有的算法是
- 从磁盘复制到缓冲区
- 从缓冲区复制到磁盘
我自己的经验是，此缓冲区大小对于调整成熟.我已经确定应用程序的一部分为 4KB，另一部分为 256KB.我怀疑您的代码正在遭受如此大的缓冲区的困扰.使用 1KB、2KB、4KB、8KB、16KB、32KB 和 64KB 的缓冲区运行一些基准测试以证明这一点.
My own experience has been that this buffer size is ripe for tuning. I've settled on 4KB for one part of my application, 256KB for another. I suspect your code is suffering with such a large buffer. Run some benchmarks with buffers of 1KB, 2KB, 4KB, 8KB, 16KB, 32KB and 64KB to prove it to yourself.
不要执行读取和写入同一磁盘的 Java 基准测试.
如果你这样做了，那么你实际上是在对磁盘进行基准测试，而不是 Java.我还建议，如果您的 CPU 不忙，那么您可能遇到了其他一些瓶颈.
If you do, then you are really benchmarking the disk, and not Java. I would also suggest that if your CPU is not busy, then you are probably experiencing some other bottleneck.
如果不需要，请不要使用缓冲区.
如果目标是另一个磁盘或网卡，为什么要复制到内存?对于较大的文件，产生的延迟是非常重要的.
Why copy to memory if your target is another disk or a NIC? With larger files, the latency incured is non-trivial.
正如其他人所说，使用 FileChannel.transferTo() 或 FileChannel.transferFrom().这里的关键优势是 JVM 使用操作系统对 DMA 的访问(直接内存访问)，如果展示.(这取决于实现，但是通用 CPU 上的现代 Sun 和 IBM 版本很适合使用.) 发生的情况是数据直接往返于磁盘、总线，然后到达目的地... 通过 RAM 或 CPU 绕过任何电路.
Like other have said, use FileChannel.transferTo() or FileChannel.transferFrom(). The key advantage here is that the JVM uses the OS's access to DMA (Direct Memory Access), if present. (This is implementation dependent, but modern Sun and IBM versions on general purpose CPUs are good to go.) What happens is the data goes straight to/from disc, to the bus, and then to the destination... bypassing any circuit through RAM or the CPU.
我日以继夜地工作的网络应用程序非常需要 IO.我也做过微基准测试和现实世界的基准测试.结果在我的博客上，看看:
The web app I spent my days and night working on is very IO heavy. I've done micro benchmarks and real-world benchmarks too. And the results are up on my blog, have a look-see:
使用生产数据和环境
微基准测试容易失真.如果可以，请努力从您计划执行的操作、您期望的负载和您期望的硬件上收集数据.
Micro-benchmarks are prone to distortion. If you can, make the effort to gather data from exactly what you plan to do, with the load you expect, on the hardware you expect.
我的基准测试是可靠的，因为它们是在生产系统、强大的系统、负载下的系统上进行的，这些系统收集在日志中.不是我的笔记本电脑的 7200 RPM 2.5" SATA 驱动器，而我在 JVM 工作我的硬盘时密切注视.
My benchmarks are solid and reliable because they took place on a production system, a beefy system, a system under load, gathered in logs. Not my notebook's 7200 RPM 2.5" SATA drive while I watched intensely as the JVM work my hard disc.
你在跑什么?很重要.
这篇关于Java NIO FileChannel 与 FileOutputstream 的性能/实用性的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
