输入流非常缓慢地读取大文件，为什么？ [英] Input stream reads large files very slowly, why?

问题描述

我试图提交一个500 MB的文件。
我可以加载它，但我想提高性能。
这是慢速代码：

  File dest = getDestinationFile（source，destination）; 
 if（dest == null）return false; 
 
 in = new BufferedInputStream（new FileInputStream（source））; 
 out = new BufferedOutputStream（new FileOutputStream（dest））; 
 byte [] buffer = new byte [1024 * 20]; 
 int i = 0; 
 
 //这个while循环非常慢
 while（（i = in.read（buffer））！= -1）{
 out.write（buffer，0，一世）; //<  -  SLOW HERE 
 out.flush（）; 
 
 $ / code> 

如何找到缓慢的原因？

不是字节数组大小/缓冲区大小是否足够？
您有什么想法可以提高性能吗？



感谢您提前帮忙！


解决方案

你不应该循环刷新。
您正在使用BufferedOutputStream。这意味着在缓存一些数据量之后，会将数据刷新到文件中。
您的代码只是在写入少量数据之后刷新数据来杀死性能。

尝试这样做：

  while（（i = in.read（buffer））！= -1）{
 out.write（buffer，0，i）; <  -  SLOW HERE 
} 
 out.flush（）; 
  

.. ::编辑：在以下评论的回应:: ..

在我看来，你根本不应该使用缓冲区。您正在使用缓冲（输出/输入）流，这意味着它们有自己的缓冲区来读取磁盘中的数据包并保存数据包。我不是100％确定在使用额外的缓冲区的性能，但我希望你能显示我会怎么做：

 文件dest = getDestinationFile（source，destination）; 
 if（dest == null）return false; 
 
 in = new BufferedInputStream（new FileInputStream（source））; 
 out = new BufferedOutputStream（new FileOutputStream（dest））; 
 
 int i; （（i = in.read（））！= -1）{
 out.write（i）; 
 while 
} 
 out.flush（）; 
  

在我的版本中，您只需要阅读一个BYTE（no int。Read doc：
< a href =http://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html#read%28%29 =nofollow> http://docs.oracle。 com / javase / 7 / docs / api / java / io / InputStream.html＃read（）

这个方法返回int，但这只是一个BYTE），但是没有必要读一个完整的缓冲区（所以你不必担心它的大小）。



也许你应该多读一些关于流的东西，以便更好地理解与它们有关的东西。

I am trying to submit a 500 MB file. I can load it but I want to improve the performance. This is the slow code:

File dest = getDestinationFile(source, destination);
if(dest == null) return false;

in = new BufferedInputStream(new  FileInputStream(source));
out = new BufferedOutputStream(new  FileOutputStream(dest));
byte[] buffer = new byte[1024 * 20];
int i = 0;

// this while loop is very slow
while((i = in.read(buffer)) != -1){
   out.write(buffer, 0, i); //<-- SLOW HERE
   out.flush();
}

How can I find why it is slow?
Isn't the byte array size / buffer size sufficient? Do you have any ideas to improve the performance or?

Thanks in advance for any help

解决方案

You should not flush in loop. You are using BufferedOutputStream. This mean that after "caching" some amount of data it flushes data to file. Your code just kills performance by flushing data after writing a little amount of data.

try do this like that:

while((i = in.read(buffer)) != -1){
out.write(buffer, 0, i); <-- SLOW HERE
}
out.flush();

..:: Edit: in response of comment below ::..
In my opinion you should not use buffer at all. You are using Buffered(Output/Input)Stream which means that they have his own buffer to read "package" of data from disk and save "package" of data. Im not 100% sure about performance in using additional buffer but I want you to show how I would do that:

File dest = getDestinationFile(source, destination);
if(dest == null) return false;

in = new BufferedInputStream(new  FileInputStream(source));
out = new BufferedOutputStream(new  FileOutputStream(dest));

int i;
while((i = in.read()) != -1){
   out.write(i);
}
out.flush();

In my version you will just read a BYTE (no a int. Read doc:
http://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html#read()
this method returns int but this is just a BYTE) but there is no need to read a whole buffer (so you don't need to be worry about size of it).

Probably you should read more about streams to better understand what is nessesary to do with them.

这篇关于输入流非常缓慢地读取大文件，为什么？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！