在Java中有效地从URL读取文件到byte [] [英] Efficiently read file from URL into byte[] in Java

问题描述

我试图找到一个更高效的从远程URL读取文件并将其保存到字节数组中的方法。这是我目前有：

 私人字节[] fetchRemoteFile（字符串位置）抛出异常{
 URL url =新的网址（位置）; 
 InputStream is = null; 
 byte [] bytes = null; 
 try {
 is = url.openStream（）; 
 bytes = IOUtils.toByteArray（is）; 
} catch（IOException e）{
 //处理错误
} 
 finally {
 if（is！= null）is.close（）; 
} 
返回字节; 
 
 code 
 
 
 $ b 
正如你所看到的，我现在将URL传递给方法，使用InputStream对象来读取文件的字节。此方法使用Apache Commons IOUtils。但是，这种方法调用往往需要相对较长的时间来运行。当一个接一个地检索数百，数千或数十万个文件时，它变得非常慢。有没有办法改进这种方法，使其运行更有效率？我已经考虑多线程，但我想保存这是最后的手段。
 
解决方案
你做的方式似乎是绝对好的。
 
但是如果你说：
 
 
 
 
然而，这种方法调用趋于花费相当长的时间到
运行
 
 
您可能有以下问题：
 
 
 
 
 
网络，连接问题
  
 
 
如果您正在使用多线程，请确保虚拟机参数 -XmsYYYYM 和。
   -XmxYYYYM 配置好，因为如果不是你可以面对问题，那你的处理器就不用全部核心。前段时间我遇到过这个问题。 
I'm trying to find a more efficient method of reading a file from a remote URL and saving it into a byte array. Here is what I currently have:
private byte[] fetchRemoteFile(String location) throws Exception {
  URL url = new URL(location);
  InputStream is = null;
  byte[] bytes = null;
  try {
    is = url.openStream ();
    bytes = IOUtils.toByteArray(is);
  } catch (IOException e) {
    //handle errors
  }
  finally {
    if (is != null) is.close();
  }
  return bytes;
}
As you can see, I currently pass the URL into the method, where it uses an InputStream object to read in the bytes of the file. This method uses Apache Commons IOUtils. However, this method call tends to take a relatively long time to run. When retrieving hundreds, thousands, or hundreds of thousands of files one right after another, it gets quite slow. Is there a way I could improve this method so that it runs more efficiently? I have considered multithreading but I would like to save that as a last resort.
解决方案
Your way of doing it seems like absolutely ok.

But if you saying:

  
"However, this method call tends to take a relatively long time to
  run"
You can have follow problems :


Network, connection issue
Are you sure that download each file in separate thread?


If you are using multithreading for that, be sure that VM args -XmsYYYYM and -XmxYYYYM  configured well, because if not you can face problem , that your processor not using all cores. I have faced this problem some time ago.

                        
这篇关于在Java中有效地从URL读取文件到byte []的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！