更快的方式来读取文件 [英] Faster way to read file

问题描述

我正在研究一个有大约400个输入文件和大约40个输出文件的程序。
这很简单：它读取每个输入文件，并生成一个新的文件，但更大（基于算法）。

我正在使用read ）方法从BufferedReader：

 字符串编码=ISO-8859-1; 
 FileInputStream fis = new FileInputStream（nextFile）; 
 BufferedReader reader = new BufferedReader（new InputStreamReader（fis，encoding））; 
 char [] buffer = new char [8192]; 
  

要读取我使用的输入文件：

  private String getNextBlock（）throws IOException {
 boolean isNewFile = false; 
 
 int n = reader.read（buffer，0，buffer.length）; 
 if（n == -1）{
 return null; 
} else {
 return new String（buffer，0，n）; 
 
 
 
 
 $ b $ p 
 $ b 
每个块都在进行一些检查（比如看块中的一些字符串），然后我写它到一个文件：
 
 
  BufferedWriter writer = new BufferedWriter（new OutputStreamWriter （
 FileOutputStream（fileName），encoding））; 
 
 writer.write（textToWrite）; 
  
问题是需要大约12分钟。 
我试图找到更快的东西。 
任何人都有一些更好的想法？
 
 
 
谢谢。
解决方案
您应该可以在这里找到答案： 
 
 
  http://nadeausoftware.com/articles/2008/02/java_tip_how_read_files_quickly  
 
 
 
为了获得最佳的Java读取性能，要记住的事情：
 
 $ ul 
 
 
通过一次读取一个数组而不是一次一个字节来最小化I / O操作。一个8K字节的数组是一个很好的大小。
 
 
通过一次获取一个数组而不是一次一个字节来最小化方法调用。使用数组索引来获取数组中的字节。
  
 
 
如果您不需要线程安全性，请最小化线程同步锁。可以减少对线程安全类的方法调用，也可以使用非线程安全的类，如FileChannel和MappedByteBuffer。
 最小化JVM之间的数据复制/ OS，内部缓冲区和应用程序阵列。使用带有内存映射的FileChannel，或直接或包装数组ByteBuffer。 
  
I am working on a program that has about 400 input files and about 40 output files.
It's simple: It reads each input file and it generates a new file with but much bigger(based on a algorithm).

I'm using read() method from BufferedReader:
String encoding ="ISO-8859-1";
FileInputStream fis = new FileInputStream(nextFile);
BufferedReader reader = new BufferedReader(new InputStreamReader(fis, encoding));
char[] buffer = new char[8192] ;
To read the input files I'm using this: 
private String getNextBlock() throws IOException{
    boolean isNewFile = false;

    int n = reader.read(buffer, 0, buffer.length);
    if(n == -1) {
        return null;
    } else {
        return new String(buffer,0,n);
    }
}
With each block I'm doing some checkings (like looking some string inside the block) and then I'm writing it into a file:
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(
        new FileOutputStream("fileName"), encoding));

writer.write(textToWrite);
The problem is that it takes about 12 minutes.
I'm trying to find something else much faster.
Anyone have some idea about something better?

Thanks.
解决方案
You should be able to find a answer here:

http://nadeausoftware.com/articles/2008/02/java_tip_how_read_files_quickly

For the best Java read performance, there are four things to remember:


Minimize I/O operations by reading an array at a time, not a byte at a time. An 8Kbyte array is a good size.
Minimize method calls by getting data an array at a time, not a byte at a time. Use array indexing to get at bytes in the array.
Minimize thread synchronization locks if you don't need thread safety. Either make fewer method calls to a thread-safe class, or use a non-thread-safe class like FileChannel and MappedByteBuffer.
Minimize data copying between the JVM/OS, internal buffers, and application arrays. Use FileChannel with memory mapping, or a direct or wrapped array ByteBuffer.


                        
这篇关于更快的方式来读取文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！