任何提高读取文件性能的方法，都优于缓冲读取器 [英] Any way to improve the performance for reading a file ,better than buffered reader

问题描述

我正在通过缓冲读取器机制读取日志文件，以毫秒为单位的总执行时间:12944，请告知我如何提高性能并降低这个时间，请告知 nio 比缓冲读取器的性能更好..!!文件大小为 10MB，因为它是一个日志文件..！！还请告知如何用 nio 也可以实现同样的事情..!!

I am reading a log file through buffered reader mechanism which is taking Total execution time taken in millis: 12944 ,please advise how can I improve the performance and bring down this time , Please advise is nio is more better performance than buffered Reader..!! The file size is of 10MB since it is a log file..!! please advise also how this same thing could be achieved with nio also..!!

public class BufferedRedeem
{

    public static void main(String[] args)
    {

        BufferedReader br = null;
        long startTime = System.currentTimeMillis();

        try
        {
            String sCurrentLine;
            br = new BufferedReader(new FileReader("C://abc.log"));

            while ((sCurrentLine = br.readLine()) != null)
            {

            }
            long elapsedTime = System.currentTimeMillis() - startTime;

            System.out.println("Total execution time taken in millis: " + elapsedTime);
        }
        catch (IOException e)
        {
            e.printStackTrace();
        }
        finally
        {
            try
            {
                if (br != null)
                    br.close();
            }
            catch (IOException ex)
            {
                ex.printStackTrace();
            }
        }
    }
}

推荐答案

因为 OP 很想知道如何使用 NIO 来完成这项工作.

Since the OP is keen to see how this could be done using NIO.

由于文件较小，很难看出差异，但可以测量.

As the file is small, it is hard to see the difference but it can be measured.

public static void main(String... args) throws IOException {
    PrintWriter pw = new PrintWriter("abc.log");
    for (int i = 0; i < 100 * 1000; i++) {
        pw.println("0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789");
    }
    pw.close();

    long start2 = System.nanoTime();
    int count2 = 0;
    BufferedReader br = new BufferedReader(new FileReader("abc.log"));
    while (br.readLine() != null) count2++;
    br.close();
    long time2 = System.nanoTime() - start2;
    System.out.printf("IO: Took %,d ms to read %,d lines%n", time2 / 1000 / 1000, count2);

    long start = System.nanoTime();
    FileChannel fc = new FileInputStream("abc.log").getChannel();
    ByteBuffer bb = ByteBuffer.allocateDirect((int) fc.size());
    fc.read(bb);
    fc.close();
    bb.flip();

    CharBuffer cb = ByteBuffer.allocateDirect(bb.remaining() * 2).order(ByteOrder.nativeOrder()).asCharBuffer();
    CharsetDecoder cd = Charset.forName("UTF-8").newDecoder();
    cd.decode(bb, cb, true);
    cb.flip();
    StringBuilder sb = new StringBuilder();
    int count = 0;
    while (cb.remaining() > 0) {
        char ch = cb.get();
        if (isEndOfLine(cb, ch)) {
            // process sb
            count++;
            sb.setLength(0);
        } else {
            sb.append(ch);
        }
    }
    long time = System.nanoTime() - start;
    System.out.printf("NIO as UTF-8: Took %,d ms to read %,d lines%n", time / 1000 / 1000, count);

    long start3 = System.nanoTime();
    FileChannel fc2 = new FileInputStream("abc.log").getChannel();
    MappedByteBuffer bb2 = fc2.map(FileChannel.MapMode.READ_ONLY, 0, fc2.size());
    bb.flip();
    StringBuilder sb3 = new StringBuilder();
    int count3 = 0;
    while (bb2.remaining() > 0) {
        char ch = (char) bb2.get();
        if (isEndOfLine(bb2, ch)) {
            // process sb
            count3++;
            sb3.setLength(0);
        } else {
            sb3.append(ch);
        }
    }
    fc2.close();
    long time3 = System.nanoTime() - start3;
    System.out.printf("NIO as ISO-8859-1: Took %,d ms to read %,d lines%n", time3 / 1000 / 1000, count3);


}

private static boolean isEndOfLine(CharBuffer cb, char ch) {
    if (ch == '
') {
        if (cb.remaining() >= 1 && cb.get() == '
') {
            return true;
        }
        cb.position(cb.position() - 1);
        return true;
    } else if (ch == '
') {
        return true;
    }
    return false;
}

private static boolean isEndOfLine(ByteBuffer bb, char ch) {
    if (ch == '
') {
        if (bb.remaining() >= 1 && bb.get() == '
') {
            return true;
        }
        bb.position(bb.position() - 1);
        return true;
    } else if (ch == '
') {
        return true;
    }
    return false;
}

打印每行 102 字节长，因此文件大约为 10 MB.

prints each line is 102 bytes long so the file is ~ 10 MB.

IO: Took 112 ms to read 100,000 lines
NIO as UTF-8: Took 207 ms to read 100,000 lines
NIO as ISO-8859-1: Took 87 ms to read 100,000 lines

正如我之前提到的，使用 NIO 节省 35 毫秒的额外复杂性不太值得.

As I mentioned before, its unlikely to be worth the extra complexity of using NIO to save 35 ms.

顺便说一句:如果你有一个 HDD 并且文件不在内存中，那么只有你的驱动器的速度很重要.

BTW: If you have a HDD and the file is not in memory, only the speed of your drive will matter.

这篇关于任何提高读取文件性能的方法，都优于缓冲读取器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！