任何提高读取文件性能的方法，都优于缓冲读卡器 [英] Any way to improve the performance for reading a file ,better than buffered reader

问题描述

我正在通过缓冲读卡器机制读取一个日志文件，该机制占用了以毫秒为单位的总执行时间：12944，请告知我如何提高性能并降低此时间，请告知nio是否比缓冲读卡器更好的性能.. !!文件大小为10MB，因为它是一个日志文件.. !!请告知如何使用nio也可以实现同样的目的.. !!

I am reading a log file through buffered reader mechanism which is taking Total execution time taken in millis: 12944 ,please advise how can I improve the performance and bring down this time , Please advise is nio is more better performance than buffered Reader..!! The file size is of 10MB since it is a log file..!! please advise also how this same thing could be achieved with nio also..!!

public class BufferedRedeem
{

    public static void main(String[] args)
    {

        BufferedReader br = null;
        long startTime = System.currentTimeMillis();

        try
        {
            String sCurrentLine;
            br = new BufferedReader(new FileReader("C://abc.log"));

            while ((sCurrentLine = br.readLine()) != null)
            {

            }
            long elapsedTime = System.currentTimeMillis() - startTime;

            System.out.println("Total execution time taken in millis: " + elapsedTime);
        }
        catch (IOException e)
        {
            e.printStackTrace();
        }
        finally
        {
            try
            {
                if (br != null)
                    br.close();
            }
            catch (IOException ex)
            {
                ex.printStackTrace();
            }
        }
    }
}

推荐答案

由于OP很想知道如何使用NIO完成这项工作。

Since the OP is keen to see how this could be done using NIO.

由于文件较小，很难看到区别但可以测量。

As the file is small, it is hard to see the difference but it can be measured.

public static void main(String... args) throws IOException {
    PrintWriter pw = new PrintWriter("abc.log");
    for (int i = 0; i < 100 * 1000; i++) {
        pw.println("0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789");
    }
    pw.close();

    long start2 = System.nanoTime();
    int count2 = 0;
    BufferedReader br = new BufferedReader(new FileReader("abc.log"));
    while (br.readLine() != null) count2++;
    br.close();
    long time2 = System.nanoTime() - start2;
    System.out.printf("IO: Took %,d ms to read %,d lines%n", time2 / 1000 / 1000, count2);

    long start = System.nanoTime();
    FileChannel fc = new FileInputStream("abc.log").getChannel();
    ByteBuffer bb = ByteBuffer.allocateDirect((int) fc.size());
    fc.read(bb);
    fc.close();
    bb.flip();

    CharBuffer cb = ByteBuffer.allocateDirect(bb.remaining() * 2).order(ByteOrder.nativeOrder()).asCharBuffer();
    CharsetDecoder cd = Charset.forName("UTF-8").newDecoder();
    cd.decode(bb, cb, true);
    cb.flip();
    StringBuilder sb = new StringBuilder();
    int count = 0;
    while (cb.remaining() > 0) {
        char ch = cb.get();
        if (isEndOfLine(cb, ch)) {
            // process sb
            count++;
            sb.setLength(0);
        } else {
            sb.append(ch);
        }
    }
    long time = System.nanoTime() - start;
    System.out.printf("NIO as UTF-8: Took %,d ms to read %,d lines%n", time / 1000 / 1000, count);

    long start3 = System.nanoTime();
    FileChannel fc2 = new FileInputStream("abc.log").getChannel();
    MappedByteBuffer bb2 = fc2.map(FileChannel.MapMode.READ_ONLY, 0, fc2.size());
    bb.flip();
    StringBuilder sb3 = new StringBuilder();
    int count3 = 0;
    while (bb2.remaining() > 0) {
        char ch = (char) bb2.get();
        if (isEndOfLine(bb2, ch)) {
            // process sb
            count3++;
            sb3.setLength(0);
        } else {
            sb3.append(ch);
        }
    }
    fc2.close();
    long time3 = System.nanoTime() - start3;
    System.out.printf("NIO as ISO-8859-1: Took %,d ms to read %,d lines%n", time3 / 1000 / 1000, count3);


}

private static boolean isEndOfLine(CharBuffer cb, char ch) {
    if (ch == '\r') {
        if (cb.remaining() >= 1 && cb.get() == '\n') {
            return true;
        }
        cb.position(cb.position() - 1);
        return true;
    } else if (ch == '\n') {
        return true;
    }
    return false;
}

private static boolean isEndOfLine(ByteBuffer bb, char ch) {
    if (ch == '\r') {
        if (bb.remaining() >= 1 && bb.get() == '\n') {
            return true;
        }
        bb.position(bb.position() - 1);
        return true;
    } else if (ch == '\n') {
        return true;
    }
    return false;
}

打印每行长度为102个字节，因此文件大小为~10 MB。 / p>

prints each line is 102 bytes long so the file is ~ 10 MB.

IO: Took 112 ms to read 100,000 lines
NIO as UTF-8: Took 207 ms to read 100,000 lines
NIO as ISO-8859-1: Took 87 ms to read 100,000 lines

正如我所提到的之前，它不太可能值得使用NIO节省35毫秒的额外复杂性。

As I mentioned before, its unlikely to be worth the extra complexity of using NIO to save 35 ms.

顺便说一句：如果你有一个硬盘并且文件不在内存中，那么只有硬盘的速度才有意义。

BTW: If you have a HDD and the file is not in memory, only the speed of your drive will matter.

这篇关于任何提高读取文件性能的方法，都优于缓冲读卡器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！