BufferedReader用于大型ByteBuffer？ [英] BufferedReader for large ByteBuffer?

问题描述

有没有办法用BufferedReader读取ByteBuffer而不必先将它变成String？我想通过一个相当大的ByteBuffer读取文本行，出于性能原因，我想避免将其写入磁盘。在ByteBuffer上调用toString不起作用，因为生成的String太大（它会抛出java.lang.OutOfMemoryError：Java堆空间）。我原以为在API中会有一些东西将ByteBuffer包装在合适的阅读器中，但我似乎找不到合适的东西。

Is there a way to read a ByteBuffer with a BufferedReader without having to turn it into a String first? I want to read through a fairly large ByteBuffer as lines of text and for performance reasons I want to avoid writing it to the disk. Calling toString on the ByteBuffer doesn't work because the resulting String is too large (it throws java.lang.OutOfMemoryError: Java heap space). I would have thought there would be something in the API to wrap a ByteBuffer in a suitable reader, but I can't seem to find anything suitable.

这是一个缩写代码示例说明了我在做什么）：

Here's an abbreviated code sample the illustrates what I am doing):

// input stream is from Process getInputStream()
public String read(InputStream istream)
{
  ReadableByteChannel source = Channels.newChannel(istream);
  ByteArrayOutputStream ostream = new ByteArrayOutputStream(bufferSize);
  WritableByteChannel destination = Channels.newChannel(ostream);
  ByteBuffer buffer = ByteBuffer.allocateDirect(writeBufferSize);

  while (source.read(buffer) != -1)
  {
    buffer.flip();
    while (buffer.hasRemaining())
    {
      destination.write(buffer);
    }
    buffer.clear();
  }

  // this data can be up to 150 MB.. won't fit in a String.
  result = ostream.toString();
  source.close();
  destination.close();
  return result;
}

// after the process is run, we call this method with the String
public void readLines(String text)
{
  BufferedReader reader = new BufferedReader(new StringReader(text));
  String line;

  while ((line = reader.readLine()) != null)
  {
    // do stuff with line
  }
}

推荐答案

目前尚不清楚为什么要使用字节缓冲区来启动用。如果你有一个 InputStream 并且你想读取它的行，你为什么不使用 InputStreamReader 包裹在 BufferedReader 中？让NIO参与的好处是什么？

It's not clear why you're using a byte buffer to start with. If you've got an InputStream and you want to read lines for it, why don't you just use an InputStreamReader wrapped in a BufferedReader? What's the benefit in getting NIO involved?

在 ByteArrayOutputStream上调用 toString（） / code>对我来说听起来不错，即使你有足够的空间：最好将它作为一个字节数组并将其包装在 ByteArrayInputStream 中然后是 InputStreamReader ，如果你真的必须有一个 ByteArrayOutputStream 。如果确实想要调用 toString（），至少要使用带有字符编码名称的重载 - 否则它会使用系统默认值，这可能不是你想要的。

Calling toString() on a ByteArrayOutputStream sounds like a bad idea to me even if you had the space for it: better to get it as a byte array and wrap it in a ByteArrayInputStream and then an InputStreamReader, if you really have to have a ByteArrayOutputStream. If you really want to call toString(), at least use the overload which takes the name of the character encoding to use - otherwise it'll use the system default, which probably isn't what you want.

编辑：好的，所以你真的想要使用NIO。您最终仍在写入 ByteArrayOutputStream ，因此您最终会获得包含数据的BAOS。如果您想避免复制该数据，则需要从 ByteArrayOutputStream 派生，例如：

Okay, so you really want to use NIO. You're still writing to a ByteArrayOutputStream eventually, so you'll end up with a BAOS with the data in it. If you want to avoid making a copy of that data, you'll need to derive from ByteArrayOutputStream, for instance like this:

public class ReadableByteArrayOutputStream extends ByteArrayOutputStream
{
    /**
     * Converts the data in the current stream into a ByteArrayInputStream.
     * The resulting stream wraps the existing byte array directly;
     * further writes to this output stream will result in unpredictable
     * behavior.
     */
    public InputStream toInputStream()
    {
        return new ByteArrayInputStream(array, 0, count);
    }
}

然后你可以创建输入流，将其包装成一个 InputStreamReader ，将其包装在 BufferedReader 中，然后你就离开了。

Then you can create the input stream, wrap it in an InputStreamReader, wrap that in a BufferedReader, and you're away.

这篇关于BufferedReader用于大型ByteBuffer？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！