读取Java中的大文件 [英] Read large files in Java

问题描述

我需要一个知道Java的人的建议和内存问题。
我有一个大文件（像1.5GB），我需要剪切这个文件（例如100个小文件）较小的文件。

我通常知道如何做（使用 BufferedReader ），但我想知道你是否有任何有关内存的建议，或提示如何更快地执行。 p>

我的文件包含文本，它不是二进制的，我每行大约有20个字符。

解决方案

首先，如果你的文件包含二进制数据，那么使用 BufferedReader 会是一个很大的错误（因为你将数据转换成String，这是不必要的并可能容易地破坏数据）;您应该使用 BufferedInputStream 。如果它是文本数据，并且您需要沿着线条拆分，那么使用 BufferedReader 可以（假设文件包含明确的长度）。

关于内存，如果您使用了体积大小的缓冲区（我将至少使用1MB来确保HD大部分是顺序读取和写入），则不应该有任何问题。 >

如果速度是一个问题，你可以看看 java.nio 包 - 这些应该更快比 java.io ，

I need the advice from someone who knows Java very well and the memory issues. I have a large file (something like 1.5GB) and I need to cut this file in many (100 small files for example) smaller files.

I know generally how to do it (using a BufferedReader), but I would like to know if you have any advice regarding the memory, or tips how to do it faster.

My file contains text, it is not binary and I have about 20 character per line.

解决方案

First, if your file contains binary data, then using BufferedReader would be a big mistake (because you would be converting the data to String, which is unnecessary and could easily corrupt the data); you should use a BufferedInputStream instead. If it's text data and you need to split it along linebreaks, then using BufferedReader is OK (assuming the file contains lines of a sensible length).

Regarding memory, there shouldn't be any problem if you use a decently sized buffer (I'd use at least 1MB to make sure the HD is doing mostly sequential reading and writing).

If speed turns out to be a problem, you could have a look at the java.nio packages - those are supposedly faster than java.io,

这篇关于读取Java中的大文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！