Java的:写大文件? [英] java : writing large files?

问题描述

问候， 我从数据库中获取了大量记录并将其写入文件.我想知道写入大文件的最佳方法是什么. (1Gb-10Gb).

Greetings , I get huge number of records from database and write into a file.I was wondering what the best way to write huge files. (1Gb - 10Gb).

当前我正在使用BufferedWriter

Currently I am using BufferedWriter

BufferedWriter mbrWriter=new BufferedWriter(new FileWriter(memberCSV));
while(done){
 //do writings
}
mbrWriter.close();

推荐答案

如果您真的坚持使用Java，那么最好的方法是在数据输入后立即立即编写因此，不要先将 all 的数据从ResultSet收集到Java的内存中.否则，您至少需要Java中那么多的可用内存.

If you really insist using Java for this, then the best way would be to write immediately as soon as the data comes in and thus not to collect all the data from ResultSet into Java's memory first. You would need at least that much of free memory in Java otherwise.

因此，例如

while (resultSet.next()) {
    writer.write(resultSet.getString("columnname"));
    // ...
}

也就是说，大多数体面的DB都带有内置的CSV导出功能，这些功能无疑比Java中的效率更高.您没有提到要使用哪个版本，但是如果以MySQL为例，则可以使用

That said, most decent DB's ships with builtin export-to-CSV capabilities which are undoubtely way more efficient than you could ever do in Java. You didn't mention which one you're using, but if it was for example MySQL, you could have used the LOAD DATA INFILE for this. Just refer the DB-specific documentation. Hope this gives new insights.

这篇关于Java的:写大文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！