如何对非常大的文件进行排序 [英] How do I sort very large files

问题描述

我有一些文件应该根据每行开头的 id 进行排序.这些文件大约为 2-3 GB.

我尝试将所有数据读入 ArrayList 并对它们进行排序.但是内存不足以保存所有这些.它不起作用.

I have some files that should be sorted according to id at the beginning of each line. The files are about 2-3 gb.

I tried to read all data into an ArrayList and sort them. But memory is not enough to keep them all. It does not work.

线条看起来像

0052304 0000004000000000000000000000000000000041 约翰泰迪 000023
0022024 0000004000000000000000000000000000000041 George Clan 00013

如何对文件进行排序??

How can I sort the files??

推荐答案

这不完全是 Java 问题.您需要研究一种有效的算法来对未完全读入内存的数据进行排序.对 Merge-Sort 进行一些调整可以实现这一点.

That isn't exactly a Java problem. You need to look into an efficient algorithm for sorting data that isn't completely read into memory. A few adaptations to Merge-Sort can achieve this.

看看这个:http://en.wikipedia.org/wiki/Merge_sort

和:http://en.wikipedia.org/wiki/External_sorting

基本上这里的想法是将文件分成更小的部分，对它们进行排序(使用合并排序或其他方法)，然后使用合并排序中的合并来创建新的排序文件.

Basically the idea here is to break the file into smaller pieces, sort them (either with merge sort or another method), and then use the Merge from merge-sort to create the new, sorted file.

这篇关于如何对非常大的文件进行排序的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！