将来自大型csv文件的小随机样本加载到R数据框中 [英] Load a small random sample from a large csv file into R data frame

问题描述

要处理的csv文件不适合内存。

The csv file to be processed does not fit into the memory. How can one read ~20K random lines of it to do basic statistics on the selected data frame?

推荐答案

你也可以只做一些基本的统计数据

You can also just do it in the terminal with perl.

perl -ne'print if（rand（）< .01）'biglist.txt> subset.txt

这不一定会给你2万行。 （这里它会占用大约.01或总行数的1％）。但是，它会真的很快，你会有一个很好的副本的两个文件在你的目录。然后，您可以加载较小的文件到R，但你想要的。

This won't necessarily get you exactly 20,000 lines. (Here it'll grab about .01 or 1% of the total lines.) It will, however, be really really fast, and you'll have a nice copy of both files in your directory. You can then load the smaller file into R however you want.

这篇关于将来自大型csv文件的小随机样本加载到R数据框中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！