将大文件读入R [英] Reading large files into R

问题描述

我是R的新手，但我知道它扼杀了大文件。我试图读取一个200MB的数据文件。我已经尝试了在csv格式，也将其转换为制表符分隔的txt，但在这两种情况下，我用尽了我的4GB的RAM文件加载之前。

I am a newbie to R, but I am aware that it chokes on "big" files. I am trying to read a 200MB data file. I have tried it in csv format and also converting it to tab delimited txt but in both cases I use up my 4GB of RAM before the file loads.

是正常的R将使用4GB或内存来加载一个200MB的文件，或者可能有错误的文件，它正在导致R除了数据之外还读一堆空白？

Is it normal that R would use 4GB or memory to load a 200MB file, or could there be something wrong with the file and it is causing R to keep reading a bunch of nothingness in addition to the data?

推荐答案

从？read.table

Less memory will be used if colClasses is specified as one of the six atomic vector classes.

...

Using nrows, even as a mild over-estimate, will help memory usage.

使用这两个参数。

请确保为数值数据正确指定 numeric 。请参阅：在read.csv 中指定colClasses

Ensure that you properly specify numeric for your numeric data. See here: Specifying colClasses in the read.csv

不要低估nrows。

And do not under-estimate nrows.

如果您运行的是64位R，则可以尝试32位版本。

If you're running 64-bit R, you might try the 32-bit version. It will use less memory to hold the same data.

另请参阅：在R中扩展内存大小限制

See here also: Extend memory size limit in R

这篇关于将大文件读入R的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！