我可以在R中并行读取1个大CSV文件吗？ [英] Can I read 1 big CSV file in parallel in R?

问题描述

我有一个大的csv文件，它需要老年读。我可以在R中使用像parallel或相关的包并行读取这个？我尝试使用mclapply，但它不工作。

I have a big csv file and it takes ages to read. Can I read this in parallel in R using a package like "parallel" or related? I've tried using mclapply, but it is not working.

推荐答案

根据OP的注释， fread 从 data.table 包工作。代码如下：

Based upon the comment by the OP, fread from the data.table package worked. Here's the code:

library(data.table)
dt <- fread("myFile.csv")

在OP的情况下，读取一个1.2GB的文件并且读取 .csv 花了大约4-5分钟，只需14秒钟 fread 。

In the OP's case, read in time for a 1.2GB file with read.csv it took about 4-5 minutes and just 14 seconds with fread.

这篇关于我可以在R中并行读取1个大CSV文件吗？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！