任何想法如何调试这个FF错误？ [英] Any ideas on how to debug this FF error?

问题描述

我尝试使用FF读取R中的大型CSV文件。

我首先使用csv.read读取了前10行的代码，以确保我不会做任何蠢事：

  trainFileName =./TrainingSet/SplitFiles/7_train_data.csv
 trainSet <  -  read.csv（trainFileName，header = TRUE，nrows = 10）
 length（trainSet [1，]）
 length（trainSet [，1]）$ ​​b $ b  
 
 
 
这告诉我：
  trainFileName =./TrainingSet/SplitFiles/7_train_data.csv
> trainSet<  -  read.csv（trainFileName，header = TRUE，nrows = 10）
> length（trainSet [1，]）
 [1] 4505 
> length（trainSet [，1]）$ ​​b $ b [1] 10 
  
现在我尝试用FF重复这个专长：
  trainSet<  -  read.csv.ffdf（file = trainFileName，header = TRUE，nrows = 10，VERBOSE = TRUE）
  
这里我们失败：
  read.table.ffdf 1..10（10）csv-read = 0.552sec 
 if（dfile == getOption fftempdir））finalizer < - deleteelse finalizer < - close：参数的长度为零
 setwd（cwd）中的错误：预期的字符参数
  
我无法在任何地方找到关于此错误的任何更多信息，我看不到我可以做什么更简单，所以在我深入FF源，有没有任何想法？
 
 
 
我试过加载整个文件，而不是前10行，指定列数据类型和总是相同的错误。 
 
 
 
提前感谢。
解决方案
列。在ff中，每列都是一个文件。您无法打开文件系统可以同时打开的文件
 
 
 
要查看ff如果打开的文件太多就会失败的地方运行此命令：
  require（ff）
x<  -  list（）
 for（i in 1：100000）{
 print（i）
x [[i]] < -  ff（rnorm（10））
 open（x [[i]]）
} code> 
对我来说，这在1022个打开文件失败，但我还有一些其他打开文件。
 
I am trying to use FF to read a large CSV file in R.

I first read the first 10 rows of the code with csv.read to make sure I'm not doing anything stupid:
trainFileName = "./TrainingSet/SplitFiles/7_train_data.csv"
trainSet <- read.csv(trainFileName, header=TRUE, nrows=10)
length(trainSet[1,])
length(trainSet[,1])
This tells me:
> trainFileName = "./TrainingSet/SplitFiles/7_train_data.csv"
> trainSet <- read.csv(trainFileName, header=TRUE, nrows=10)
> length(trainSet[1,])
[1] 4505
> length(trainSet[,1])
[1] 10
So far so good.  Now I try to repeat this feat with FF:
trainSet <- read.csv.ffdf(file = trainFileName, header = TRUE, nrows = 10, VERBOSE = TRUE)
And here we fail with:
read.table.ffdf 1..10 (10)  csv-read=0.552sec
Error in if (dfile ==         getOption("fftempdir")) finalizer <- "delete" else finalizer <- "close" : argument is of length zero
Error in setwd(cwd) : character argument expected
I can't find any more info on this error anywhere, and I can't see how I can do anything simpler, so before I delve into the FF source, does anyone have any ideas?

I've tried loading the whole file instead of the first 10 rows, specifying column data types and always the same error.

Thanks in advance.
解决方案
yes, you have too many columns. In ff, each column is a file. You can not open more files than your filesystem can open at the same time

To have a look where ff will fail if you have too many files open run this:
require(ff)
x <- list()
for(i in 1:100000){
  print(i)
  x[[i]] <- ff(rnorm(10))
  open(x[[i]] )
}
For me, this failed at 1022 open files but I still had a few other open as well.

                        
这篇关于任何想法如何调试这个FF错误？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！