添加新的“列”到csv数据文件 [英] Adding new "columns" to csv data file in Tcl

问题描述

我正在处理一个大测量数据，大约30K个键值
对。测量具有迭代次数。在每次迭代之后，创建具有30K kay-value对的
数据文件（非csv）。我想以某种方式
creata一个形式的csv文件：

I am dealing with a "large" measurement data, approximately 30K key-value pairs. The measurements have number of iterations. After each iteration a datafile (non-csv) with 30K kay-value pairs is created. I want to somehow creata a csv file of form:

Key1,value of iteration1,value of iteration2,...
Key2,value of iteration1,value of iteration2,...
Key2,value of iteration1,value of iteration2,...
...

现在，我想知道如何有效地将每个迭代测试
数据作为列添加到Tcl中的csv文件。所以，似乎在任何一种情况下，I
将需要加载整个csv文件到一些变量（数组/列表）和工作
每个元素通过添加新的测量数据。这似乎有点低效。
有另一种方法，可能吗？

Now, I was wondering about efficient way of adding each iteration mesurement data as a columns to csv file in Tcl. So, far it seems that in either case I will need to load whole csv file into some variable(array/list) and work on each element by adding new measurement data. This seems somewhat inefficient. Is there another way, perhaps?

推荐答案

因为CSV文件基本上是文本文件，数据输入和再写出。没有其他方法来扩展列数，因为数据基本上是行主。最简单的方法来做你想说的话（毕竟，30k对不是这么多）是使用 csv package 来做解析工作。这个代码可以做你正在寻找...

Because CSV files are fundamentally text files, you have to load all the data in and write it out again. There's no other way to expand the number of columns since the data is fundamentally row-major. The easiest way to do what you say you want (after all, 30k-pairs isn't that much) is to use the csv package to do the parsing work. This code might do what you're looking for…

package require csv
package require struct::matrix

# Load the file into a matrix
struct::matrix data
set f [open mydata.csv]
csv::read2matrix $f data , auto
close $f

# Add your data
set newResults {}
foreach key [data get column 0] {
    lappend newResults [computeFrom $key]; # This is your bit!
}
data add column $newResults

# Write back out again
set f [open mydata.csv w]
csv::writematrix data $f
close $f

你最好使用数据库。 metakit 和 sqlite3 与Tcl工作得很好，并能很好地处理这种任务。

You would probably be better off using a database though. Both metakit and sqlite3 work very well with Tcl, and handle this sort of task well.

这篇关于添加新的“列”到csv数据文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！