方式存储大量的实验和相应的结果 [英] Way to store large amount of experiments and corresponding results

问题描述

我有一个实验设置，每次运行生成几个csv和文本文件。在我的研究中，我每天并行运行一些并尝试存储和组织他们在一个目录继承人正在变得相当低效和乏味。

I have an experiment setup that produces a few csv and text files every run. In my research I run a number of these in parallel each day and trying to store and organized them in a directory heir-achy is getting rather inefficient and tedious.

一个想法我想做的是使用MySQL或某物存储运行描述以及相关的输出。然而，这样的效率低，我在表字段中存储了一些巨大的字符串。

One idea I thought of doing was using MySQL or something to store the the run description along with the associated output. This however has the inefficiency in that I am storing a number of gigantic Strings in the table fields.

基本上我想知道一个系统是否存在，可以自动组织输出。这样它将给出运行描述的列表，并且在选择此项时，您可以请求相应的输出。

Basically I would like to know if a system exists that can automatically organize the output of runs. Such that it would give a list of the run descriptions, and upon choosing this you can request the corresponding outputs. This wouldn't have to be in any particular language as I can hopefully just add the step to a shell script.

推荐答案

我可以使用任何特定的语言假设csv和txt文件对于每次运行减去扩展名而相同地命名。你考虑过一个简单的java或python脚本，它可以取得一个文件夹中的所有csv和txt文件，并将它们放入一个XML文件，其中所有的信息将被保存，以及来自链接csv文件和txt文件的数据在一起，这意味着对于几百个文件对（csv和txt），你可以结束一个单独的XML文件，可以打开以查看单独的运行结果？

I assume that the csv and txt files are identically named for each run minus the extensions. Have you considered a simple java or python script that could take all of the csv and txt files in a folder and put them into a single XML file where all of the information would be saved and with the data from the linked csv file and txt file together, meaning that for hundreds of file pairs (csv and txt) you could end up with a single XML file that could be opened to see the separate run results?

这篇关于方式存储大量的实验和相应的结果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！