Python 2.7:将数据追加到Pandas中的表 [英] Python 2.7: Appending Data to Table in Pandas

问题描述

我正在从图像文件中读取数据，并且希望将此数据附加到单个HDF文件中.这是我的代码:

I am reading data from image files and I want to append this data into a single HDF file. Here is my code:

datafile = pd.HDFStore(os.path.join(path,'imageData.h5'))
for file in fileList: 
     data = {'X Position' :  pd.Series(xpos, index=index1),
             'Y Position' :  pd.Series(ypos, index=index1),
             'Major Axis Length' :  pd.Series(major, index=index1),
             'Minor Axis Length' :  pd.Series(minor, index=index1), 
             'X Velocity' :  pd.Series(xVelocity, index=index1),
             'Y Velocity' :  pd.Series(yVelocity, index=index1) }
    df = pd.DataFrame(data)
    datafile['df'] = df
    datafile.close()

这显然是不正确的，因为每次循环运行时，它都会用新数据覆盖新数据集.

This is obviously incorrect as it overwrites each set of data with the new one each time the loop runs.

如果我使用

datafile.append('df',df)    

OR

df.to_hdf(os.path.join(path,'imageData.h5'), 'df', append=True, format = 'table')

我得到了错误:

ValueError: Can only append to Tables

我已经参考了文档和其他 SO问题，但无济于事.

I have referred to the documentation and other SO questions, without avail.

因此，我希望有人能解释为什么它不起作用以及如何将所有数据成功附加到一个文件中.如果有必要，我愿意使用其他方法(也许是pyTables).

So, I am hoping someone can explain why this isn't working and how I can successfully append all the data to one file. I am willing to use a different method (perhaps pyTables) if necessary.

任何帮助将不胜感激.

推荐答案

这将在0.11中运行.创建组后(例如，用于存储数据的标签，此处为"df").如果您存储的是 fixed 格式，它将被覆盖(如果尝试追加，则会出现上述错误味精)；如果您编写 table 格式，则可以 附加.请注意，在0.11中， to_hdf 不能正确地将关键字传递给基础函数，因此您只能使用它编写 fixed 格式.

This will work in 0.11. Once you create a group (e.g the label where you are storing data, the 'df' here). If you store a fixed format it will overwrite (and if you try to append will give you the above error msg); if you write a table format you can append. Note that in 0.11, to_hdf does not correctly pass keywords thru to the underlying function so you can use it ONLY to write a fixed format.

datafile = pd.HDFStore(os.path.join(path,'imageData.h5'),mode='w')
for file in fileList: 
     data = {'X Position' :  pd.Series(xpos, index=index1),
             'Y Position' :  pd.Series(ypos, index=index1),
             'Major Axis Length' :  pd.Series(major, index=index1),
             'Minor Axis Length' :  pd.Series(minor, index=index1), 
             'X Velocity' :  pd.Series(xVelocity, index=index1),
             'Y Velocity' :  pd.Series(yVelocity, index=index1) }
    df = pd.DataFrame(data)
    datafile.append('df',df)
datafile.close

这篇关于Python 2.7:将数据追加到Pandas中的表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！