如何在追加模式下将DataFrame导出为to_json-Python Pandas? [英] How to export DataFrame to_json in append mode - Python Pandas?

问题描述

我有一个字典列表格式的现有json文件.

I have an existing json file in a format of list of dicts.

$cat output.json
[{'a':1, 'b':2}, {'a':2, 'b':3}]

我有一个DataFrame

And I have a DataFrame

df = pd.DataFrame({'a':pd.Series([1,2], index=list('CD')), \
              "b":pd.Series([3,4], index=list('CD')})

我想用to_json保存"df"以将其附加到文件output.json:

I want to save "df" with to_json to append it to file output.json:

df.to_json('output.json', orient='records')  #  mode='a' not available for to_json

* to_csv有添加模式='a'，但to_json确实没有.

* There is append mode='a' for to_csv, but not for to_json really.

预期生成的output.json文件将是:

The expected generated output.json file will be:

    [{'a':1, 'b':2}, {'a':2, 'b':3}, {'a':1, 'b':3}, {'a':2, 'b':4}]

现有文件output.json可能很大(例如，兆字节)，是否可以在不加载文件的情况下附加新的数据框结果?

The existing file output.json can be huge (say Tetabytes), is it possible to append the new dataframe result without loading the file?

• http://pandas.pydata.org/pandas-docs/dev/generated/pandas.DataFrame.to_json.html
• http://pandas.pydata.org/pandas-docs/dev/generated/pandas.DataFrame.to_csv.html

• http://pandas.pydata.org/pandas-docs/dev/generated/pandas.DataFrame.to_json.html
• http://pandas.pydata.org/pandas-docs/dev/generated/pandas.DataFrame.to_csv.html

推荐答案

不，如果不使用pandas或json模块重写整个文件，则无法追加到json文件.您可以通过以a模式打开文件并寻求正确的位置并插入数据来手动"修改文件.我不会推荐这个.如果文件要大于RAM，最好只使用json以外的文件格式.

No, you can't append to a json file without re-writing the whole file using pandas or the json module. You might be able to modify the file "manually" by opening the file in a mode and seeking to the correct position and inserting your data. I wouldn't recommend this though. Better to just use a file format other than json if your file is going to be larger than your RAM.

此 answer 可能有帮助.它不会创建有效的json文件(而是每行都是一个json字符串)，但是其目标与您的目标非常相似.

This answer also might help. It doesn't create valid json files (instead each line is a json string), but its goal is very similar to yours.

这篇关于如何在追加模式下将DataFrame导出为to_json-Python Pandas?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！