保存 pandas 数据框以分隔不带NaN的json [英] Saving a pandas dataframe to separate jsons without NaNs
问题描述
我有一个带有某些NaN值的数据框.
I have a dataframe with some NaN values.
这是一个示例数据框:
sample_df = pd.DataFrame([[1,np.nan,1],[2,2,np.nan], [np.nan, 3, 3], [4,4,4],[np.nan,np.nan,5], [6,np.nan,np.nan]])
它看起来像:
获取json之后我做了什么:
What I did after to get a json:
sample_df.to_json(orient = 'records')
哪个给:
'[{"0":1.0,"1":null,"2":1.0},{"0":2.0,"1":2.0,"2":null},{"0":null,"1":3.0,"2":3.0},{"0":4.0,"1":4.0,"2":4.0},{"0":null,"1":null,"2":5.0},{"0":6.0,"1":null,"2":null}]'
我想将此数据帧保存到一个json中,每个json中有2行,但没有Nan值.这是我尝试执行的操作:
I want to save this dataframe to a json with 2 rows in each json, but with none of the Nan values. Here is how I tried to do it:
df_dict = dict((n, sample_df.iloc[n:n+2, :]) for n in range(0, len(sample_df), 2))
for k, v in df_dict.items():
print(k)
print(v)
for d in (v.to_dict('record')):
for k,v in list(d.items()):
if type(v)==float:
if math.isnan(v):
del d[k]
json.dumps(df_dict)
我想要的输出:
'[{"0":1.0,"2":1.0},{"0":2.0,"1":2.0}]'->在一个.json文件中 '.[{"1":3.0,"2":3.0},{"0":4.0,"1":4.0,"2":4.0}]'->在第二个.json文件中 '.[{"2":5.0},{"0":6.0}]'->在第三个.json文件中
'[{"0":1.0,"2":1.0},{"0":2.0,"1":2.0}]' -> in one .json file '[{"1":3.0,"2":3.0},{"0":4.0,"1":4.0,"2":4.0}]' -> in second .json file '[{"2":5.0},{"0":6.0}]' -> in third .json file
推荐答案
使用apply
删除NaN
s,groupby
进行分组,并dfGroupBy.apply
进行JSONify.
Use apply
to drop NaN
s, groupby
to group and dfGroupBy.apply
to JSONify.
s = sample_df.apply(lambda x: x.dropna().to_dict(), 1)\
.groupby(sample_df.index // 2)\
.apply(lambda x: x.to_json(orient='records'))
s
0 [{"0":1.0,"2":1.0},{"0":2.0,"1":2.0}]
1 [{"1":3.0,"2":3.0},{"0":4.0,"1":4.0,"2":4.0}]
2 [{"2":5.0},{"0":6.0}]
dtype: object
最后,遍历.values
并保存到单独的JSON文件中.
Finally, iterate over .values
and save to separate JSON files.
import json
for i, j_data in enumerate(s.values):
json.dump(j_data, open('File{}.json'.format(i + 1), 'w'))
这篇关于保存 pandas 数据框以分隔不带NaN的json的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!