如何在 Python 中从 NASA Weather Insight API 展平嵌套的 JSON [英] How to flatten a nested JSON from the NASA Weather Insight API in Python
本文介绍了如何在 Python 中从 NASA Weather Insight API 展平嵌套的 JSON的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
您好，我正在尝试通过 Mars Insight API 显示火星天气.发生的问题是数据以 JSON 格式返回并且具有三个级别 API 文档.我可以用 3-4 种不同的方法拉主键没有问题，但是当我尝试获取二级或三级键时，它就会崩溃.
导入请求导入json将熊猫导入为 pd从 pandas.io.json 导入 json_normalizeAPI_url = "https://api.nasa.gov/insight_weather/?api_key=nTal99zKlhGbl0N8F0V9iUofifMdcwyOHw64CrVm&feedtype=json&ver=1.0"API_data = requests.get(API_url).json()# 定义天气数据属性#AT = {'AT':API_data['sol_keys'[1,2,3]]}#PRE = {'PRE':API_data['sol_keys']}#HWS = {'HWS':API_data['sol_keys']}#Season= {'Season':API_data['sol_keys']}#WD = {'WD':API_data['sol_keys']}#most_common = {'most_common':API_data['sol_keys']}context = {'sol_keys': API_data["sol_keys"]}数据 =json_normalize(API_data, 'sol_keys', '301','AT')打印(数据) 解决方案
使用递归扁平化嵌套的 dicts
- 在 Python 中递归思考
- 在 Python 中展平 JSON 对象
- 扁平化
flatten_json函数，将用于扁平化data
def flatten_json(nested_json: dict, exclude: list=['']) ->字典:"""展平嵌套字典列表."""输出 = dict()def flatten(x: (list, dict, str), name: str='', exclude=exclude):如果 type(x) 是字典:对于 x 中的 a:如果不在排除中:flatten(x[a], f'{name}{a}_')elif type(x) 是列表:我 = 0对于 x 中的 a:flatten(a, f'{name}{i}_')我 += 1别的:出[名称[:-1]] = x展平(nested_json)回来将pandas导入为pd从 pandas.io.json 导入 json_normalize进口请求API_url = "https://api.nasa.gov/insight_weather/?api_key=nTal99zKlhGbl0N8F0V9iUofifMdcwyOHw64CrVm&feedtype=json&ver=1.0"API_data = requests.get(API_url).json()# 创建一个字典列表:这些是每个 sol_key 的值data = [API_data[x] for x in API_data['sol_keys']]# 如果您还希望将 sol_key 包含在数据中# 它需要作为键:值对重新添加对于 i, enumerate(data, 301) 中的值:value.update({'sol_key': i})# 展开所有值df = pd.DataFrame([flatten_json(x) for x in data])输出
<预类= 郎吡prettyprint-越权"> <代码> sol_key AT_av AT_ct AT_mn AT_mx First_UTC HWS_av HWS_ct HWS_mn HWS_mx Last_UTC PRE_av PRE_ct PRE_mn PRE_mx季WD_1_compass_degrees WD_1_compass_point WD_1_compass_right WD_1_compass_up WD_1_ct WD_10_compass_degrees WD_10_compass_point WD_10_compass_right WD_10_compass_up WD_10_ct WD_11_compass_degrees WD_11_compass_point WD_11_compass_right WD_11_compass_up WD_11_ct WD_12_compass_degrees WD_12_compass_point WD_12_compass_rightWD_12_compass_up WD_12_ct WD_13_compass_degrees WD_13_compass_point WD_13_compass_right WD_13_compass_up WD_13_ct WD_2_compass_degrees WD_2_compass_point WD_2_compass_right WD_2_compass_up WD_2_ct WD_3_compass_degrees WD_3_compass_point WD_3_compass_right WD_3_compass_up WD_3_ct WD_5_compass_degrees WD_5_compass_point WD_5_compass_right WD_5_compass_up WD_5_ct WD_6_compass_degrees WD_6_compass_point WD_6_compass_right WD_6_compass_up WD_6_ct WD_7_compass_degrees WD_7_compass_point WD_7_compass_right WD_7_compass_up WD_7_ct WD_8_compass_degrees WD_8_compass_point WD_8_compass_right WD_8_compass_up WD_8_ct WD_9_compass_degrees WD_9_compass_point WD_9_compass_right WD_9_compass_up WD_9_ct WD_most_common_compass_degrees WD_most_common_compass_point WD_most_common_compass_right WD_most_common_compass_up WD_most_common_ct WD_14_compass_degrees WD_14_compass_point WD_14_compass_right WD_14_compass_up WD_14_ct WD_0_compass_degrees WD_0_compass_point WD_0_compass_right WD_0_compass_up WD_0_ct301 -69.684 342720 -103.886 -26.371 2019-10-01T11:46:39Z 4.630 158626 0.129 17.919 2019-10-02T12:26:13Z 727.941 153492 711.7187 743.1005弹簧22.5 NNE 0.382683 0.92388 4.0 225.0 SW -0.707107 -0.707107 26723 247.5 WSW -0.92388 -0.382683 15528 270.0W¯¯-1.0 -0.0 3136 292.5西北偏西-0.92388 0.382683 2.0 45.0 NE 0.707107 0.707107 6.0 67.5 ENE 0.92388 0.382683 688 112.5 ESE 0.92388 -0.382683 3387 135.0 SE 0.707107 -0.707107 40327 157.5 SSE 0.382683 -0.92388 31608 180.0 S 0.0 -1.0 8520 202.5 SSW -0.382683 -0.92388 28697 135.0 SE 0.707107 N NaN Na2N Na2N Na07N N Na07N N Na0 8520 202.5 SSW302 -68.977 339696 -102.032 -25.338 2019-10-02T12:26:14Z 4.781 154660 0.208 20.153 2019-10-03T13:05:49Z 727.076 168657 710.8055 741.8326弹簧22.5 NNE 0.382683 0.92388 1.0 225.0 SW -0.707107 -0.707107 32482 247.5 WSW -0.92388 -0.382683 1508 270.0W¯¯-1.0 -0.0 27的NaN楠楠楠楠45.0 NE 0.707107 0.707107 16.0 67.5 ENE 0.92388 0.382683 1757 112.5 ESE 0.92388 -0.382683 2178 135.0 SE 0.707107 -0.707107 25516 157.5 SSE 0.382683 -0.92388 36367 180.0 S 0.0 -1.0 26800 202.5 SSW -0.382683 -0.92388 28008 157.5 SSE 0.382683 NaN Na3 N Na6N Na6N Na6N 202.5 SSW303 -67.094 257650 -103.946 -26.523 2019-10-03T13:05:50Z 4.911 113599 0.131 19.147 2019-10-04T13:45:24Z 724.189 110794 711.2929 741.7360弹簧22.5 NNE 0.382683 0.92388 6.0 225.0 SW -0.707107 -0.707107 16663 247.5 WSW -0.92388 -0.382683 5999 270.0W¯¯-1.0 -0.0 8920 292.5西北偏西-0.92388 0.382683 23.0 45.0 NE 0.707107 0.707107 12.0 67.5 ENE 0.92388 0.382683 507 112.5 ESE 0.92388 -0.382683 1041 135.0 SE 0.707107 -0.707107 21889 157.5 SSE 0.382683 -0.92388 29209 180.0 S 0.0 -1.0 9400 202.5 SSW -0.382683 -0.92388 19919 157.5 SSE 0.382683 -0.92N 7 N 0.382683 -0.92N 7 N 7 N 7 N 0.92 N 7304 -68.042 308602 -104.325 -25.869 2019-10-04T13:45:25Z 4.959 140757 0.132 18.224 2019-10-05T14:25:00Z 724.808 152271 707.9475 741.3935弹簧22.5 NNE 0.382683 0.92388 6.0 225.0 SW -0.707107 -0.707107 18480 247.5 WSW -0.92388 -0.382683 9226 270.0W¯¯-1.0 -0.0 16455 292.5西北偏西-0.92388 0.382683 12.0 45.0 NE 0.707107 0.707107 2.0 67.5 ENE 0.92388 0.382683 1006 112.5 ESE 0.92388 -0.382683 1622 135.0 SE 0.707107 -0.707107 27717 157.5 SSE 0.382683 -0.92388 36692 180.0 S 0.0 -1.0 13210 202.5 SSW -0.382683 -0.92388 16329 157.5 SSE 0.382683 Na N Na6N Na6N Na6N Na6N 202.5 SSW305 -71.205 229742 -104.059 -27.287 2019-10-05T14:25:01Z 4.874 103937 0.128 22.241 2019-10-06T15:04:35Z 722.192 157557 708.6817 738.4189弹簧的NaN楠楠楠楠225.0 SW -0.707107 -0.707107 15124 247.5 WSW -0.92388 -0.382683 4252 270.0W¯¯-1.0 -0.0 3027 292.5西北偏西-0.92388 0.382683 11.0楠楠楠楠的NaN 67.5 ENE 0.92388 0.382683 71 112.5 ESE 0.92388 -0.382683 712 135.0 SE 0.707107 -0.707107 15842 157.5 SSE 0.382683 -0.92388 34545 180.0 S 0.0 -1.0 13445 202.5 SSW -0.382683 -0.92388 16908 157.5 SSE 0.382683 N Na4N Na4N Na4N Na4N Na85N 202.5 SSW306 -72.664 215500 -102.655 -25.681 2019-10-06T15:04:36Z 4.437 101771 0.131 17.113 2019-10-07T15:44:09Z 720.791 125256 706.1014 740.7565弹簧22.5 NNE 0.382683 0.92388 1.0 225.0 SW -0.707107 -0.707107 16025 247.5 WSW -0.92388 -0.382683 2200 270.0W¯¯-1.0 -0.0 6820 292.5西北偏西-0.92388 0.382683 63.0 45.0 NE 0.707107 0.707107 3.0 67.5 ENE 0.92388 0.382683 265 112.5 ESE 0.92388 -0.382683 747 135.0 SE 0.707107 -0.707107 15702 157.5 SSE 0.382683 -0.92388 20971 180.0 S 0.0 -1.0 18328 202.5 SSW -0.382683 -0.92388 20646 157.5 SSE 0.382683 N Na0N Na0N N Na09N Na09N N 202.5 SSW307 -71.995 175881 -102.027 -26.828 2019-10-07T15:44:10Z 4.948 82571 0.206 18.374 2019-10-08T10:12:49Z 724.898 87860 704.6372 739.6598弹簧22.5 NNE 0.382683 0.92388 7.0 225.0 SW -0.707107 -0.707107 13459 247.5 WSW -0.92388 -0.382683 9642 270.0W¯¯-1.0 -0.0 6382的NaN楠楠楠楠45.0 NE 0.707107 0.707107 3.0 67.5 ENE 0.92388 0.382683 171 112.5 ESE 0.92388 -0.382683 655 135.0 SE 0.707107 -0.707107 12847 157.5 SSE 0.382683 -0.92388 19655 180.0 S 0.0 -1.0 12628 202.5 SSW -0.382683 -0.92388 7121 157.5 SSE 0.382683 N0.05N 0.92 N 0.5N 0.92 N 0.92Hello I am trying to display the mars weather from the Mars Insight API. The issue that is occurring is that the data is returning in JSON format and has three levels API Documentation. I can pull the primary key no problem with 3-4 different methods but when I try to get a secondary or tertiary key that is when it falls apart.
import requests
import json
import pandas as pd
from pandas.io.json import json_normalize
API_url = "https://api.nasa.gov/insight_weather/?api_key=nTal99zKlhGbl0N8F0V9iUofifMdcwyOHw64CrVm&feedtype=json&ver=1.0"
API_data = requests.get(API_url).json()
# define weather data attributes
#AT = {'AT':API_data['sol_keys'[1,2,3]]}
#PRE = {'PRE':API_data['sol_keys']}
#HWS = {'HWS':API_data['sol_keys']}
#Season= {'Season':API_data['sol_keys']}
#WD = {'WD':API_data['sol_keys']}
#most_common = {'most_common':API_data['sol_keys']}
context = {'sol_keys': API_data["sol_keys"]}
data =json_normalize(API_data, 'sol_keys', '301','AT')
print (data)
解决方案
Use recursion to flatten the nested dicts
- Thinking Recursively in Python
- Flattening JSON objects in Python
- flatten
- The
flatten_jsonfunction, will be used to flattendata
def flatten_json(nested_json: dict, exclude: list=['']) -> dict:
"""
Flatten a list of nested dicts.
"""
out = dict()
def flatten(x: (list, dict, str), name: str='', exclude=exclude):
if type(x) is dict:
for a in x:
if a not in exclude:
flatten(x[a], f'{name}{a}_')
elif type(x) is list:
i = 0
for a in x:
flatten(a, f'{name}{i}_')
i += 1
else:
out[name[:-1]] = x
flatten(nested_json)
return out
import pandas as pd
from pandas.io.json import json_normalize
import requests
API_url = "https://api.nasa.gov/insight_weather/?api_key=nTal99zKlhGbl0N8F0V9iUofifMdcwyOHw64CrVm&feedtype=json&ver=1.0"
API_data = requests.get(API_url).json()
# create a list of dicts: these are the values of each sol_key
data = [API_data[x] for x in API_data['sol_keys']]
# if you also want the sol_key to be included in the data
# it needs to be added back in as a key: value pair
for i, value in enumerate(data, 301):
value.update({'sol_key': i})
# expand all the values
df = pd.DataFrame([flatten_json(x) for x in data])
Output
sol_key AT_av AT_ct AT_mn AT_mx First_UTC HWS_av HWS_ct HWS_mn HWS_mx Last_UTC PRE_av PRE_ct PRE_mn PRE_mx Season WD_1_compass_degrees WD_1_compass_point WD_1_compass_right WD_1_compass_up WD_1_ct WD_10_compass_degrees WD_10_compass_point WD_10_compass_right WD_10_compass_up WD_10_ct WD_11_compass_degrees WD_11_compass_point WD_11_compass_right WD_11_compass_up WD_11_ct WD_12_compass_degrees WD_12_compass_point WD_12_compass_right WD_12_compass_up WD_12_ct WD_13_compass_degrees WD_13_compass_point WD_13_compass_right WD_13_compass_up WD_13_ct WD_2_compass_degrees WD_2_compass_point WD_2_compass_right WD_2_compass_up WD_2_ct WD_3_compass_degrees WD_3_compass_point WD_3_compass_right WD_3_compass_up WD_3_ct WD_5_compass_degrees WD_5_compass_point WD_5_compass_right WD_5_compass_up WD_5_ct WD_6_compass_degrees WD_6_compass_point WD_6_compass_right WD_6_compass_up WD_6_ct WD_7_compass_degrees WD_7_compass_point WD_7_compass_right WD_7_compass_up WD_7_ct WD_8_compass_degrees WD_8_compass_point WD_8_compass_right WD_8_compass_up WD_8_ct WD_9_compass_degrees WD_9_compass_point WD_9_compass_right WD_9_compass_up WD_9_ct WD_most_common_compass_degrees WD_most_common_compass_point WD_most_common_compass_right WD_most_common_compass_up WD_most_common_ct WD_14_compass_degrees WD_14_compass_point WD_14_compass_right WD_14_compass_up WD_14_ct WD_0_compass_degrees WD_0_compass_point WD_0_compass_right WD_0_compass_up WD_0_ct
301 -69.684 342720 -103.886 -26.371 2019-10-01T11:46:39Z 4.630 158626 0.129 17.919 2019-10-02T12:26:13Z 727.941 153492 711.7187 743.1005 spring 22.5 NNE 0.382683 0.92388 4.0 225.0 SW -0.707107 -0.707107 26723 247.5 WSW -0.92388 -0.382683 15528 270.0 W -1.0 -0.0 3136 292.5 WNW -0.92388 0.382683 2.0 45.0 NE 0.707107 0.707107 6.0 67.5 ENE 0.92388 0.382683 688 112.5 ESE 0.92388 -0.382683 3387 135.0 SE 0.707107 -0.707107 40327 157.5 SSE 0.382683 -0.92388 31608 180.0 S 0.0 -1.0 8520 202.5 SSW -0.382683 -0.92388 28697 135.0 SE 0.707107 -0.707107 40327 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
302 -68.977 339696 -102.032 -25.338 2019-10-02T12:26:14Z 4.781 154660 0.208 20.153 2019-10-03T13:05:49Z 727.076 168657 710.8055 741.8326 spring 22.5 NNE 0.382683 0.92388 1.0 225.0 SW -0.707107 -0.707107 32482 247.5 WSW -0.92388 -0.382683 1508 270.0 W -1.0 -0.0 27 NaN NaN NaN NaN NaN 45.0 NE 0.707107 0.707107 16.0 67.5 ENE 0.92388 0.382683 1757 112.5 ESE 0.92388 -0.382683 2178 135.0 SE 0.707107 -0.707107 25516 157.5 SSE 0.382683 -0.92388 36367 180.0 S 0.0 -1.0 26800 202.5 SSW -0.382683 -0.92388 28008 157.5 SSE 0.382683 -0.923880 36367 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
303 -67.094 257650 -103.946 -26.523 2019-10-03T13:05:50Z 4.911 113599 0.131 19.147 2019-10-04T13:45:24Z 724.189 110794 711.2929 741.7360 spring 22.5 NNE 0.382683 0.92388 6.0 225.0 SW -0.707107 -0.707107 16663 247.5 WSW -0.92388 -0.382683 5999 270.0 W -1.0 -0.0 8920 292.5 WNW -0.92388 0.382683 23.0 45.0 NE 0.707107 0.707107 12.0 67.5 ENE 0.92388 0.382683 507 112.5 ESE 0.92388 -0.382683 1041 135.0 SE 0.707107 -0.707107 21889 157.5 SSE 0.382683 -0.92388 29209 180.0 S 0.0 -1.0 9400 202.5 SSW -0.382683 -0.92388 19919 157.5 SSE 0.382683 -0.923880 29209 315.0 NW -0.707107 0.707107 11.0 NaN NaN NaN NaN NaN
304 -68.042 308602 -104.325 -25.869 2019-10-04T13:45:25Z 4.959 140757 0.132 18.224 2019-10-05T14:25:00Z 724.808 152271 707.9475 741.3935 spring 22.5 NNE 0.382683 0.92388 6.0 225.0 SW -0.707107 -0.707107 18480 247.5 WSW -0.92388 -0.382683 9226 270.0 W -1.0 -0.0 16455 292.5 WNW -0.92388 0.382683 12.0 45.0 NE 0.707107 0.707107 2.0 67.5 ENE 0.92388 0.382683 1006 112.5 ESE 0.92388 -0.382683 1622 135.0 SE 0.707107 -0.707107 27717 157.5 SSE 0.382683 -0.92388 36692 180.0 S 0.0 -1.0 13210 202.5 SSW -0.382683 -0.92388 16329 157.5 SSE 0.382683 -0.923880 36692 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
305 -71.205 229742 -104.059 -27.287 2019-10-05T14:25:01Z 4.874 103937 0.128 22.241 2019-10-06T15:04:35Z 722.192 157557 708.6817 738.4189 spring NaN NaN NaN NaN NaN 225.0 SW -0.707107 -0.707107 15124 247.5 WSW -0.92388 -0.382683 4252 270.0 W -1.0 -0.0 3027 292.5 WNW -0.92388 0.382683 11.0 NaN NaN NaN NaN NaN 67.5 ENE 0.92388 0.382683 71 112.5 ESE 0.92388 -0.382683 712 135.0 SE 0.707107 -0.707107 15842 157.5 SSE 0.382683 -0.92388 34545 180.0 S 0.0 -1.0 13445 202.5 SSW -0.382683 -0.92388 16908 157.5 SSE 0.382683 -0.923880 34545 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
306 -72.664 215500 -102.655 -25.681 2019-10-06T15:04:36Z 4.437 101771 0.131 17.113 2019-10-07T15:44:09Z 720.791 125256 706.1014 740.7565 spring 22.5 NNE 0.382683 0.92388 1.0 225.0 SW -0.707107 -0.707107 16025 247.5 WSW -0.92388 -0.382683 2200 270.0 W -1.0 -0.0 6820 292.5 WNW -0.92388 0.382683 63.0 45.0 NE 0.707107 0.707107 3.0 67.5 ENE 0.92388 0.382683 265 112.5 ESE 0.92388 -0.382683 747 135.0 SE 0.707107 -0.707107 15702 157.5 SSE 0.382683 -0.92388 20971 180.0 S 0.0 -1.0 18328 202.5 SSW -0.382683 -0.92388 20646 157.5 SSE 0.382683 -0.923880 20971 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
307 -71.995 175881 -102.027 -26.828 2019-10-07T15:44:10Z 4.948 82571 0.206 18.374 2019-10-08T10:12:49Z 724.898 87860 704.6372 739.6598 spring 22.5 NNE 0.382683 0.92388 7.0 225.0 SW -0.707107 -0.707107 13459 247.5 WSW -0.92388 -0.382683 9642 270.0 W -1.0 -0.0 6382 NaN NaN NaN NaN NaN 45.0 NE 0.707107 0.707107 3.0 67.5 ENE 0.92388 0.382683 171 112.5 ESE 0.92388 -0.382683 655 135.0 SE 0.707107 -0.707107 12847 157.5 SSE 0.382683 -0.92388 19655 180.0 S 0.0 -1.0 12628 202.5 SSW -0.382683 -0.92388 7121 157.5 SSE 0.382683 -0.923880 19655 NaN NaN NaN NaN NaN 0.0 N 0.0 1.0 1.0
这篇关于如何在 Python 中从 NASA Weather Insight API 展平嵌套的 JSON的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
