将pyspark数据框转换为python词典列表 [英] Convert pyspark dataframe into list of python dictionaries

问题描述

我是pyspark的新手，正在尝试将pyspark.sql.dataframe转换为词典列表.

Hi I'm new to pyspark and I'm trying to convert pyspark.sql.dataframe into list of dictionaries.

下面是我的数据框，类型是< class'pyspark.sql.dataframe.DataFrame'>:

Below is my dataframe, the type is <class 'pyspark.sql.dataframe.DataFrame'>:

+------------------+----------+------------------------+
|             title|imdb_score|Worldwide_Gross(dollars)|
+------------------+----------+------------------------+
| The Eight Hundred|       7.2|               460699653|
| Bad Boys for Life|       6.6|               426505244|
|             Tenet|       7.8|               334000000|
|Sonic the Hedgehog|       6.5|               308439401|
|          Dolittle|       5.6|               245229088|
+------------------+----------+------------------------+

我想将其转换为:

[{"title":"The Eight Hundred", "imdb_score":7.2, "Worldwide_Gross(dollars)":460699653},
 {"title":"Bad Boys for Life", "imdb_score":6.6, "Worldwide_Gross(dollars)":426505244},
 {"title":"Tenet", "imdb_score":7.8, "Worldwide_Gross(dollars)":334000000},
 {"title":"Sonic the Hedgehog", "imdb_score":6.5, "Worldwide_Gross(dollars)":308439401},
 {"title":"Dolittle", "imdb_score":5.6, "Worldwide_Gross(dollars)":245229088}]

我应该怎么做?预先感谢！

How should I do this? Thanks in advance!

推荐答案

您可以将每一行映射到字典中并收集结果:

You can map each row into a dictionary and collect the results:

df.rdd.map(lambda row:row.asDict()).collect()

这篇关于将pyspark数据框转换为python词典列表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！