如何在Python中创建示例Spark dataFrame? [英] How to create a sample Spark dataFrame in Python?

问题描述

我想创建一个示例DataFrame，但是以下代码不起作用:

I want to create a sample DataFrame but the following code is not working:

df = spark.createDataFrame(["10","11","13"], ("age"))

## ValueError
## ...
## ValueError: Could not parse datatype: age

预期结果是:

age
10
11
13

推荐答案

以下代码不起作用

the following code is not working

对于单个元素，您需要一个模式作为类型

With single element you need a schema as type

spark.createDataFrame(["10","11","13"], "string").toDF("age")

或DataType:

from pyspark.sql.types import StringType

spark.createDataFrame(["10","11","13"], StringType()).toDF("age")

具有名称元素的元素应为元组，并且模式应为顺序:

With name elements should be tuples and schema as sequence:

spark.createDataFrame([("10", ), ("11", ), ("13",  )], ["age"])

这篇关于如何在Python中创建示例Spark dataFrame?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！