AttributeError: 'DataFrame' 对象没有属性 'map' [英] AttributeError: 'DataFrame' object has no attribute 'map'
本文介绍了AttributeError: 'DataFrame' 对象没有属性 'map'的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我想使用以下代码转换要添加的 spark 数据框:
I wanted to convert the spark data frame to add using the code below:
from pyspark.mllib.clustering import KMeans
spark_df = sqlContext.createDataFrame(pandas_df)
rdd = spark_df.map(lambda data: Vectors.dense([float(c) for c in data]))
model = KMeans.train(rdd, 2, maxIterations=10, runs=30, initializationMode="random")
详细的错误信息是:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-11-a19a1763d3ac> in <module>()
1 from pyspark.mllib.clustering import KMeans
2 spark_df = sqlContext.createDataFrame(pandas_df)
----> 3 rdd = spark_df.map(lambda data: Vectors.dense([float(c) for c in data]))
4 model = KMeans.train(rdd, 2, maxIterations=10, runs=30, initializationMode="random")
/home/edamame/spark/spark-2.0.0-bin-hadoop2.6/python/pyspark/sql/dataframe.pyc in __getattr__(self, name)
842 if name not in self.columns:
843 raise AttributeError(
--> 844 "'%s' object has no attribute '%s'" % (self.__class__.__name__, name))
845 jc = self._jdf.apply(name)
846 return Column(jc)
AttributeError: 'DataFrame' object has no attribute 'map'
有谁知道我在这里做错了什么?谢谢!
Does anyone know what I did wrong here? Thanks!
推荐答案
你不能map
一个数据帧,但是你可以通过执行spark_df将数据帧转换成一个RDD并映射它.rdd.map()
.在 Spark 2.0 之前,spark_df.map
将别名为 spark_df.rdd.map()
.对于 Spark 2.0,您必须首先显式调用 .rdd
.
You can't map
a dataframe, but you can convert the dataframe to an RDD and map that by doing spark_df.rdd.map()
. Prior to Spark 2.0, spark_df.map
would alias to spark_df.rdd.map()
. With Spark 2.0, you must explicitly call .rdd
first.
这篇关于AttributeError: 'DataFrame' 对象没有属性 'map'的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文