如何在 sparkR 中绑定两个数据框列? [英] How to do bind two dataframe columns in sparkR?

问题描述

如何在spark 1.4的SparkR中绑定两列dataframe

How to bind two columns of dataframe in SparkR of spark 1.4

TIA，阿伦

推荐答案

没有办法做到这一点.这是一个关于 Scala 中 spark (1.3) 的问题.能够做到这一点的唯一方法是使用某种 row.numbering，因为这样您就可以在 row.number 上加入.为什么?因为你只能基于其他已经存在的列连接表或添加列

There is no way to do this. Here is a question on spark (1.3) in scala. The only way to be able to do this, is having some kind of row.numbering, because then you are able to join on row.number. Why? Because you can only join tables or add columns based on other already existing columns

data1 <- createDataFrame(sqlContext, data.frame(a=c(1,2,3)))
data2 <- createDataFrame(sqlContext, data.frame(b=c(2,3,4)))

然后

withColumn(data1,"b",data1$a + 1)

是允许的，但是

withColumn(data1,"b",data2$b)

不是.从 Spark 将您的 DataFrame 切成块来存储它的那一刻起，它就不知道如何绑定它们(它不知道行排序)，只有当您有 row.numbers 时才知道.

is not. From the moment Spark cuts your DataFrame in blocks to store it, it has no idea how to bind them (it has no idea of a row sequencing), only when you have row.numbers.

这篇关于如何在 sparkR 中绑定两个数据框列?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！