使用Java在Spark DataFrame中两列的区别 [英] difference of two columns in spark dataframe using java

问题描述

我尝试了以下文章中提到的选项，但在Java中不起作用-减法Spark数据框中有两列为空的

请让我知道如何使用Java中的Spark查找数据集的两列之间的差异.

解决方案

您可以采用另一种方法.让我们假设，您想要具有区别的其他列:

  Dataset< Row>ds =//您的数据集ds.withColumn("diff_col"，new Column("col1").$ minus(new Column("col2"))); 

I tried the options mentioned in the following post but it didn't work in java - finding the difference of two columns in spark dataframes and appending to a new column

subtract two columns with null in spark dataframe

Please let me know how can I find difference between two column of a dataset using Spark in Java.

解决方案

You can do it next way. Lets assume, you want additional column with difference:

Dataset<Row> ds = // your dataset
ds.withColumn("diff_col", new Column("col1").$minus(new Column("col2"));

这篇关于使用Java在Spark DataFrame中两列的区别的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！