从另一个RDD中减去一个RDD无法正常工作 [英] Subtract an RDD from another RDD doesn't work correctly

问题描述

我想从另一个RDD中减去一个RDD.我查看了文档，发现减可以做到这一点.实际上，当我测试减时，最终的RDD保持不变，并且值未删除！

I want to subtract an RDD from another RDD. I looked into the documentation and I found that subtract can do that. Actually, when I tested subtract, the final RDD remains the same and the values are not removed!

还有其他功能可以做到吗?还是我错误地使用了减?

Is there any other function to do that? Or am I using subtract incorrectly?

这是我使用的代码:

 val vertexRDD: org.apache.spark.rdd.RDD[(VertexId, Array[Int])]
 val clusters  = vertexRDD.takeSample(false, 3)
 val clustersRDD: RDD[(VertexId, Array[Int])] = sc.parallelize(clusters)
 val final = vertexRDD.subtract(clustersRDD)
 final.collect().foreach(println(_))

推荐答案

通常不支持或至少不建议执行诸如对可变类型的减法(在此示例中为Array)之类的集合操作.

Performing set operations like subtract with mutable types (Array in this example) is usually unsupported, or at least not recommended.

尝试使用不可变类型.

我相信 WrappedArray 是用于在集合中存储数组的相关容器，但我不确定.

I believe WrappedArray is the relevant container for storing arrays in sets, but i'm not sure.

这篇关于从另一个RDD中减去一个RDD无法正常工作的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！