为什么我无法使用Spark中的map函数更改节点的属性? [英] why i cant change the property of nodes using map function in Spark?

问题描述

我正在Spark中使用GraphX来处理图形.我有一个 val common_neighbors:RDD [VertexId] ，其中包含一些vertexId.我使用地图功能将其转换为诸如(node，1)的结构，该节点是顶点的ID，而1是其初始属性.转换代码如下:

i am working with GraphX in Spark to process a graph. i have a val common_neighbors: RDD[VertexId] that holds some vertexId. i use map function to transform it into a structure such as (node,1), which node is the ID of the vertex and 1 is its initial property. the code for transforming is below:

val p =common_neighbors.map(x=>(x,1))

我有一个具有如下结构的图:(node，node_property(label，isDefined)).例如(1，(14，true)).这意味着ID = 1的节点具有label = 14和isDefined.

i have a graph that has a structure such as: (node,node_property(label,isDefined)). for example (1,(14,true)). this means node with ID=1 has label=14 and isDefined.

如果图形中节点的标签高于5，我想以并行和分布式的方式转换 p 中的nodes属性.代码如下:

i want to transform the nodes property in p in a parallel and distributed way if the node's label in the graph is higher than 5. the code is below:

val x=p.map(node=>{


        val temp_property=graph.vertices.filter(x=>x._1==node._1).values.take(1)
        if(temp_property(0).label > 5) {

          (node._1,((temp_property(0).label)+5))
        }
        })

但是当我执行代码时，出现错误.这是什么问题?我该如何解决?

But when i execute the code, i get Errors. what is the problem with this?how can i fix this?

推荐答案

据我了解您的问题，您在另一个RDD中使用了RDD，但事实并非如此.除此之外，您可以执行以下操作:您可以使用以下代码将common_neighbors与真实图形连接起来:

as i understood your problem, you are using a RDD inside another RDD, that its not true. instead of that you can do this: you can join your common_neighbors with the real graph by using below code:

val new_val=common_neighbors.join(work_graph.vertices)

然后通过一些映射转换，可以在new_val中构建图形的结构，然后可以使用map或mapValues对new_val的值进行所有操作

then through some map transformation you can make the structure of your graph in the new_val and then you can use map or mapValues to do every operation on values of the new_val

这篇关于为什么我无法使用Spark中的map函数更改节点的属性?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！