Apache Giraph - 由于一次只有1个任务，因此无法在拆分主/从模式下运行 [英] Apache Giraph - Cannot run in split master / worker mode since there is only 1 task at a time

问题描述

我使用PageRank Benchmark示例以hadoop 2.2.0运行Giraph 1.0.0

突然间，我得到了这个错误结果：

线程中的异常mainjava.lang.IllegalArgumentException：
checkLocalJobRunnerConfiguration：使用LocalJobRunner时，由于每次只有一项任务，因此只能有
一个工人！
org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration（GiraphJob.java:151）
at org.apache.giraph.job.GiraphJob.run（GiraphJob.java:225）at
org.apache.giraph.benchmark.GiraphBenchmark.run（GiraphBenchmark.java:90）
at org.apache.hadoop.util.ToolRunner.run（ToolRunner.java:70）at
org.apache .hadoop.util.ToolRunner.run（ToolRunner.java:84）at
org.apache.giraph.benchmark.PageRankBenchmark.main（PageRankBenchmark.java:71）
at sun.reflect.NativeMethodAccessorImpl.invoke0 （Native Method）

sun.reflect.NativeMethodAccessorImpl.invoke（NativeMethodAccessorImpl.java:57）在
的java.lang.reflect.Method.invoke（Method.java:606）org.apache.hadoop.util.RunJar.main（RunJar.java:212）

blockquote>

当我将工作人员数量改为1时，我得到了：

lockquote

Ex线程main中的ception java.lang.IllegalArgumentException：
checkLocalJobRunnerConfiguration：使用LocalJobRunner时，
无法以分离的主/从模式运行，因为每次
只有1个任务！
org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration（GiraphJob.java:157）
at org.apache.giraph.job.GiraphJob.run（GiraphJob.java:225）at
org.apache.giraph.benchmark.GiraphBenchmark.run（GiraphBenchmark.java:90）
at org.apache.hadoop.util.ToolRunner.run（ToolRunner.java:70）at
org.apache .hadoop.util.ToolRunner.run（ToolRunner.java:84）at
org.apache.giraph.benchmark.PageRankBenchmark.main（PageRankBenchmark.java:71）
at sun.reflect.NativeMethodAccessorImpl.invoke0 （Native Method）

sun.reflect.NativeMethodAccessorImpl.invoke（NativeMethodAccessorImpl.java:57）在
的java.lang.reflect.Method.invoke（Method.java:606）org.apache.hadoop.util.RunJar.main（RunJar.java:212）

blockquote>

任何解决方案？

解决方案

我认为你不是在群集上运行？

您可以在giraph-site.xml中禁用拆分master worker行为。

如果我在演示虚拟机中运行，

  giraph.SplitMasterWorker = false 
  

如果你只是想在一次性exeuction期间禁用它，你也可以将它作为命令行参数传递给你的程序。

c> -ca giraph.SplitMasterWorker = false

例如，我为大数据演讲这：

 ＃！/ bin / bash 
 
 yarn jar /root/giraph-0.0.1 -SNAPSHOT-jar -with-dependencies.jar org.apache.giraph.GiraphRunner at.jku.tk.steinbauer.bigdata.giraph.MaxInDegreeComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip / user / hue / graph / tinygraph.txt -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op / user / hue / graph / degree -w 1 -ca giraph.SplitMasterWorker = false 
  

I ran Giraph 1.0.0 with hadoop 2.2.0 using the PageRank Benchmark example here.

Suddenly I got this error result:

Exception in thread "main" java.lang.IllegalArgumentException: checkLocalJobRunnerConfiguration: When using LocalJobRunner, must have only one worker since only 1 task at a time! at org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:151) at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:225) at org.apache.giraph.benchmark.GiraphBenchmark.run(GiraphBenchmark.java:90) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.giraph.benchmark.PageRankBenchmark.main(PageRankBenchmark.java:71) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

When I changed the number of workers to 1, I got:

Exception in thread "main" java.lang.IllegalArgumentException: checkLocalJobRunnerConfiguration: When using LocalJobRunner, you cannot run in split master / worker mode since there is only 1 task at a time! at org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:157) at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:225) at org.apache.giraph.benchmark.GiraphBenchmark.run(GiraphBenchmark.java:90) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.giraph.benchmark.PageRankBenchmark.main(PageRankBenchmark.java:71) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

Any solutions?

解决方案

Hi I assume you are not running on a cluster? If I run in our demo VMs I get the same error.

You can disable the split master worker behaviour in giraph-site.xml

giraph.SplitMasterWorker=false

If you just want to disable this during a one-shot exeuction you can also pass it as a command-line parameter to your program.

-ca giraph.SplitMasterWorker=false

For instance I run a demo for my Big Data lecture like this:

#!/bin/bash

yarn jar /root/giraph-0.0.1-SNAPSHOT-jar-with-dependencies.jar org.apache.giraph.GiraphRunner at.jku.tk.steinbauer.bigdata.giraph.MaxInDegreeComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/hue/graph/tinygraph.txt -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat  -op /user/hue/graph/degree -w 1 -ca giraph.SplitMasterWorker=false

这篇关于Apache Giraph - 由于一次只有1个任务，因此无法在拆分主/从模式下运行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！