如何将JMX远程连接到Dataproc上的Spark worker [英] How to connect with JMX remotely to Spark worker on Dataproc
问题描述
通过添加以下内容,我可以很好地连接到驱动程序:
I can connect to the driver just fine by adding the following:
spark.driver.extraJavaOptions=-Dcom.sun.management.jmxremote \
-Dcom.sun.management.jmxremote.port=9178 \
-Dcom.sun.management.jmxremote.authenticate=false \
-Dcom.sun.management.jmxremote.ssl=false
但是做...
spark.executor.extraJavaOptions=-Dcom.sun.management.jmxremote \
-Dcom.sun.management.jmxremote.port=9178 \
-Dcom.sun.management.jmxremote.authenticate=false \
-Dcom.sun.management.jmxremote.ssl=false
...只会在驱动程序上产生一堆错误...
... only yield a bunch of errors on the driver ...
Container id: container_1501548048292_0024_01_000003
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:972)
at org.apache.hadoop.util.Shell.run(Shell.java:869)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1170)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:236)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:305)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Container exited with a non-zero exit code 1
...最终使工作崩溃.
... and finally crashes the job.
工人没有任何错误,只需退出:
There are no errors on the workers, it simply exits with:
[org.apache.spark.util.ShutdownHookManager] - Shutdown hook called
Spark v2.2.0
,并且该群集是一个简单的1m-2w配置,并且我的作业运行时没有执行者参数而没有问题.
Spark v2.2.0
, and the cluster is a simple 1m-2w-configuration, and my jobs run without issues without the executor parameters.
推荐答案
正如 Rick Mortiz 指出的那样,问题是执行器jmx的端口冲突.
As Rick Mortiz pointed out, the issue was conflicting ports for the executor jmx.
设置:
-Dcom.sun.management.jmxremote.port=0
产生一个随机端口,并从Spark中删除了错误.要使用do找出最终使用哪个端口,请执行以下操作:
yields a random port, and removed the errors from Spark. To figure out which port it ends up using do:
netstat -alp | grep LISTEN.*<executor-pid>/java
列出了该进程当前打开的端口.
which lists the currently open ports for that process.
这篇关于如何将JMX远程连接到Dataproc上的Spark worker的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!