将命令行参数传递给jvm(java)映射器任务 [英] pass a command line argument to jvm(java) mapper task

问题描述

我想调试映射器的某些部分，为此我需要将一些命令行参数传递给启动映射器的jvm(java)进程.有哪些不同的方法?

I want to debug some parts of my mapper for which I need to pass some command line arguments to the jvm(java) process which starts the mapper. What are the different ways to do this?

我想出了一种更改MapTaskRunner.java的方法，但我想避免编译整个hadoop包.应该使用一些简单的方法来使用配置文件将额外的命令行参数传递给jvm映射器进程.

I figured out one way to change MapTaskRunner.java, but I want to avoid compiling the whole hadoop package. There should be some simple way using a configuration file to pass extra command line arguments to the jvm mapper process.

推荐答案

我想您正在mapred-config.xml中寻找以下配置:

I guess you are looking for the following configuration in mapred-config.xml:

<property>
    <name>mapred.child.java.opts</name>
    <value>-Xmx4096m -XX:+UseConcMarkSweepGC</value>
</property>

在值部分，您可以使用-D设置参数.

In value part you may set your arguments using -D.

或者，您可以在终端中设置HADOOP_OPTS:

Alternatively you may set the HADOOP_OPTS in your terminal:

export HADOOP_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5000"

有关本地调试hadoop作业的更多信息，请参见

For more info on local debugging hadoop jobs, see here.

这篇关于将命令行参数传递给jvm(java)映射器任务的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！