Oozie Shell动作记忆限制 [英] Oozie shell action memory limit

问题描述

我们有一个带有外壳操作的oozie工作流程，该操作比默认情况下由Yarn给出的地图任务需要更多的内存.

We have an oozie workflow with a shell action that needs more memory than what a map task is given by Yarn by default.

我们如何给它更多的存储空间?

How can we give it more memory?

我们尝试将以下配置添加到操作中:

We have tried adding the following configuration to the action:

<configuration>
  <property>
    <name>mapreduce.map.memory.mb</name>
    <value>6144</value> <!-- for example -->
  </property>
</configuration>

我们都将其设置为内联配置(在workflow.xml中)和jobXml. 都没有任何作用.

We have both set this as an inline (in the workflow.xml) configuration and as a jobXml. Neither has had any effect.

推荐答案

我们找到了答案:

shell操作作为oozie启动器"映射任务执行，并且此任务不使用常规配置属性.

A shell action is executed as an oozie "launcher" map task, and this task does not use the normal configuration properties.

相反，您必须在属性前面加上"oozie.launcher"，才能将其应用于启动程序任务.

Instead you have to prefix the properties with "oozie.launcher" to make them apply to the launcher task.

因此，在本例中，如果我们对shell操作使用以下配置，则它可以正常工作.

So in our case, if we use the following configuration for our shell action, it works.

<configuration>
  <property>
    <name>oozie.launcher.mapreduce.map.memory.mb</name>
    <value>6144</value> <!-- for example -->
  </property>
</configuration>

这在oozie文档中并不明显. 我们在这里找到了这个: http://downright-amazed.blogspot.com/2012/02/configure- oozies-launcher-job.html

This is not obvious from the oozie documentation. We found this here: http://downright-amazed.blogspot.com/2012/02/configure-oozies-launcher-job.html

这篇关于Oozie Shell动作记忆限制的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！