Mesos 0.21.0上的Hadoop 2.5.0与库0.0.8执行程序错误 [英] Hadoop 2.5.0 on Mesos 0.21.0 with library 0.0.8 executor error

问题描述

stderr 在执行map-reduce作业时记录以下内容：

  root @ dbpc42：/ tmp / mesos / slaves / 20141201-225046-698725789-5050-19765-S24 / frameworks / 20141201-225046-698725789-5050-19765-0016 / executors / executor_Task_Tracker_2 / runs / latest＃ls 
 hadoop-2.5.0-cdh5.2.0 hadoop-2.5.0-cdh5.2.0.tgz stderr stdout 
  

stderr 的内容：

$ b

警告：在InitGoogleLogging写入STDERR I1202
19：41：40.323521 7223 fetcher.cpp：76]获取URI
'hdfs：// dbpc41：9000 / hadoop-2.5.0-cdh5.2.0.tgz'I1202 19 ：41：40.323685
7223 fetcher.cpp：105]将资源从
'hdfs：// dbpc41：9000 / hadoop-2.5.0-cdh5.2.0.tgz'下载到
'/ TMP / mesos /从站/ 20141201-225046-698725789-5050-19765-S24 /框架/ 20141201-225046-698725789-5050-19765-0016 /执行人/ executor_Task_Tracker_0 /运行/ 59c9a5ac-c376-4e9e-b9b7-63a24ae84dcd / hadoop- 2.5.0-cdh5.2.0。 tgz'

I1202 19：41：48.779220 7223 fetcher.cpp：64]提取的资源
'/ tmp / mesos / slaves / 20141201-225046-698725789- 5050-19765-S24 / frameworks / 20141201-225046-698725789-5050-19765-0016 / executors / executor_Task_Tracker_0 / runs / 59c9a5ac-c376-4e9e-b9b7-63a24ae84dcd / hadoop-2.5.0-cdh5.2.0.tgz'
到
'/ tmp / mesos / slaves / 20141201-225046-698725789-5050-19765-S24 / frameworks / 20141201-225046-698725789-5050-19765-0016 / executors / executor_Task_Tracker_0 / runs / 59c9a5ac-c376 -4e9e-b9b7-63a24ae84dcd'

sh：1：cd：can not cd to hadoop-2.5.0-cdh5.2.0

sh：1：./bin/hadoop：不是
找到

stdout 是空的。

有趣的部分是，当我在master运行的同一节点上运行slave时，问题不会出现，并且任务成功完成。只有远程奴隶引入了这个问题。什么可能是问题？

解决方案

你需要做的是给+ x访问，无论你的mesos-work-dir是什么位于。你不需要777递归。

The stderr logs the following while running a map-reduce job:

root@dbpc42:/tmp/mesos/slaves/20141201-225046-698725789-5050-19765-S24/frameworks/20141201-225046-698725789-5050-19765-0016/executors/executor_Task_Tracker_2/runs/latest# ls
hadoop-2.5.0-cdh5.2.0  hadoop-2.5.0-cdh5.2.0.tgz  stderr  stdout

Contents of stderr:

WARNING: Logging before InitGoogleLogging() is written to STDERR I1202 19:41:40.323521 7223 fetcher.cpp:76] Fetching URI 'hdfs://dbpc41:9000/hadoop-2.5.0-cdh5.2.0.tgz' I1202 19:41:40.323685 7223 fetcher.cpp:105] Downloading resource from 'hdfs://dbpc41:9000/hadoop-2.5.0-cdh5.2.0.tgz' to '/tmp/mesos/slaves/20141201-225046-698725789-5050-19765-S24/frameworks/20141201-225046-698725789-5050-19765-0016/executors/executor_Task_Tracker_0/runs/59c9a5ac-c376-4e9e-b9b7-63a24ae84dcd/hadoop-2.5.0-cdh5.2.0.tgz'

I1202 19:41:48.779220 7223 fetcher.cpp:64] Extracted resource '/tmp/mesos/slaves/20141201-225046-698725789-5050-19765-S24/frameworks/20141201-225046-698725789-5050-19765-0016/executors/executor_Task_Tracker_0/runs/59c9a5ac-c376-4e9e-b9b7-63a24ae84dcd/hadoop-2.5.0-cdh5.2.0.tgz' into '/tmp/mesos/slaves/20141201-225046-698725789-5050-19765-S24/frameworks/20141201-225046-698725789-5050-19765-0016/executors/executor_Task_Tracker_0/runs/59c9a5ac-c376-4e9e-b9b7-63a24ae84dcd'

sh: 1: cd: can't cd to hadoop-2.5.0-cdh5.2.0

sh: 1: ./bin/hadoop: not found

The stdout is empty.

The interesting part is, that when I run the slave on the same node where the master runs, the problem does not appear and tasks finish successfully. Only remote slaves introduce this problem. What could be the problem?

解决方案

What you need to do is to give +x access wherever your mesos-work-dir is located. You don't need 777 recursively.

这篇关于Mesos 0.21.0上的Hadoop 2.5.0与库0.0.8执行程序错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！