/ bin / bash：/ bin / java：没有这样的文件或目录 [英] /bin/bash: /bin/java: No such file or directory
问题描述
我试图在Mac OS X EL Captain 10.11上使用Java 1.7 SDK和Hadoop2.7.1运行一个简单的wordcount MapReduce程序，并且在我的容器日志stderr
/ bin /中收到以下错误消息： bash：/ bin / java：没有这样的文件或目录
应用程序日志 -
5/11/27 02:52:33 WARN util.NativeCodeLoader：无法为您的平台加载native-hadoop库......在适用的情况下使用builtin-java类
15/11/27 02： 52:33 INFO client.RMProxy：连接到ResourceManager，位于/192.168.200.96:8032
15/11/27 02:52:34 INFO input.FileInputFormat：要处理的总输入路径：0
15 / 11/27 02:52:34信息mapreduce.JobSubmitter：分割次数：0
15/11/27 02:52:34信息mapreduce.JobSubmitter：提交作业的标记：job_1448608832342_0003
15/11 / 27 02:52:34 INFO impl.YarnClientImpl：提交的应用程序application_1448608832342_0003
15/11/27 02:52:34信息mapreduce.Job：跟踪作业的URL：http：//mymac.l ocal：8088 / proxy / application_1448608832342_0003 /
15/11/27 02:52:34信息mapreduce.Job：正在运行的作业：job_1448608832342_0003
15/11/27 02:52:38信息mapreduce.Job： Job job_1448608832342_0003以超级模式运行：false
15/11/27 02:52:38信息mapreduce.Job：map 0％reduce 0％
15/11/27 02:52:38信息mapreduce。作业：作业job_1448608832342_0003失败，状态为失败，原因是：应用程序application_1448608832342_0003因AM容器失败2次失败appattempt_1448608832342_0003_000002退出并返回exitCode：127
有关更详细的输出，请查看应用程序跟踪页面：http：//mymac.local：8088 / cluster / app / application_1448608832342_0003然后，点击每次尝试日志的链接。
诊断：容器启动异常。
容器id：container_1448608832342_0003_02_000001
退出代码：127
堆栈轨迹：ExitCodeException exitCode = 127：
at org.apache.hadoop.util.Shell.runCommand（Shell.java:545 ）
在org.apache.hadoop.util.Shell.run（Shell.java:456）
在org.apache.hadoop.util.Shell $ ShellCommandExecutor.execute（Shell.java:722）
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer（DefaultContainerExecutor.java:211）
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call （ContainerLaunch.java:302）
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call（ContainerLaunch.java:82）
at java.util.concurrent.FutureTask .run（FutureTask.java:262）
at java.util.concurrent.ThreadPoolExecutor.runWorker（ThreadPoolExecutor.java:1145）$ b $ at java.util.concurrent.ThreadPoolExecutor $ Worker.run（ThreadPoolExecutor.java ：615）
在java.lang .Thread.run（Thread.java:745）
使用非零退出代码退出的容器127
尝试失败。申请失败。
15/11/27 02:52:38信息mapreduce.Job：Counters：0
命令我正在运行
hadoop jar wordcount.jar org.myorg.WordCount / user / gangadharkadam / input / / user / gangadharkadam / output /
我的ENV变量是 -
export JAVA_HOME = / Library / Java / JavaVirtualMachines / jdk1.7.0_79.jdk / Contents / Home
export HADOOP_HOME = / usr / local / hadoop / hadoop-2.7.1
export HADOOP_MAPRED_HOME = / usr / local / hadoop / hadoop-2.7.1
export HADOOP_COMMON_HOME = / usr / local / hadoop / hadoop-2.7.1 $ b $ export HADOOP_HDFS_HOME = / usr / local / hadoop / hadoop-2.7.1
export YARN_HOME = / usr / local / hadoop / hadoop-2.7.1
export HADOOP_CONF_DIR = / usr / local / hadoop / hadoop-2.7.1 / etc / hadoop
export HADOOP_CLASSPATH = $ JAVA_HOME / lib / tools.jar
$ b $ export PATH = $ PATH：$ JAVA_HOME / bin：$ HADOOP_HOME / bin：HADOOP_HOME / sbin：$ M2_HOME / bin：$ ANT_HOME /斌：$ IVY_HOME /斌：$ FB_HOME /斌：$ MYSQL_HOME /斌：$ MYSQL_HOME / lib目录下：$ SQOOP_HOME /箱
这个问题似乎是因为YARN使用不同的路径来执行不同的JAVA可执行文件， 。 stderr中失败任务的本地日志显示 -
/ bin / bash：/ bin / java：没有这样的文件或目录
创建一个从$ JAVA_HOM / bin / java到/ bin / java的软链接，但是在El Captian OS X中，但它不允许创建一个软链接。新OS X EL Captian具有无根用户登录，用户无法在/ bin /等特定受限制文件夹上创建任何内容。
任何解决此问题的方法都会得到高度赞赏。提前感谢。解决方案
此解决方案适用于Hadoop版本2.6.0及更早版本。
禁用SIP并创建符号链接确实提供了解决方法。
更好的解决方案是修复hadoop-config.sh，以便它正确地获得您的JAVA_HOME
在 HADOOP_HOME / libexec / hadoop -config.sh 查找下面的if条件＃如果未设置，尝试设置JAVA_HOME
删除导出JAVA_HOME行中的额外括号，如下所示。如果[-x / usr / libexec / java_home];更改此
然后
export JAVA_HOME =（$（/ usr / libexec / java_home））
else
export JAVA_HOME =（/ Library / Java / Home）
$
至
if [ -x / usr / libexec / java_home];然后
//注意额外的括号被删除
export JAVA_HOME = $（/ usr / libexec / java_home）
else
export JAVA_HOME = / Library / Java / Home
$
在您进行此更改后重新启动纱线。
更详细的信息可以在这里找到 https：//issues.apache。 org / jira / browse / HADOOP-8717 ，似乎Hadoop 3.0.0-alpha1是第一个修正版本。
I was trying to run a simple wordcount MapReduce Program using Java 1.7 SDK and Hadoop2.7.1 on Mac OS X EL Captain 10.11 and I am getting the following error message in my container log "stderr" /bin/bash: /bin/java: No such file or directory
Application Log-
5/11/27 02:52:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/11/27 02:52:33 INFO client.RMProxy: Connecting to ResourceManager at /192.168.200.96:8032
15/11/27 02:52:34 INFO input.FileInputFormat: Total input paths to process : 0
15/11/27 02:52:34 INFO mapreduce.JobSubmitter: number of splits:0
15/11/27 02:52:34 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1448608832342_0003
15/11/27 02:52:34 INFO impl.YarnClientImpl: Submitted application application_1448608832342_0003
15/11/27 02:52:34 INFO mapreduce.Job: The url to track the job: http://mymac.local:8088/proxy/application_1448608832342_0003/
15/11/27 02:52:34 INFO mapreduce.Job: Running job: job_1448608832342_0003
15/11/27 02:52:38 INFO mapreduce.Job: Job job_1448608832342_0003 running in uber mode : false
15/11/27 02:52:38 INFO mapreduce.Job: map 0% reduce 0%
15/11/27 02:52:38 INFO mapreduce.Job: Job job_1448608832342_0003 failed with state FAILED due to: Application application_1448608832342_0003 failed 2 times due to AM Container for appattempt_1448608832342_0003_000002 exited with exitCode: 127
For more detailed output, check application tracking page:http://mymac.local:8088/cluster/app/application_1448608832342_0003Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1448608832342_0003_02_000001
Exit code: 127
Stack trace: ExitCodeException exitCode=127:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
at org.apache.hadoop.util.Shell.run(Shell.java:456)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 127
Failing this attempt. Failing the application.
15/11/27 02:52:38 INFO mapreduce.Job: Counters: 0
Command I am Running
hadoop jar wordcount.jar org.myorg.WordCount /user/gangadharkadam/input/ /user/gangadharkadam/output/
My ENV variable are-
export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home
export HADOOP_HOME=/usr/local/hadoop/hadoop-2.7.1
export HADOOP_MAPRED_HOME=/usr/local/hadoop/hadoop-2.7.1
export HADOOP_COMMON_HOME=/usr/local/hadoop/hadoop-2.7.1
export HADOOP_HDFS_HOME=/usr/local/hadoop/hadoop-2.7.1
export YARN_HOME=/usr/local/hadoop/hadoop-2.7.1
export HADOOP_CONF_DIR=/usr/local/hadoop/hadoop-2.7.1/etc/hadoop
export HADOOP_CLASSPATH=$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:HADOOP_HOME/sbin:$M2_HOME/bin:$ANT_HOME/bin:$IVY_HOME/bin:$FB_HOME/bin:$MYSQL_HOME/bin:$MYSQL_HOME/lib:$SQOOP_HOME/bin
The problem seems to be because YARN is using different path for JAVA executable different then you have in your OS. the local logs for the failed task in "stderr" shows- /bin/bash: /bin/java: No such file or directory
I tried to create a soft link from $JAVA_HOM/bin/java to /bin/java but in El Captian OS X but its not allowing to create a softlink. The New OS X EL Captian has a rootless login and user can not create anything on certain restricted folders like /bin/. Any work around on this issue is highly appreciated.Thanks in advance.
This answer is applicable for Hadoop version 2.6.0 and earlier. Disabling SIP and creating a symbolic link does provide a workaround. A better solution is to fix the hadoop-config.sh so it picks up your JAVA_HOME correctly
In HADOOP_HOME/libexec/hadoop-config.sh look for the if condition below # Attempt to set JAVA_HOME if it is not set
Remove extra parentheses in the export JAVA_HOME lines as below. Change this
if [ -x /usr/libexec/java_home ]; then
export JAVA_HOME=($(/usr/libexec/java_home))
else
export JAVA_HOME=(/Library/Java/Home)
fi
to
if [ -x /usr/libexec/java_home ]; then
// note that the extra parentheses are removed
export JAVA_HOME=$(/usr/libexec/java_home)
else
export JAVA_HOME=/Library/Java/Home
fi
Restart yarn after you have made this change.
More detailed info can be found here https://issues.apache.org/jira/browse/HADOOP-8717 and seems that Hadoop 3.0.0-alpha1 is the first release with the fix.
这篇关于/ bin / bash：/ bin / java：没有这样的文件或目录的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
