Hadoop 伪分布式模式 - Datanode 和 tasktracker 未启动 [英] Hadoop pseudo distributed mode - Datanode and tasktracker not starting
问题描述
我正在运行 Red Hat Enterprise Linux Server 6.4(圣地亚哥)发行版，上面安装了 Hadoop 1.1.2.我已经进行了启用伪分布式模式所需的配置.但是在尝试运行 hadoop 时，datanode 和 tasktracker 没有启动.
I am running a Red Hat Enterprise Linux Server release 6.4 (Santiago) distribution with Hadoop 1.1.2 installed on it. I have made the required configurations to enable the pseudo distributed mode. But on trying to run hadoop, the datanode and tasktracker don't start.
我无法将任何文件复制到 hdfs.
I am not able to copy any files to hdfs.
[hduser@is-joshbloom-hadoop hadoop]$ hadoop dfs -put README.txt /input
Warning: $HADOOP_HOME is deprecated.
13/05/23 16:42:00 WARN hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /input could only be replicated to 0 nodes, instead of 1
也在尝试 hadoop-daemon.sh start datanode 之后我得到消息:
Also after trying hadoop-daemon.sh start datanode I get the message:
starting datanode, logging to /usr/local/hadoop/libexec/../logs/hadoop-hduser-datanode-is-joshbloom-hadoop.out
任务跟踪器也是如此.但是当我对 namenode、secondarynamenode、jobtracker 尝试相同的命令时，它们似乎正在运行.
same goes for tasktracker. But when I try the same command for namenode, secondarynamenode, jobtracker they are seem to be running.
namenode running as process 32933. Stop it first.
我尝试了以下解决方案:
I tried the following solutions:
- 重新格式化名称节点
- 重新安装hadoop
- 安装不同版本的 hadoop (1.0.4)
似乎都不起作用.我在 Mac 和 amazon ubuntu VM 上遵循了相同的安装步骤，并且运行良好.
None seem to work. I have followed the same installation steps on my Mac and on amazon ubuntu VM and it works perfectly.
我怎样才能让 hadoop 工作?谢谢！
How can I get hadoop working? Thanks!
*更新**
*UPDATE**
这里是namenode的日志条目
Here is the log entry of namenode
2013-05-23 16:27:44,087 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting DataNode
STARTUP_MSG: host = java.net.UnknownHostException: is-joshbloom-hadoop: is-joshbloom-hadoop
STARTUP_MSG: args = []
STARTUP_MSG: version = 1.1.2
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1 -r 1440782; compiled by 'hortonfo' on Thu Jan 31 02:03:24 UTC 2013
************************************************************/
2013-05-23 16:27:44,382 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2013-05-23 16:27:44,432 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
2013-05-23 16:27:44,446 ERROR org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Error getting localhost name. Using 'localhost'...
java.net.UnknownHostException: is-joshbloom-hadoop: is-joshbloom-hadoop
at java.net.InetAddress.getLocalHost(InetAddress.java:1438)
at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.getHostname(MetricsSystemImpl.java:463)
at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.configureSystem(MetricsSystemImpl.java:394)
at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.configure(MetricsSystemImpl.java:390)
at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:152)
at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:133)
at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.init(DefaultMetricsSystem.java:40)
at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.initialize(DefaultMetricsSystem.java:50)
at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1589)
at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1608)
at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1734)
at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1751)
Caused by: java.net.UnknownHostException: is-joshbloom-hadoop
at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:866)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1258)
at java.net.InetAddress.getLocalHost(InetAddress.java:1434)
... 11 more
2013-05-23 16:27:44,453 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2013-05-23 16:27:44,453 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started
2013-05-23 16:27:44,768 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered.
2013-05-23 16:27:44,914 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library
2013-05-23 16:27:45,212 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.net.UnknownHostException: is-joshbloom-hadoop: is-joshbloom-hadoop
at java.net.InetAddress.getLocalHost(InetAddress.java:1438)
at org.apache.hadoop.security.SecurityUtil.getLocalHostName(SecurityUtil.java:271)
at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:289)
at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:301)
at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1651)
at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1590)
at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1608)
at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1734)
at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1751)
Caused by: java.net.UnknownHostException: is-joshbloom-hadoop
at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:866)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1258)
at java.net.InetAddress.getLocalHost(InetAddress.java:1434)
... 8 more
2013-05-23 16:27:45,228 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at java.net.UnknownHostException: is-joshbloom-hadoop: is-joshbloom-hadoop
************************************************************/
*更新***
*UPDATE***
/etc/hosts的内容
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
推荐答案
修改您的 /etc/hosts 以包含主机名环回映射:
Amend your /etc/hosts to include a hostname loopback mapping:
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
127.0.1.1 is-joshbloom-hadoop
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
您的问题是您的机器不知道如何将主机名 is-joshbloom-hadoop 解析为特定的 IP 地址.通常有两个地方/方法解析发生 - 通过 DNS 服务器或使用本地主机文件(主机文件优先).
Your problem is your machine doesn't know how to resolve the hostname is-joshbloom-hadoop to a specific IP address. There are typically two places/methods resolution occurs - either via a DNS server or using a local hosts file (hosts file takes precedence).
以上对您的hosts文件的修改允许您的机器将机器名称is-joshbloom-hadoop解析为IP地址127.0.1.1.操作系统具有范围 127.0.0.0/8 的内部环回地址，因此您可以在此处命名任何地址.在我的 Ubuntu 笔记本电脑上，它使用 127.0.1.1 并且我确定它会在操作系统之间发生变化，但我的猜测是不使用 127.0.0.1 你没有如果您以后更改机器名称，请在 localhost 行中搜索它.
The above amendment to your hosts file allows you machine to resolve the machine name is-joshbloom-hadoop to the IP address 127.0.1.1. The OS has an internal loopback address for the range 127.0.0.0/8, so you could name any address in here. On my Ubuntu laptop, it uses the 127.0.1.1 and i'm sure it changes between OS's, but my guess is by not using 127.0.0.1 you don't have to search for it in the localhost line if you change your machine name in future.
这篇关于Hadoop 伪分布式模式 - Datanode 和 tasktracker 未启动的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
