UnknownHostException格式化HDFS时 [英] UnknownHostException while formatting HDFS

问题描述

我使用以下说明。在Hadoop配置文件中，所有内容都设置为localhost。但是，当我格式化名称节点时，会出现下面的异常。当我向 / etc / hosts 文件中添加一个 192.168.1.101 CentOSHost 条目时，异常消失，我可以运行格式化/启动HDFS并运行MR作业。

即使没有连接到网络，也没有添加条目到 / etc / hosts 文件。如何做到这一点？

12/08/27 22:17:15 WARN net.DNS：无法确定主机的地址 - 返回到localhost地址
java.net.UnknownHostException：CentOSHost：CentOSHost

at java.net.InetAddress.getLocalHost（InetAddress.java:1360）

在org.apache.hadoop.net.DNS.resolveLocalHostIPAddress（DNS.java:283）
$ or $ $ b $ org.apache.hadoop.net.DNS。（DNS.java:59）
at org .apache.hadoop.hdfs.server.namenode.NNStorage.newBlockPoolID（NNStorage.java:1017）

at org.apache.hadoop.hdfs.server.namenode.NNStorage.newNamespaceInfo（NNStorage.java:565 ）

at org.apache.hadoop.hdfs.server.namenode.FSImage.format（FSImage.java:145）

at org.apache.hadoop.hdfs.server.namenode .Node.format（NameNode.java:724）

at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode（NameNode.java:1095）
$ b $ at at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode（NameNode.java:1095） .apache.hadoop.hdfs.server.namenode.NameNode.ma in（NameNode.java:1193）

解决方案

它看起来像一些配置返回的地方/使用主机名称作为 CentOSHost 。

hostname --fqdn 返回给您？

对于Hadoop，重要的是名称查找和反向查找成功。您应该能够解析IP地址并从IP地址解析主机名（反向解析）。这可以使用上述命令进行测试。

要使反向分辨率正常工作，必须输入/ etc / hosts。除非条目和配置指向本地主机。即使在这种情况下，主机名--fqdn 应该返回为 localhost 。

I have installed CDH4 on CentOS 6.3 64-bit in Pseudo Distributed mode using the following instructions. Everything is set to localhost in the Hadoop configuration files. But, still when I format the name node the below exception appears. When I add an 192.168.1.101 CentOSHost entry to the /etc/hosts file the exception goes away and I am able to run format/start HDFS and run MR jobs.

I want to run MR jobs even when I am not connected to the network without adding an entry to the /etc/hosts file. How to get this done?

12/08/27 22:17:15 WARN net.DNS: Unable to determine address of the host-falling back to "localhost" address java.net.UnknownHostException: CentOSHost: CentOSHost
at java.net.InetAddress.getLocalHost(InetAddress.java:1360)
at org.apache.hadoop.net.DNS.resolveLocalHostIPAddress(DNS.java:283)
at org.apache.hadoop.net.DNS.(DNS.java:59) at org.apache.hadoop.hdfs.server.namenode.NNStorage.newBlockPoolID(NNStorage.java:1017)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.newNamespaceInfo(NNStorage.java:565)
at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:145)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:724)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1095)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1193)

解决方案

It looks like some where the configuration is returning/ using the hostname as CentOSHost.

What does hostname --fqdn returns to you?

For Hadoop, it is important that name look-up and reverse look-up work successfully. You should be able to resolve the ip-address and resolve hostname from the ip-address (Reverse resolution). This can be tested using the above command.

The entry to /etc/hosts is required for the reverse resolution to work. Unless the entry and the configuration are pointing to localhost. Even in that case the hostname --fqdn should return as localhost.

这篇关于UnknownHostException格式化HDFS时的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！