Hadoop 2.6.1警告：WARN util.NativeCodeLoader [英] Hadoop 2.6.1 Warning: WARN util.NativeCodeLoader

问题描述

我在OS X 10.10.5上运行hadoop 2.6.1。
我收到此警告：

I'm running hadoop 2.6.1 on OS X 10.10.5. I'm getting this warning:

WARN util.NativeCodeLoader：无法为您的平台加载native-hadoop库...使用builtin-java类，其中适用

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

我已经读过这个问题可能是由运行32位本机库libhadoop.so.1.0.0和64位版本的hadoop引起的。我已经检查了我的libhadoop.so.1.0.0版本，它是64位。

I've read that this problem can be caused by running a 32bit native library libhadoop.so.1.0.0 with a 64 bit version of hadoop. I've checked my version of libhadoop.so.1.0.0 and it is 64 bit.

$ find ~/hadoop-2.6.1/ -name libhadoop.so.1.0.0 -ls
136889669     1576 -rwxr-xr-x    1 davidlaxer       staff              806303 Sep 16 14:18 /Users/davidlaxer/hadoop-2.6.1//lib/native/libhadoop.so.1.0.0

$ file /Users/davidlaxer/hadoop-2.6.1//lib/native/libhadoop.so.1.0.0
    /Users/davidlaxer/hadoop-2.6.1//lib/native/libhadoop.so.1.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped

$ env | grep HADOOP
HADOOP_HOME=/Users/davidlaxer/hadoop-2.6.1
HADOOP_COMMON_LIB_NATIVE_DIR=/Users/davidlaxer/hadoop-2.6.1/lib/native
HADOOP_INSTALL=/Users/davidlaxer/hadoop-2.6.1
HADOOP_CONF_DIR=/Users/davidlaxer/hadoop-2.6.1/etc/hadoop
HADOOP_OPTS=-Djava.library.path=/Users/davidlaxer/hadoop-2.6.1/lib

$ hadoop version
Hadoop 2.6.1
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r b4d876d837b830405ccdb6af94742f99d49f9c04
Compiled by jenkins on 2015-09-16T21:07Z
Compiled with protoc 2.5.0
From source with checksum ba9a9397365e3ec2f1b3691b52627f
This command was run using /Users/davidlaxer/hadoop-2.6.1/share/hadoop/common/hadoop-common-2.6.1.jar

$ java -version
java version "1.8.0_05"
Java(TM) SE Runtime Environment (build 1.8.0_05-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode)

$ hadoop checknative -a
15/09/26 11:01:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Native library checking:
hadoop:  false 
zlib:    false 
snappy:  false 
lz4:     false 
bzip2:   false 
openssl: false 
15/09/26 11:01:29 INFO util.ExitUtil: Exiting with status 1


$ sudo port list zlib
Password:
zlib                           @1.2.8          archivers/zlib
$ sudo port list snappy
snappy                         @1.1.1          archivers/snappy
$ sudo port list lz4
lz4                            @r130           archivers/lz4
$ sudo port list bzip2
bzip2                          @1.0.6          archivers/bzip2
$ sudo port list openssl
openssl                        @1.0.2d         devel/openssl

$env | grep CLASS

CLASSPATH=/users/davidlaxer/trunk/core/src/test/java/:/Users/davidlaxer/hadoop-2.6.1-src/hadoop-dist/target/hadoop-dist-2.6.1.jar:/Users/davidlaxer/clojure/target:/Users/davidlaxer/hadoop-2.6.1/lib/native:

˚任何想法？

推荐答案

我遇到了同样的问题。幸运的是问题已经解决了。
您可以在下面看到解决方案。

I had same problem. Fortunately the problem has been solved. You can see the solution below.

有几次显示警告不允许您将文件上传到HDFS。
所以当没有警告时，没有这样的问题。

A few times displaying the warning does not let you upload a file to HDFS. So when there is no warning, there is no such problem.

打开 hadoop-env.sh 并转到文件的结尾并尝试添加以下行

Open the hadoop-env.sh and go to the end of the file and try to add the following lines

export HADOOP_HOME_WARN_SUPPRESS=1
export HADOOP_ROOT_LOGGER="WARN,DRFA"

这篇关于Hadoop 2.6.1警告：WARN util.NativeCodeLoader的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！