解决unable to load native-library问题
系统配置:win7 64位机上安装的VMWARE10
Ubuntu14.10
Hadoop2.7.0
从hadoop下载了最新的hadoop-2.7.,经过漫长的等待,终于编译成功,然后按照网上的教程(wwwblogs/kinglau/p/3794433.html),安装单机模式,配置好之后,通过执行hadoop自带实例WordCount验证是否安装成功。结果发现报unable to load native-hadoop library for your platform … 如下图所示:
在网上查了很多资料,有的让更改lib/native里的链接、有的说是hadoop2以后版本自带的本地库是32位版本的,让从网上下载64位的,然后覆盖lib/native里的所有文件、有的说让重新编译,我很认真的实现了所有网上的有可能的做法,结果很沮丧的发现还是unable to …
最后一不小心就到hadoop(/docs/current/hadoop-project-dist/hadoop-common/NativeLibraries.html)里去答案,发现可以首先用NativeLibraryChecker 工具来检查native libraries是否被正确的下载。原文如下:NativeLibraryChecker is a tool to check whether native libraries are loaded correctly. Y ou can launch NativeLibraryChecker as follows:
我用sbin/start-all.sh打开hadoop,然后用hadoop checknative –a 指令检查,发现全是false,如下所示:
继续看hadoop官方资料,发现原文里有关于hadoop Runtime的描述,原文如下:
发现最后的If something goes wrong…正是我现在的问题,再看一下前边的描述,再仔细看,发现需要更改-Djava.library.path=<path>。突然觉得这个-Djava.library.path很面熟,就打开了~/.bashrc,在这儿终于到了-Djava.library.path,原来设置如下:
export JAVA_HOME=/home/brian/hadoop/jdk
export PATH=$JAVA_HOME/bin:$PATH
export M2_HOME=/home/brian/hadoop/maven
PATH=$M2_HOME/bin:$PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
export ANT_HOME=/home/brian/hadoop/ant
export PATH=$ANT_HOME/bin:$PATH
export FINDBUGS_HOME=/home/brian/hadoop/findbugs
export PATH=$FINDBUGS_HOME/bin:$PATH
#HADOOP VARIABLES START
export HADOOP_INSTALL=/home/brian/hadoop/hadoop
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
unableexport HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib"
#HADOOP VARIABLES END
于是突然心血来潮,给lib后边加了native,更给完后如下所示:
export JAVA_HOME=/home/brian/hadoop/jdk
export PATH=$JAVA_HOME/bin:$PATH
export M2_HOME=/home/brian/hadoop/maven
PATH=$M2_HOME/bin:$PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
export ANT_HOME=/home/brian/hadoop/ant
export PATH=$ANT_HOME/bin:$PATH
export FINDBUGS_HOME=/home/brian/hadoop/findbugs
export PATH=$FINDBUGS_HOME/bin:$PATH
#HADOOP VARIABLES START
export HADOOP_INSTALL=/home/brian/hadoop/hadoop
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib/native"
#HADOOP VARIABLES END
本来没有报任何希望(因为native本来就包括在lib文件夹里),只是机械的再次check了一下,结果奇迹出现了:
然后重新hadoop自带实例WordCount,结果看到了讨厌的WARN一去不返!!!
版权声明:本站内容均来自互联网,仅供演示用,请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198,我们将在24小时内删除。
发表评论