hadoop2.x-eclipse插件编译
警告:本人测试,以下hadoop-2.7.2版本插件编译完成之后,安装插件之后出现错误Could not initialize class org.apache.hadoop.hdfs.DFSConfigKeys。
所以本人重新安装替换hadoop-2.6.5,编译通过,没有问题,插件可用。
(ps:我不清楚是我当初2.7.2配置出错了,还是本身版本问题。编译步骤没有错误)
需要的资源:
作者环境是
windows10 64bit
1、hadoop2.x-eclipse插件源码
2、ant编译工具,可到官网下载
3、jdk,可到官网下载
4、eclipse-luna-4.4
5、hadoop-2.7.2
hadoop插件源码包下载地址:https://github.com/winghc/hadoop2x-eclipse-plugin
一、首先安装JDK,环境变量配置就不多说了。
二、ant安装配置,同样可检索配置方法。可能遇到的问题:
Unable to locate tools.jar. Expected to find it in C:\Program Files\Java\jre\lib
解决办法:将jdk1.7.0\lib\目录下的tools.jar拷贝到jre\lib目录中,即可解决。
三、将hadoop2.7.2以及hadoop2x-eclipse-plugin-master分别解压至D盘目录下。D:\hadoop-2.7.2 和 D:\hadoop2x-eclipse-plugin-master 以及eclipse 目录D:\eclipse
四、命令行cmd进入D:\hadoop2x-eclipse-plugin-master目录
五、输入命令:ant jar -Dversion=2.7.2 -Dhadoop.version=2.7.2 -Declipse.home=D:/eclipse -Dhadoop.home=D:/hadoop-2.7.2 回车编译。
编译在ivy-resolve-common:卡住。原因是有些依赖包没有下载下来。一般网速可以是能够通过的
解决办法:在hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\build.xml 文件中找到 <target name="compile" depends="init, ivy-retrieve-common" unless="skip.contrib">
去掉depends <target name="compile" unless="skip.contrib">
之后再次执行,会出现一下jar包版本不对应的问题
本人的jar包部分是这样修改的,全部对应hadoop-2.7.2对应的版本号,即hadoop2x-eclipse-plugin-master\ivy\libraries.properties 对应 hadoop-2.7.2\share\hadoop\common\lib下的jar版本号。
<copy file="${hadoop.home}/share/hadoop/common/lib/protobuf-java-${protobuf.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<copy file="${hadoop.home}/share/hadoop/common/lib/log4j-${log4j.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<copy file="${hadoop.home}/share/hadoop/common/lib/commons-cli-${commons-cli.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<copy file="${hadoop.home}/share/hadoop/common/lib/commons-configuration-${commons-configuration.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<copy file="${hadoop.home}/share/hadoop/common/lib/commons-lang-${commons-lang.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<copy file="${hadoop.home}/share/hadoop/common/lib/commons-collections-${commons-collections.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<copy file="${hadoop.home}/share/hadoop/common/lib/jackson-core-asl-${jackson.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<copy file="${hadoop.home}/share/hadoop/common/lib/jackson-mapper-asl-${jackson.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<copy file="${hadoop.home}/share/hadoop/common/lib/slf4j-log4j12-${slf4j-log4j12.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<copy file="${hadoop.home}/share/hadoop/common/lib/slf4j-api-${slf4j-api.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<copy file="${hadoop.home}/share/hadoop/common/lib/guava-${guava.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<copy file="${hadoop.home}/share/hadoop/common/lib/hadoop-auth-${hadoop.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<copy file="${hadoop.home}/share/hadoop/common/lib/commons-cli-${commons-cli.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<copy file="${hadoop.home}/share/hadoop/common/lib/netty-${netty.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<copy file="${hadoop.home}/share/hadoop/common/lib/htrace-core-${htrace.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<-- 以下添加内容-->
<copy file="${hadoop.home}/share/hadoop/common/lib/commons-httpclient-${commons-httpclient.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<copy file="${hadoop.home}/share/hadoop/common/lib/commons-io-${commons-io.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<copy file="${hadoop.home}/share/hadoop/common/lib/servlet-api-${servlet-api.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<jar
jarfile="${build.dir}/hadoop-${name}-${hadoop.version}.jar"
manifest="${root}/META-INF/MANIFEST.MF">
<manifest>
<attribute name="Bundle-ClassPath"
value="classes/,
lib/hadoop-mapreduce-client-core-${hadoop.version}.jar,
lib/hadoop-mapreduce-client-common-${hadoop.version}.jar,
lib/hadoop-mapreduce-client-jobclient-${hadoop.version}.jar,
lib/hadoop-auth-${hadoop.version}.jar,
lib/hadoop-common-${hadoop.version}.jar,
lib/hadoop-hdfs-${hadoop.version}.jar,
lib/protobuf-java-${protobuf.version}.jar,
lib/log4j-${log4j.version}.jar,
lib/commons-cli-${commons-cli.version}.jar,
lib/commons-configuration-${commons-configuration.version}.jar,
lib/commons-httpclient-${commons-httpclient.version}.jar,
lib/commons-lang-${commons-lang.version}.jar,
lib/commons-collections-${commons-collections.version}.jar,
lib/jackson-core-asl-${jackson.version}.jar,
lib/jackson-mapper-asl-${jackson.version}.jar,
lib/slf4j-log4j12-${slf4j-log4j12.version}.jar,
lib/slf4j-api-${slf4j-api.version}.jar,
lib/guava-${guava.version}.jar,
lib/netty-${netty.version}.jar,
lib/htrace-core-${htrace.version}.jar,
<-- 以下添加内容-->
lib/commons-io-${commons-io.version}.jar,
lib/servlet-api-${servlet-api.version}.jar"/>
即编译成功
可能会缺少commons-httpclient-3.1.jar,commons-io-2.4.jar,servlet-api-2.5.jar三个包,导致远程连接失败
更多推荐

所有评论(0)