Linux环境编译Hadoop源码包

hadoop2.7.4

centos6.5

1.准备的资料

源码根目录下有个BUILDINT.txt，打开即可看见里面关于编译hadoop的一些环境要求

/opt/software/ 存放软件安装包

/opt/module/ 存放解压包

2.安装JDK(1.8)

[root@master-node software]# tar -zxvf jdk-8u211-linux-x64.tar.gz -C /opt/module/ [root@master-node software]# vim /etc/profile #Java配置 export JAVA_HOME=/opt/module/jdk1.8.0_211 export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar export PATH=$JAVA_HOME/bin:$PATH [root@master-node software]# source /etc/profile [root@master-node software]# java -version

3.安装maven(3.6.1)

[root@master-node software]# tar -zxvf apache-maven-3.6.1-bin.tar.gz -C /opt/module/ [root@master-node apache-maven-3.6.1]# vim conf/settings.xml <mirrors> <!--添加阿里镜像--> <mirror> <id>alimaven</id> <mirrorOf>central</mirrorOf> <name>aliyun maven</name> <url>https://maven.aliyun.com/repository/central</url> </mirror> <mirror> <id>aliyun-maven</id> <mirrorOf>*</mirrorOf> <name>aliyun maven</name> <url>http://maven.aliyun.com/nexus/content/groups/public</url> </mirror> </mirrors> [root@master-node software]# vim /etc/profile #Maven配置 export MAVEN_HOME=/opt/module/apache-maven-3.6.1 export PATH=$PATH:$MAVEN_HOME/bin [root@master-node software]# source /etc/profile [root@master-node software]# mvn -version

4.安装cmake,zlib-devel,openssl-devel 依赖库

[root@master-node software]# yum -y install cmake zlib-devel openssl-devel

5.安装ProtocolBuffer 2.5.0

[root@slave-node1 software]# tar -zxvf protobuf-2.5.0.tar.gz -C /opt/module/ [root@slave-node1 software]# cd /opt/module/protobuf-2.5.0/ [root@master-node protobuf-2.5.0]# ./configure --prefix=/opt/module/protobuf [root@master-node protobuf-2.5.0]# make [root@master-node protobuf-2.5.0]# make check [root@master-node protobuf-2.5.0]# make install [root@master-node protobuf-2.5.0]# vim /etc/profile #Protobuf配置 export PROTOC_HOME=/opt/module/protobuf export PATH=$PATH:$PROTOC_HOME/bin [root@master-node protobuf-2.5.0]# source /etc/profile [root@master-node protobuf-2.5.0]# protoc --version

到此，编译工具安装基本完成

6.编译Hadoop

1）解压源码到/opt/module/目录

[root@master-node software]# tar -zxvf hadoop-2.7.4-src.tar.gz -C /opt/module/

2）进入到Hadoop源码主目录，通过maven命令执行编译

[root@master-node software]# cd /opt/module/hadoop-2.7.4-src/ [root@master-node hadoop-2.7.4-src]# mvn package -Pdist,native -DskipTests -Dtar

编译成功

3）编译成功的64位hadoop包在/opt/module/hadoop-2.7.4-src/hadoop-dist/target目录下

[root@master-node target]# cd /opt/module/hadoop-2.7.4-src/hadoop-dist/target