iDatamining.org

I am looking for projects to work on
Please contact with me at yiyu.jia@iDataMining.org!

Sunday, October 28, 2012

eclipse mapreduce plugin build for Hadoop 1.0.4



Here is a eclipse plugin built with Eclipse Juno and hadoop 1.0.4.

download jdk1.7 compatible from www.idatamining.org

download jdk1.6 compatible from www.idatamining.org

copy the hadoop-eclipse-plugin-1.0.4.jar into eclipse plugins directory and enjoy it.   

[Edit] See how to build eclipse plugin from the source code here: build hadoop eclipse plugin from the source code

Saturday, October 6, 2012

Building hadoop 1.1 from the source code on centOS 6.3

1) prepare jdk
  1. Download oracle jdk compressed version for linux
  2. tar -xvf jdk-7u10-linux-x64.tar.gz into folder /usr/lib/jvm
  3. alternatives --install /usr/bin/java java /usr/lib/jvm/jdk1.7.0_10-sun/bin/java  99 --slave /usr/bin/keytool keytool /usr/lib/jvm/jdk1.7.0_10-sun/bin/keytool --slave /usr/bin/rmiregistry rmiregistry /usr/lib/jvm/jdk1.7.0_10-sun/rmiregistry
  4. alternatives --install /usr/bin/javac javac /usr/lib/jvm/jdk1.7.0_10-sun/bin/javac  99 --slave /usr/bin/jar jar /usr/lib/jvm/jdk1.7.0_10-sun/bin/jar --slave /usr/bin/rmic rmic /usr/lib/jvm/jdk1.7.0_10-sun/rmic
  5. alternatives --config java
  6. ln -s /usr/lib/jvm/jdk1.7.0_10-sun/ /usr/java/default (no need if we did not try oracle jdk rpm installer)

 2) prepare ant and ivy
  1. check ant installation: rpm -qa | grep  ant
  2. yum install ant
  3. download ivy from apache
  4. copy ivy jar files and dependent jars to /usr/share/ant/lib/

3) make sure automake is installed
  1. yum install automake

4) make sure libtool is installed
  1. yum install libtool

5) check out hadoop source code
 svn checkout http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1/

6) run ant command to build hadoop 1.1