Wednesday, January 23, 2013

setup hadoop development environment in eclipse

  1. Finding the version of Maven from maven homepage and running the wget command from the dir you want to extract maven too.
  2. untar the file: tar xvf apache-maven-3.0.4-bin.tar.gz
  3. Next add the env variables to your ~/.bashrc file
    export M2_HOME=/usr/local/apache-maven/apache-maven-3.0.4 
    export M2=$M2_HOME/bin 
    export PATH=$M2:$PATH 
  4. Verify everything is working with the following command
     mvn -version
  5. make sure g++ is installed
  6. download and install protocol buffer compiler and install protocol buffer java library in mvn repository
    export MAVEN_OPTS='-Xms384M -Xmx512M -XX:MaxPermSize=256M'
  7. Fix classpath variable error in Eclipse if needed.
    Open the Eclipse Preferences: Window -> Preferences
    Go to [Java - Build Path - Classpath Variables]
    Click New and set its name as M2_REPO
    Click Folder and select your Maven repository folder. For example, /home/yiyujia/.m2/repository
    A little bit more about maven environment settting.
  8. Treat hadoop programming as normal maven project. Adding hadoop dependencies, coding and testing.
Note: "mvn eclipse:eclipse" is not needed for new version eclipse.

No comments:

Post a Comment