hadoop-1.2.1 伪分布配置

首先JDK安装及相关环境变量配置

# Java environment setting
JAVA_HOME=/usr/java/default
CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar
PATH=$JAVA_HOME/bin:$PATH
JRE_HOME=$JAVA_HOME/jre
export JAVA_HOME CLASSPATH PATH JRE_HOME

# Maven configuration
M2_HOME=/usr/local/apache-maven/default
PATH=$PATH:$M2_HOME/bin

# hadoop setting
HADOOP_HOME=/home/asn/opt/hadoop-1.2.
PATH=$PATH:$HADOOP_HOME/bin

1. 修改主机名

[asn@vm-centos6-asn opt]$ cat /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=vm-centos6-asn

在/etc/hosts文件中，添加主机名映射

[root@vm-centos6-asn ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 vm-centos6-asn
::         localhost localhost.localdomain localhost6 localhost6.localdomain6

2. hadoop配置

1）core-site.xml

 <?xml version="1.0"?>
 <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

 <!-- Put site-specific property overrides in this file. -->

 <configuration>
     <property>
         <name>fs.default.name</name>
         <value>hdfs://192.168.56.101:9000</value>
     </property>

     <property>
         <name>hadoop.tmp.dir</name>
         <value>/home/asn/opt/hadoop-tmp</value>
     </property>
 </configuration>

2) hdfs-site.xml

 <?xml version="1.0"?>
 <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

 <!-- Put site-specific property overrides in this file. -->

 <configuration>
     <property>
         <name>dfs.data.dir</name>
         <value>/home/asn/opt/hadoop-data</value>
     </property>

     <property>
         <name>dfs.replication</name>
         <value>1</value>
     </property>
 </configuration>

3) mapred-site.xml

 <?xml version="1.0"?>
 <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

 <!-- Put site-specific property overrides in this file. -->

 <configuration>

     <property>
         <name>mapred.job.tracker</name>
         <value>192.168.56.101:9001</value>
     </property>
 </configuration>

4)  设置hadoop-env.sh文件中的JAVA_HOME环境变量

[asn@vm-centos6-asn conf]$ cat hadoop-env.sh
# Set Hadoop-specific environment variables here.

JAVA_HOME环境变量必须配置，其他环境变量可选。 当运行一个分布式配置时，最好配置该文件中的JAVA_HOME环境变量，以便于在远程节点上被正确定义。
# The only required environment variable is JAVA_HOME.  All others are optional.
# When running a distributed configuration it is best to
# set JAVA_HOME in this file, so that it is correctly defined on remote nodes.

# The java implementation to use.  Required.
export JAVA_HOME=/usr/java/default

# Extra Java CLASSPATH elements.  Optional.
# export HADOOP_CLASSPATH=

# The maximum amount of heap to use, in MB. Default is .
# export HADOOP_HEAPSIZE=

# Extra Java runtime options.  Empty by default.
# export HADOOP_OPTS=-server

3. 免密码登陆ssh配置

[asn@vm-centos6-asn ~]$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/asn/.ssh/id_rsa):   私钥存放文件
Created directory '/home/asn/.ssh'.
Enter passphrase (empty for no passphrase):    直接回车
Enter same passphrase again:  直接回车
Your identification has been saved in /home/asn/.ssh/id_rsa.
Your public key has been saved in /home/asn/.ssh/id_rsa.pub.   公钥存放文件
The key fingerprint is:
6e::e0:::b5:6b:fb:::::1e:b2:8f: asn@vm-centos6-asn
The key's randomart image is:
+--[ RSA ]----+
|  ...     ..oo=  |
|   . .   .   = . |
|  . .   .   . . E|
|   . . .     o . |
|    = o S   . o  |
|   . = =     .   |
|    . + +        |
|     . +         |
|      .          |
+-----------------+
[asn@vm-centos6-asn ~]$ cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys   把公钥拷贝到认证文件中

[asn@vm-centos6-asn ~]$ ssh localhost
The authenticity of host 'localhost (::1)' can't be established.
RSA key fingerprint is bd:c8:f0:::be:ee:2f::6d::::d7:c0:.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (RSA) to the list of known hosts.  首次连接提示是否把localhost添加到已知的主机列表中
Last login: Tue Dec  ::  from localhost
[asn@vm-centos6-asn ~]$ exit
logout
Connection to localhost closed.

[asn@vm-centos6-asn ~]$ ssh localhost      直接免密码登录成功
Last login: Tue Dec  ::  from localhost
[asn@vm-centos6-asn ~]$ 

4. hadoop  namenode -format  格式化名字节点

5. start-all.sh 启动

[asn@vm-centos6-asn ~]$ start-all.sh
starting namenode, logging to /home/asn/opt/hadoop-1.2./libexec/../logs/hadoop-asn-namenode-vm-centos6-asn.out
localhost: starting datanode, logging to /home/asn/opt/hadoop-1.2./libexec/../logs/hadoop-asn-datanode-vm-centos6-asn.out
localhost: starting secondarynamenode, logging to /home/asn/opt/hadoop-1.2./libexec/../logs/hadoop-asn-secondarynamenode-vm-centos6-asn.out
starting jobtracker, logging to /home/asn/opt/hadoop-1.2./libexec/../logs/hadoop-asn-jobtracker-vm-centos6-asn.out
localhost: starting tasktracker, logging to /home/asn/opt/hadoop-1.2./libexec/../logs/hadoop-asn-tasktracker-vm-centos6-asn.out

[asn@vm-centos6-asn ~]$ jps
 Jps
 NameNode
 SecondaryNameNode
 JobTracker
 TaskTracker
 DataNode