VJia - Software Development

Blog, Code, Life

手动卸载Oracle

| Comments

直接使用安装完成后的Oracle自带的Uninstallation程序，也是可以部分完成我们所需的卸载功能，但是终究还是会遗留一部分需要手工参与的步骤。

而我今天所面临的问题是：2块硬盘，在上个礼拜其中一块硬盘罢工，幸好不是系统盘，免去了我要重装系统的麻烦；但麻烦的是损坏的F盘也是Oracle 11g所在磁盘，导致Oracle中服务及应用都不可使用。

虽然我手动将Oracle原有目录及文件（F:\app\user_name…）拷贝了回去到原位置，但Oracle已然不能使用。尝试了2种方式去重装，没有成功：

  • 运行Uninstallation，中止；

  • 运行installation，企图覆盖原有程序，中止。

看到有些文章，说可以通过手动删除Oracle注册表项操作，但是不太想通过此种方式，但是在看到stackoverflow上一篇帖子后，处于对他们的信任，还是使用这种方式：

Six-Step Process

  1. Delete the Oracle services: In the registry, go to \HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services and delete everything that starts with “Oracle”

  2. Remove other Oracle stuff from the registry: Go to \HKEY_LOCAL_MACHINE\SOFTWARE\ and delete the key ORACLE

  3. Reboot

  4. Delete all the Oracle software from the directories where you installed it

  5. Delete the Oracle software inventory: Delete the directory C:\Program Files\Oracle. You must do this no matter where you installed your Oracle software – the Oracle installer automatically writes information here.

  6. Delete all shortcuts from your Start menu.

参考链接： How do I a manual uninstall of Oracle

每年度阅读回顾

| Comments

每年度阅读回顾

在网络上一个空间中，看到了Amazon的CTO，Werner Vogels在他的[博客][1]中，列出了2012年读过的书籍和文章，这是一个Back to Basics的系列。

其他一些别人推荐或读过的书籍有：

  • <Linux/Unix设计思想> by Mike Gancarz

Linux/Unix设计思想

(本书适合入门，初步体会Linux/Unix的设计)

  • <认知盈余> by 克莱·舍基

认知盈余

  • <七周七语言 理解多种编程范式> by Bruce A. Tate

七周七语言

—

[1] http://www.allthingsdistributed.com/2012/12/paper-readings-2012.html “The Back-to-Basics Readings of 2012”

在百度BAE中安装Wordpress

| Comments

BAE

1) 参考文章，如何在百度BAE安装discuz论坛之全方位解析（图解）。

2) 在参考文章中描述的是如何安装BAE版Discuz，目前打算安装Wordpress。移步到百度BAE安装（WP）WordPress博客详细图文教程进行安装WP。

3) 安装完成，访问http://phpplay.duapp.com 总是报错“建立数据库连接时出错”；参考这里，导入了wpback.sql。继续报错。

http://mawenjian.net/p/994.html

Single-Node Setup of Hadoop

| Comments

1) 配置hadoop-env.sh

走到这一步时，是参照Michael-noll.com上的内容，但是需要配置$HADOOP_HOME/conf/hadoop-env.sh了，我不清楚他在写此篇博客时是基于Hadoop哪个版本，但是在我当前版本-2.0.5-alpha，并没有此目录；所以以下内容和记录转向参考Install Hadoop 2.0.1-alpha Yarn Next-Gen.

1
2
3
4
5
6

hduser@ubuntu:~/hadoop-2.0.5-alpha/etc/hadoop$ hadoop version
Hadoop 2.0.5-alpha
Subversion http://svn.apache.org/repos/asf/hadoop/common -r 1488459
Compiled by jenkins on 2013-06-01T04:05Z
From source with checksum c8f4bd45ac25c31b815f311b32ef17
This command was run using /home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/hadoop-common-2.0.5-alpha.jar

2) 需要安装Tarball么，先不安装，待定。

3) 配置环境变量

.bashrc:

1
2
3
4
5
6
7
8
9
10
11
12

// configure follow raseshmori.wordpress.com
HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_MAPRED_HOME

HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME

HADOOP_HDFS_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME

HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export HADOOP_HDFS_HOME

4) 创建2个目录，用于存储namenode和datanode的数据

1
2
3
4
5
6
7
8
9
10
11
12
13

hduser@ubuntu:~$ pwd
/home/hduser
hduser@ubuntu:~$ ls
bak.of.bashrc  examples.desktop  hadoop-2.0.5-alpha  hadoop-2.0.5-alpha.tar.gz
hduser@ubuntu:~$ mkdir hadoop_data
hduser@ubuntu:~$ cd hadoop_data/
hduser@ubuntu:~/hadoop_data$ ls
hduser@ubuntu:~/hadoop_data$ mkdir hdfs
hduser@ubuntu:~/hadoop_data$ cd hdfs/
hduser@ubuntu:~/hadoop_data/hdfs$ mkdir namenode
hduser@ubuntu:~/hadoop_data/hdfs$ mkdir datanode
hduser@ubuntu:~/hadoop_data/hdfs$ pwd
/home/hduser/hadoop_data/hdfs

5) 设置Properties在Config文件中：

  • yarn-site.xml ($HADOOP_HOME/etc/hadoop)
1
2
3
4
5
6
7
8
9
10

// Site specifc YARN configuration properties
<property>
  <name>yarn.nodemanager.aux-services</name>
  <value>mapreduce.shuffle</value>
</property>

<property>
  <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
  <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
  • core-site.xml ($HADOOP_HOME/etc/hadoop)
1
2
3
4
5
6
7
8

<configuration>

 <property>
    <name>fs.default.name</name>
    <value>hdfs://localhost:9000</value>
 </property>

</configuration>
  • hdfs-site.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

<configuration>

 <property>
    <name>dfs.replication</name>
    <value>1</value>
 </property>

 <property>
   <name>dfs.namenode.name.dir</name>
   <value>file:/home/hduser/hadoop_data/hdfs/namenode</value>
 </property>

 <property>
   <name>dfs.datanode.data.dir</name>
   <value>file:/home/hduser/hadoop_data/hdfs/datanode</value>
 </property>
</configuration>
  • mapred-site.xml
1
2
3
4
5
6

<configuration>
 <property>
   <name>mapreduce.framework.name</name>
   <value>yarn</value>
 </property>
</configuration>

6) 格式化namenode

This step is needed only for the first time. Doing it every time will result in loss of content on HDFS.

注意1 > 1.1注意 > 1.2注意

hduser@ubuntu:~/hadoop-2.0.5-alpha/bin$ ./hadoop namenode -format

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

hduser@ubuntu:~/hadoop-2.0.5-alpha/bin$ ./hadoop namenode -format
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

13/07/31 20:42:11 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = ubuntu/127.0.1.1
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 2.0.5-alpha
STARTUP_MSG:   classpath = /home/hduser/hadoop-2.0.5-alpha/etc/hadoop:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/commons-configuration-1.6.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jettison-1.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/snappy-java-1.0.3.2.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/commons-cli-1.2.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jsp-api-2.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/commons-httpclient-3.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/xmlenc-0.52.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/guava-11.0.2.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/protobuf-java-2.4.0a.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/commons-el-1.0.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jetty-6.1.26.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jets3t-0.6.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/asm-3.2.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/zookeeper-3.4.2.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/commons-digester-1.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/avro-1.5.3.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jersey-server-1.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/commons-math-2.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/activation-1.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/commons-lang-2.5.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/commons-collections-3.2.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/paranamer-2.3.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/servlet-api-2.5.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jsch-0.1.42.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/commons-io-2.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jersey-core-1.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/commons-logging-1.1.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/commons-net-3.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jsr305-1.3.9.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/log4j-1.2.17.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/hadoop-auth-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jetty-util-6.1.26.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jersey-json-1.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/stax-api-1.0.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/commons-codec-1.4.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/hadoop-annotations-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/kfs-0.3.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/lib/slf4j-api-1.6.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/hadoop-common-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/common/hadoop-common-2.0.5-alpha-tests.jar:/home/hduser/hadoop-2.0.5-alpha/contrib/capacity-scheduler/*.jar:/home/hduser/hadoop-2.0.5-alpha/contrib/capacity-scheduler/*.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/guava-11.0.2.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/protobuf-java-2.4.0a.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/commons-el-1.0.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/asm-3.2.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/jersey-server-1.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/commons-lang-2.5.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/commons-io-2.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/jersey-core-1.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/commons-logging-1.1.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/hadoop-hdfs-2.0.5-alpha-tests.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/hdfs/hadoop-hdfs-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/snappy-java-1.0.3.2.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/aopalliance-1.0.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/protobuf-java-2.4.0a.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/netty-3.5.11.Final.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/asm-3.2.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/avro-1.5.3.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/junit-4.8.2.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/jersey-server-1.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/paranamer-2.3.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/commons-io-2.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/jersey-core-1.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/log4j-1.2.17.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/jersey-guice-1.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/guice-3.0.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/javax.inject-1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/lib/hadoop-annotations-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/hadoop-yarn-client-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/hadoop-yarn-site-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/hadoop-yarn-server-tests-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/hadoop-yarn-api-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/hadoop-yarn-server-common-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/hadoop-yarn-common-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/snappy-java-1.0.3.2.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/protobuf-java-2.4.0a.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/netty-3.5.11.Final.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/asm-3.2.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/avro-1.5.3.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/junit-4.8.2.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/jersey-server-1.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/commons-io-2.1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/jersey-core-1.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/jersey-guice-1.8.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/guice-3.0.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/javax.inject-1.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/lib/hadoop-annotations-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.5-alpha-tests.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.5-alpha.jar:/home/hduser/hadoop-2.0.5-alpha/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.0.5-alpha.jar
STARTUP_MSG:   build = http://svn.apache.org/repos/asf/hadoop/common -r 1488459; compiled by 'jenkins' on 2013-06-01T04:05Z
STARTUP_MSG:   java = 1.7.0_25
************************************************************/
Formatting using clusterid: CID-dcd1e06d-a0f7-450c-b5c8-9f81f40e0114
13/07/31 20:42:12 INFO util.HostsFileReader: Refreshing hosts (include/exclude) list
13/07/31 20:42:12 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000
13/07/31 20:42:13 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false
13/07/31 20:42:13 INFO blockmanagement.BlockManager: defaultReplication         = 1
13/07/31 20:42:13 INFO blockmanagement.BlockManager: maxReplication             = 512
13/07/31 20:42:13 INFO blockmanagement.BlockManager: minReplication             = 1
13/07/31 20:42:13 INFO blockmanagement.BlockManager: maxReplicationStreams      = 2
13/07/31 20:42:13 INFO blockmanagement.BlockManager: shouldCheckForEnoughRacks  = false
13/07/31 20:42:13 INFO blockmanagement.BlockManager: replicationRecheckInterval = 3000
13/07/31 20:42:13 INFO blockmanagement.BlockManager: encryptDataTransfer        = false
13/07/31 20:42:13 INFO namenode.FSNamesystem: fsOwner             = hduser (auth:SIMPLE)
13/07/31 20:42:13 INFO namenode.FSNamesystem: supergroup          = supergroup
13/07/31 20:42:13 INFO namenode.FSNamesystem: isPermissionEnabled = true
13/07/31 20:42:13 INFO namenode.FSNamesystem: HA Enabled: false
13/07/31 20:42:13 INFO namenode.FSNamesystem: Append Enabled: true
13/07/31 20:42:13 INFO namenode.NameNode: Caching file names occuring more than 10 times
13/07/31 20:42:13 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
13/07/31 20:42:13 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0
13/07/31 20:42:13 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension     = 30000
Re-format filesystem in Storage Directory /home/hduser/hadoop_data/hdfs/namenode ? (Y or N) y
13/07/31 20:42:19 INFO common.Storage: Storage directory /home/hduser/hadoop_data/hdfs/namenode has been successfully formatted.
13/07/31 20:42:19 INFO namenode.FSImage: Saving image file /home/hduser/hadoop_data/hdfs/namenode/current/fsimage.ckpt_0000000000000000000 using no compression
13/07/31 20:42:19 INFO namenode.FSImage: Image file of size 121 saved in 0 seconds.
13/07/31 20:42:19 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
13/07/31 20:42:19 INFO util.ExitUtil: Exiting with status 0
13/07/31 20:42:19 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1
************************************************************/

7) start HDFS processes

Name node

1
2
3
4
5
6
7
8
9
10

hduser@ubuntu:~/hadoop-2.0.5-alpha/sbin$ ./hadoop-daemon
hadoop-daemon.sh   hadoop-daemons.sh  
hduser@ubuntu:~/hadoop-2.0.5-alpha/sbin$ ./hadoop-daemon.sh  start namenode
starting namenode, logging to /home/hduser/hadoop-2.0.5-alpha/logs/hadoop-hduser-namenode-ubuntu.out
hduser@ubuntu:~/hadoop-2.0.5-alpha/sbin$ jps
The program 'jps' can be found in the following packages:
 * openjdk-6-jdk
 * openjdk-7-jdk
Ask your administrator to install one of them
// 不清楚jps命令是什么，但是这里应该对安装没有影响，已经安装了Oracle JDK 1.7

Data node

1
2

hduser@ubuntu:~/hadoop-2.0.5-alpha/sbin$ ./hadoop-daemon.sh start datanode
starting datanode, logging to /home/hduser/hadoop-2.0.5-alpha/logs/hadoop-hduser-datanode-ubuntu.out

8) start Hadoop Map-Reduce Processes

Resource Manager

1
2

hduser@ubuntu:~/hadoop-2.0.5-alpha$ sbin/yarn-daemon.sh start resourcemanager
starting resourcemanager, logging to /home/hduser/hadoop-2.0.5-alpha/logs/yarn-hduser-resourcemanager-ubuntu.out

Node Manager

1
2

hduser@ubuntu:~/hadoop-2.0.5-alpha$ sbin/yarn-daemon.sh start nodemanager
starting nodemanager, logging to /home/hduser/hadoop-2.0.5-alpha/logs/yarn-hduser-nodemanager-ubuntu.out

Job History Server

1
2

hduser@ubuntu:~/hadoop-2.0.5-alpha$ sbin/mr-jobhistory-daemon.sh start historyserver
starting historyserver, logging to /home/hduser/hadoop-2.0.5-alpha/logs/mapred-hduser-historyserver-ubuntu.out

9) 运行一个简单/经典的字符统计示例，来验证安装是否成功

1
2
3
4

hduser@ubuntu:~/hadoop-2.0.5-alpha$ mkdir in
hduser@ubuntu:~/hadoop-2.0.5-alpha$ cat > in/file    //这个命令不错
This is one line
This is another one

Add this directory to HDFS:

1
2
3
4
5

hduser@ubuntu:~/hadoop-2.0.5-alpha$ bin/hadoop dfs -copyFromLocal in /in
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
// note: dfs命令已经被弃用，应用fs命令，见[fs replace dfs](http://stackoverflow.com/questions/11715082/why-is-it-keep-showing-deprecated-error-when-running-hadoop-or-dfs-command)
hduser@ubuntu:~/hadoop-2.0.5-alpha$ ./bin/hadoop fs -copyFromLocal in /in

运行share目录中提供的wordcount示例：

hduser@ubuntu:~/hadoop-2.0.5-alpha/logs$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.0.5-alpha.jar wordcount /in /out3

to continue…

10) 运行netstat命令，查看是否Hadoop监听在已经配置的端口

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

hduser@ubuntu:~/hadoop-2.0.5-alpha/logs$ sudo netstat -plten | grep java
[sudo] password for hduser: 
tcp        0      0 0.0.0.0:19888           0.0.0.0:*               LISTEN      1001       33451       6931/java       
tcp        0      0 0.0.0.0:50070           0.0.0.0:*               LISTEN      1001       31961       6436/java       
tcp        0      0 0.0.0.0:50010           0.0.0.0:*               LISTEN      1001       32345       6510/java       
tcp        0      0 0.0.0.0:50075           0.0.0.0:*               LISTEN      1001       32352       6510/java       
tcp        0      0 0.0.0.0:10020           0.0.0.0:*               LISTEN      1001       33459       6931/java       
tcp        0      0 0.0.0.0:50020           0.0.0.0:*               LISTEN      1001       32358       6510/java       
tcp        0      0 127.0.0.1:9000          0.0.0.0:*               LISTEN      1001       31785       6436/java       
tcp6       0      0 :::8042                 :::*                    LISTEN      1001       33369       6825/java       
tcp6       0      0 :::8080                 :::*                    LISTEN      1001       33367       6825/java       
tcp6       0      0 :::8088                 :::*                    LISTEN      1001       32771       6594/java       
tcp6       0      0 :::8030                 :::*                    LISTEN      1001       32823       6594/java       
tcp6       0      0 :::8031                 :::*                    LISTEN      1001       32779       6594/java       
tcp6       0      0 :::8032                 :::*                    LISTEN      1001       32831       6594/java       
tcp6       0      0 :::8033                 :::*                    LISTEN      1001       32843       6594/java       
tcp6       0      0 :::8040                 :::*                    LISTEN      1001       33363       6825/java       
tcp6       0      0 :::37896                :::*                    LISTEN      1001       33353       6825/java       
hduser@ubuntu:~/hadoop-2.0.5-alpha/logs$

11) Web界面

  • 查看HDFS及其health信息

http://132.253.222.240:50070 或者 http://132.253.222.240:50070/dfshealth.jsp

  • 查看Hadoop All Applications：

http://132.253.222.240:8088 或者 http://132.253.222.240:8088/cluster

(这里的IP请使用localhost或者自己的对应IP，我所使用的是Ubuntu CMD)

待解问题：

  • what’s YARN ?

  • what’s jps ?