想了解hadoop,所以就下了最新的文件,搭建相关的环境,以下为自己在win7上搭建hadoop的相关步骤。
1、下载hadoop,用winRAR解压。(路径不能有空格)
2、安装java,此处就不做讲解了。(java路径不能有空格)
3、配置hadoop的环境变量HADOOP_HOME=”HADOOP的安装目录”
4、在环境变量的PATH中加入HADOOP的安装目录/bin
5、修改hadoop目录下etc/hadoop/hadoop-env.cmd文件中的JAVA_HOME变量为当前java的安装路径。
6、下载,因为在windows下运行,压缩包里面缺少 winutils.exe, hadoop.dll等文件,下载完成后,将要报下面的bin目录下的所有文件全部拷贝到hadoop目录下的bin文件夹下。
7、修改配置文件,以下四个文件全部在hadoop目录下的etc/hadoop目录下
修改core-site.xml,如下:
fs.defaultFS hdfs://localhost:9000
修改hdfs-site.xml如下:
dfs.replication 1 dfs.namenode.name.dir file:/hadoop/data/dfs/namenode dfs.datanode.data.dir file:/hadoop/data/dfs/datanode
修改yarn-site.xml如下:
yarn.nodemanager.aux-services mapreduce_shuffle yarn.nodemanager.aux-services.mapreduce.shuffle.class org.apache.hadoop.mapred.ShuffleHandler
修改mapred-site.xml如下:
mapreduce.framework.name yarn
然后打开cmd,运行hadoop namenode -format命令,运行结果基本如下:
Microsoft Windows [Version 6.1.7601]Copyright (c) 2009 Microsoft Corporation. All rights reserved. C:\Users\abhijitg>cd c:\hadoop\bin c:\hadoop\bin>hdfs namenode -format13/11/03 18:07:47 INFO namenode.NameNode: STARTUP_MSG:/************************************************************STARTUP_MSG: Starting NameNodeSTARTUP_MSG: host = ABHIJITG/x.x.x.xSTARTUP_MSG: args = [-format]STARTUP_MSG: version = 2.2.0STARTUP_MSG: classpath =STARTUP_MSG: build = Unknown -r Unknown; compiled by ABHIJITG on 2013-11-01T13:42ZSTARTUP_MSG: java = 1.7.0_03************************************************************/Formatting using clusterid: CID-1af0bd9f-efee-4d4e-9f03-a0032c22e5eb13/11/03 18:07:48 INFO namenode.HostFileManager: read includes:HostSet()13/11/03 18:07:48 INFO namenode.HostFileManager: read excludes:HostSet()13/11/03 18:07:48 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=100013/11/03 18:07:48 INFO util.GSet: Computing capacity for map BlocksMap13/11/03 18:07:48 INFO util.GSet: VM type = 64-bit13/11/03 18:07:48 INFO util.GSet: 2.0% max memory = 888.9 MB13/11/03 18:07:48 INFO util.GSet: capacity = 2^21 = 2097152 entries13/11/03 18:07:48 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false13/11/03 18:07:48 INFO blockmanagement.BlockManager: defaultReplication = 113/11/03 18:07:48 INFO blockmanagement.BlockManager: maxReplication = 51213/11/03 18:07:48 INFO blockmanagement.BlockManager: minReplication = 113/11/03 18:07:48 INFO blockmanagement.BlockManager: maxReplicationStreams = 213/11/03 18:07:48 INFO blockmanagement.BlockManager: shouldCheckForEnoughRacks = false13/11/03 18:07:48 INFO blockmanagement.BlockManager: replicationRecheckInterval = 300013/11/03 18:07:48 INFO blockmanagement.BlockManager: encryptDataTransfer = false13/11/03 18:07:48 INFO namenode.FSNamesystem: fsOwner = ABHIJITG (auth:SIMPLE)13/11/03 18:07:48 INFO namenode.FSNamesystem: supergroup = supergroup13/11/03 18:07:48 INFO namenode.FSNamesystem: isPermissionEnabled = true13/11/03 18:07:48 INFO namenode.FSNamesystem: HA Enabled: false13/11/03 18:07:48 INFO namenode.FSNamesystem: Append Enabled: true13/11/03 18:07:49 INFO util.GSet: Computing capacity for map INodeMap13/11/03 18:07:49 INFO util.GSet: VM type = 64-bit13/11/03 18:07:49 INFO util.GSet: 1.0% max memory = 888.9 MB13/11/03 18:07:49 INFO util.GSet: capacity = 2^20 = 1048576 entries13/11/03 18:07:49 INFO namenode.NameNode: Caching file names occuring more than 10 times13/11/03 18:07:49 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.999000012874603313/11/03 18:07:49 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 013/11/03 18:07:49 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension = 3000013/11/03 18:07:49 INFO namenode.FSNamesystem: Retry cache on namenode is enabled13/11/03 18:07:49 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry timeis 600000 millis13/11/03 18:07:49 INFO util.GSet: Computing capacity for map Namenode Retry Cache13/11/03 18:07:49 INFO util.GSet: VM type = 64-bit13/11/03 18:07:49 INFO util.GSet: 0.029999999329447746% max memory = 888.9 MB13/11/03 18:07:49 INFO util.GSet: capacity = 2^15 = 32768 entries13/11/03 18:07:49 INFO common.Storage: Storage directory \hadoop\data\dfs\namenode has been successfully formatted.13/11/03 18:07:49 INFO namenode.FSImage: Saving image file \hadoop\data\dfs\namenode\current\fsimage.ckpt_0000000000000000000 using no compression13/11/03 18:07:49 INFO namenode.FSImage: Image file \hadoop\data\dfs\namenode\current\fsimage.ckpt_0000000000000000000 of size 200 bytes saved in 0 seconds.13/11/03 18:07:49 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 013/11/03 18:07:49 INFO util.ExitUtil: Exiting with status 013/11/03 18:07:49 INFO namenode.NameNode: SHUTDOWN_MSG:/************************************************************SHUTDOWN_MSG: Shutting down NameNode at ABHIJITG/x.x.x.x************************************************************/
然后在cmd下切换目录到hadoop目录下的sbin目录下,运行start-all 会打开四个cmd窗口,可以打开浏览器输入 以及
如果50070打不开,就重新格式化,然后重启hadoop就可以了