win7搭建为伪分布式hadoop的步骤

这篇文章主要介绍“win7搭建为伪分布式hadoop的步骤”，在日常操作中，相信很多人在win7搭建为伪分布式hadoop的步骤问题上存在疑惑，小编查阅了各式资料，整理出简单好用的操作方法，希望对大家解答”win7搭建为伪分布式hadoop的步骤”的疑惑有所帮助！接下来，请跟着小编一起来学习吧！

创新互联公司是一家专注于成都网站设计、网站建设与策划设计,温县网站建设哪家好?创新互联公司做网站,专注于网站建设十载,网设计领域的专业建站公司;建站业务涵盖:温县等地区。温县做网站价格咨询:18982081108

window7下安装hadoop [32位]
1.下载hadoop
hadoop-2.2.0.tar.gz
解压
2.配置环境变量hadoop
3.修改%HADOOP_HOME%\etc\hadoop的hadoop-env.sh
export JAVA_HOME=D:\Java\jdk1.6.0_10#不能有空格
末尾添加[会出错就不添加]
set HADOOP_PREFIX=E:\hadoop\hadoop-2.2.0\
set HADOOP_CONF_DIR=%HADOOP_PREFIX%\etc\hadoop
set YARN_CONF_DIR=%HADOOP_CONF_DIR%
set PATH=%PATH%;%HADOOP_PREFIX%\bin

下载hadoop-common-2.2.0-bin-master

把里面的文件拷贝到hadoop/bin下

4.配置core-site.xml


        fs.default.name
        hdfs://0.0.0.0:19000


5.配置hdfs-site.xml



        dfs.replication
        1




6.配置mapred-site.xml




         mapreduce.job.user.name
         %USERNAME%



         mapreduce.framework.name
         yarn



        yarn.apps.stagingDir
        /user/%USERNAME%/staging



        mapreduce.jobtracker.address
        local




7.创建yarn-site.xml文件



        yarn.server.resourcemanager.address
        0.0.0.0:8020



        yarn.server.resourcemanager.application.expiry.interval
        60000



        yarn.server.nodemanager.address
        0.0.0.0:45454



        yarn.nodemanager.aux-services
        mapreduce_shuffle



        yarn.nodemanager.aux-services.mapreduce.shuffle.class
        org.apache.hadoop.mapred.ShuffleHandler



        yarn.server.nodemanager.remote-app-log-dir
        /app-logs



        yarn.nodemanager.log-dirs
        /dep/logs/userlogs



        yarn.server.mapreduce-appmanager.attempt-listener.bindAddress
        0.0.0.0



        yarn.server.mapreduce-appmanager.client-service.bindAddress
        0.0.0.0



        yarn.log-aggregation-enable
        true



        yarn.log-aggregation.retain-seconds
        -1



        yarn.application.classpath
        %HADOOP_CONF_DIR%,%HADOOP_COMMON_HOME%/share/hadoop/common/*,%HADOOP_COMMON_HOME%/share/hadoop/common/lib/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/lib/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/lib/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/lib/*



    切换到E:\hadoop\hadoop-2.2.0\etc\hadoop运行hadoop-env.cmd脚本，设置当前命令窗口执行环境变量
   执行 hdfs namenode -format
   16/02/27 11:36:23 INFO util.GSet: VM type       = 32-bit
16/02/27 11:36:23 INFO util.GSet: 0.029999999329447746% max memory = 992.3 MB
16/02/27 11:36:23 INFO util.GSet: capacity      = 2^16 = 65536 entries
16/02/27 11:36:24 INFO common.Storage: Storage directory \tmp\hadoop-goudcheng\d
fs\name has been successfully formatted.
16/02/27 11:36:24 INFO namenode.FSImage: Saving image file \tmp\hadoop-goudcheng
\dfs\name\current\fsimage.ckpt_0000000000000000000 using no compression
16/02/27 11:36:24 INFO namenode.FSImage: Image file \tmp\hadoop-goudcheng\dfs\na
me\current\fsimage.ckpt_0000000000000000000 of size 201 bytes saved in 0 seconds
.
16/02/27 11:36:24 INFO namenode.NNStorageRetentionManager: Going to retain 1 ima
ges with txid >= 0
16/02/27 11:36:24 INFO util.ExitUtil: Exiting with status 0
16/02/27 11:36:24 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at CDCH20100020-5/172.31.168.244

到此，关于“win7搭建为伪分布式hadoop的步骤”的学习就结束了，希望能够解决大家的疑惑。理论与实践的搭配能更好的帮助大家学习，快去试试吧！若想继续学习更多相关知识，请继续关注创新互联网站，小编会继续努力为大家带来更多实用的文章！

本文标题：win7搭建为伪分布式hadoop的步骤
路径分享：http://cdxtjz.cn/article/iiohoe.html

win7搭建为伪分布式hadoop的步骤

其他资讯