Hadoop启动没有datanode进程解决⽅法
Hadoop 启动没有 datanode 进程解决⽅法
2019-01-22 22:101930原创
本⽂链接：
1 现象说明
新搭建的hadoop 3.1.1 的环境，在启动Hadoop时，通过jps⽬录发现Slave上没有datanode进程。如下：
[cndba@hadoopmaster ~]$ jps
23234 ResourceManager
22998 SecondaryNameNode
23575 Jps
22683 NameNode
[cndba@hadoopslave1 ~]$ jps
9682 Jps
9535 NodeManager
[cndba@hadoopslave2 ~]$ jps
9356 Jps
9199 NodeManager
2 clusterID不匹配导致的问题
⽹上搜了下，⽹上的说法都是由于进⾏hadoop格式化的时候没有事先结束所有进程，或者多次进⾏了format导致的datanode的clusterID 和namenode的clusterID不匹配，从⽽在启动后没有datanode进程。
解决⽅法有两种：
⽅案⼀: 保留现有数据
1. ⽤NameNode节点的~/dfs/name/current/VERSION 中的namenode的clusterID替换所有datanode节点机器中
~/dfs/data/current/VERSION中的clusterID。
2. 重启启动hadoop:start-all.sh
这种⽅式不影响现有的数据，避免了重新的格式化。
⽅案⼆: 重新格式化
1. 执⾏./stop-all.sh关闭集
2. 删除存放hdfs数据块的⽂件夹(hadoop/tmp/)，然后重建该⽂件夹
3. 删除hadoop下的⽇志⽂件logs
4. 执⾏hadoop namenode -format格式化hadoop
5. 重启hadoop集
3 其他情况
我这⾥属于另外的情况，并不是clusterID不匹配导致的问题。
重新查看了下启动⽇志，原来是⽤户名敲错了：
[cndba@hadoopmaster hadoop]$ start-all.sh
WARNING: Attempting to start all Apache Hadoop daemons as cndba in 10 seconds.
WARNING: This is not a recommended production deployment configuration.
WARNING: Use CTRL-C to abort.
Starting namenodes on [hadoopmaster]
Starting datanodes
ERROR: datanode can only be executed by cbdba.
Starting secondary namenodes [hadoopmaster]
WARNING: YARN_CONF_DIR has been replaced by HADOOP_CONF_DIR. Using value of YARN_CONF_DIR.
Starting resourcemanager
WARNING: YARN_CONF_DIR has been replaced by HADOOP_CONF_DIR. Using value of YARN_CONF_DIR.
Starting nodemanagers
WARNING: YARN_CONF_DIR has been replaced by HADOOP_CONF_DIR. Using value of YARN_CONF_DIR.
[cndba@hadoopmaster hadoop]$
hadoop-env.sh ⽂件中敲成了cbdba：
export HDFS_DATANODE_USER="cbdba"
修改成cndba后继续启动：
[cndba@hadoopmaster hadoop]$ start-all.sh
WARNING: Attempting to start all Apache Hadoop daemons as cndba in 10 seconds.
WARNING: This is not a recommended production deployment configuration.
WARNING: Use CTRL-C to abort.
Starting namenodes on [hadoopmaster]
Starting datanodes
hadoopslave2: ERROR: Cannot set priority of datanode process 12752
hadoopslave1: ERROR: Cannot set priority of datanode process 13164
Starting secondary namenodes [hadoopmaster]
WARNING: YARN_CONF_DIR has been replaced by HADOOP_CONF_DIR. Using value of YARN_CONF_DIR.
Starting resourcemanager
WARNING: YARN_CONF_DIR has been replaced by HADOOP_CONF_DIR. Using value of YARN_CONF_DIR.
Starting nodemanagers
WARNING: YARN_CONF_DIR has been replaced by HADOOP_CONF_DIR. Using value of YARN_CONF_DIR.
[cndba@hadoopmaster hadoop]$
⼜报错：
hadoopslave2: ERROR: Cannot set priority of datanode process 12752
上从库查看datanode⽇志：
************************************************************/
2019-01-23 05:23:23,501 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: registered UNIX signal handlers for [TERM, HUP, INT] 2019-01-23 05:23:23,619 ERROR org.f.Configuration: error parsing l
at [row,col,system-id]: [50,17,"file:/home/cndba/hadoop/etc/l"]
wstx.sr.StreamScanner.throwUnexpectedChar(StreamScanner.java:653)
wstx.FromProlog(BasicStreamReader.java:2133)
wstx.sr.BasicStreamReader.closeContentTree(BasicStreamReader.java:2991)
wstx.FromTree(BasicStreamReader.java:2734)
这⾥表⾯上看是权限问题，但从⽇志看是l 配置⽂件有问题，修改配置⽂件后重启系统，正常：[cndba@hadoopmaster hadoop]$ start-all.sh
confWARNING: Attempting to start all Apache Hadoop daemons as cndba in 10 seconds.
WARNING: This is not a recommended production deployment configuration.
WARNING: Use CTRL-C to abort.
Starting namenodes on [hadoopmaster]
Starting datanodes
Starting secondary namenodes [hadoopmaster]
WARNING: YARN_CONF_DIR has been replaced by HADOOP_CONF_DIR. Using value of YARN_CONF_DIR.
Starting resourcemanager
WARNING: YARN_CONF_DIR has been replaced by HADOOP_CONF_DIR. Using value of YARN_CONF_DIR.
Starting nodemanagers
WARNING: YARN_CONF_DIR has been replaced by HADOOP_CONF_DIR. Using value of YARN_CONF_DIR.
[cndba@hadoopmaster hadoop]$
查看进程，也没有问题：
[cndba@hadoopmaster hadoop]$ jps
13030 SecondaryNameNode
12791 NameNode
13271 ResourceManager
13752 Jps
[cndba@hadoopmaster hadoop]$
[cndba@hadoopslave2 logs]$ jps
13587 Jps
13302 DataNode
13422 NodeManager
[cndba@hadoopslave2 logs]$
[root@hadoopslave1 ~]# jps
13876 NodeManager
14026 Jps
13756 DataNode
[root@hadoopslave1 ~]#

版权声明：本站内容均来自互联网，仅供演示用，请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198，我们将在24小时内删除。