数据节点 Hadoop 2.7.3 单节点(伪分布式模式)中出现错误



我对hadoop很陌生,我已经回答了我之前的问题,但是评论部分太短了,无法显示我的日志文件。这是我之前的问题.谁能帮我找到这个日志文件中的错误?我将不胜感激。谢谢。

STARTUP_MSG:   build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r baa91f7c6bc9cb92be5982de4719c1c8af91ccff; compiled by 'root' on 2016-08-18T01:41Z
STARTUP_MSG:   java = 1.8.0_121
************************************************************/
2017-03-27 16:14:50,262 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: registered UNIX signal handlers for [TERM, HUP, INT]
2017-03-27 16:14:51,049 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2017-03-27 16:14:51,131 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2017-03-27 16:14:51,133 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2017-03-27 16:14:51,134 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started
2017-03-27 16:14:51,139 INFO org.apache.hadoop.hdfs.server.datanode.BlockScanner: Initialized block scanner with targetBytesPerSec 1048576
2017-03-27 16:14:51,142 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Configured hostname is master.hadoop.lan
2017-03-27 16:14:51,151 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting DataNode with maxLockedMemory = 0
2017-03-27 16:14:51,179 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Shutdown complete.
2017-03-27 16:14:51,180 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain
java.net.BindException: Problem binding to [0.0.0.0:50010] java.net.BindException: Address already in use; For more details see:  http://wiki.apache.org/hadoop/BindException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
    at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:721)
    at org.apache.hadoop.ipc.Server.bind(Server.java:425)
    at org.apache.hadoop.ipc.Server.bind(Server.java:397)
    at org.apache.hadoop.hdfs.net.TcpPeerServer.<init>(TcpPeerServer.java:113)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.initDataXceiver(DataNode.java:897)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1111)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:429)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2374)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2261)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2308)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2485)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2509)
Caused by: java.net.BindException: Address already in use
    at sun.nio.ch.Net.bind0(Native Method)
    at sun.nio.ch.Net.bind(Net.java:433)
    at sun.nio.ch.Net.bind(Net.java:425)
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
    at org.apache.hadoop.ipc.Server.bind(Server.java:408)
    ... 10 more
  2017-03-27 16:14:51,184 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1
  2017-03-27 16:14:51,186 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
 /************************************************************

从错误日志来看,似乎一个进程已经在使用端口50010

java.net.BindException: 绑定到 [0.0.0.0:50010] 时出现问题 java.net.BindException:地址已在使用

在大多数情况下,是datanode进程本身没有正确终止。

使用端口获取进程的pid

netstat -ntpl | grep 50010
tcp 0 0 0.0.0.0:50010 0.0.0.0:* LISTEN <pid>/java

还要验证哪个进程正在使用端口

ps -ef | grep <pid>

kill流程

kill -9 <pid>

现在端口已可用,请尝试重新启动群集。

如果无法终止进程,则通过将此属性添加到 hdfs-site.xml

<property>
   <name>dfs.datanode.address</name>
   <value>hostname:different_port</value>
</property>

请参阅 Hadoop BindException 以了解此ERROR的所有可能原因。

最新更新