我使用了如下的分步指南。https://phoenixnap.com/kb/install-hadoop-ubuntu
然后我尝试在文本文件上运行mapreduce单词计数文件。问题是程序没有运行,我从容器启动中获得应用程序和异常的am容器。
有什么解决办法吗?
所有节点都在工作
6544译本3041年NameNode3842年NodeManager3219年DataNode3494年SecondaryNameNode3706年ResourceManager
下面是我的应用程序的纱线状态输出。
doop@contactkarim-VirtualBox:~/hadoop-3.3.1$ yarn app -status application_1667981786519_0006
2022-11-09 11:35:22,184 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at /127.0.0.1:8032
2022-11-09 11:35:22,522 INFO conf.Configuration: resource-types.xml not found
2022-11-09 11:35:22,522 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
Application Report :
Application-Id : application_1667981786519_0006
Application-Name : word count
Application-Type : MAPREDUCE
User : hdoop
Queue : default
Application Priority : 0
Start-Time : 1667982679380
Finish-Time : 1667982691120
Progress : 0%
State : FAILED
Final-State : FAILED
Tracking-URL : http://contactkm-VirtualBox:8088/cluster/app/application_1667981786519_0006
RPC Port : -1
AM Host : N/A
Aggregate Resource Allocation : 20250 MB-seconds, 8 vcore-seconds
Aggregate Resource Preempted : 0 MB-seconds, 0 vcore-seconds
Log Aggregation Status : DISABLED
Diagnostics : Application application_1667981786519_0006 failed 2 times due to AM Container for appattempt_1667981786519_0006_000002 exited with exitCode: 1
Failing this attempt.Diagnostics: [2022-11-09 11:31:31.113]Exception from container-launch.
Container id: container_1667981786519_0006_02_000001
Exit code: 1
[2022-11-09 11:31:31.116]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
[2022-11-09 11:31:31.116]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
For more detailed output, check the application tracking page: http://contactkarim-VirtualBox:8088/cluster/app/application_1667981786519_0006 Then click on links to logs of each attempt.
. Failing the application.
Unmanaged Application : false
Application Node Label Expression : <Not set>
AM container Node Label Expression : <DEFAULT_PARTITION>
TimeoutType : LIFETIME ExpiryTime : UNLIMITED RemainingTime : -1seconds
感谢我解决了很多问题,例如,我检查了站点设置,检查了资源。而且,我把构型看了很多遍。权限等。
我怀疑这里只有java版本。
链接的博客没有提到mapreduce,只有集群设置(我总是建议关注Apache Hadoop官方网站,而不是第三方博客)。
No appenders could be found
表示您缺少与作业一起提交的log4j.properties
文件-查看http://logging.apache.org/log4j/1.2/faq.html#noconfig了解更多信息。
你将无法看到真正的运行时错误/日志输出,直到你添加它,例如,如果你已经提交了自己的jar由maven/gradle构建到src/main/resources