在Hadoop上运行MapReduce字数会出现异常消息:系统找不到指定的路径



这是我有史以来第一个堆栈溢出问题。我已经在伪分布式模式下设置了hadoop(2.9.2)单节点集群。当我尝试运行hadoop jar C:/MapReduceClient.jar wordcount/input_dir/output_dir时,我得到了以下带有错误的日志

19/01/16 20:19:17 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/01/16 20:19:18 INFO input.FileInputFormat: Total input files to process : 1
19/01/16 20:19:19 INFO mapreduce.JobSubmitter: number of splits:1
19/01/16 20:19:19 INFO Configuration.deprecation: yarn.resourcemanager.system-metrics-publisher.enabled is deprecated. Instead, use yarn.system-metrics-publisher.enabled
19/01/16 20:19:19 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1547662294790_0002
19/01/16 20:19:19 INFO impl.YarnClientImpl: Submitted application application_1547662294790_0002
19/01/16 20:19:19 INFO mapreduce.Job: The url to track the job: http://DESKTOP-XXXXXX:8088/proxy/application_1547662294790_0002/
19/01/16 20:19:19 INFO mapreduce.Job: Running job: job_1547662294790_0002
19/01/16 20:19:19 INFO mapreduce.Job: Running job: job_1547662294790_0002
19/01/16 20:19:27 INFO mapreduce.Job: Job job_1547662294790_0002 running in uber mode : false
**19/01/16 20:19:27 INFO mapreduce.Job:  map 0% reduce 0%**
**19/01/16 20:19:27 INFO mapreduce.Job: Job job_1547662294790_0002 failed with state FAILED due to: Application application_1547662294790_0002 failed 2 times due to AM Container for appattempt_1547662294790_0002_000002 exited with  exitCode: 1**
Failing this attempt.Diagnostics: [2019-01-16 20:19:25.234]Exception from container-launch.
Container id: container_1547662294790_0002_02_000001
Exit code: 1
**Exception message: The system cannot find the path specified.**
The system cannot find the path specified.
The system cannot find the path specified.

[2019-01-16 20:19:25.236]Container exited with a non-zero exit code 1.
[2019-01-16 20:19:25.236]Container exited with a non-zero exit code 1.
For more detailed output, check the application tracking page: http://DESKTOP-XXXXX:8088/cluster/app/application_1547662294790_0002 Then click on links to logs of each attempt
. Failing the application.
19/01/16 20:19:28 INFO mapreduce.Job: Counters: 0

同样的.jar在我的另一台电脑上运行,输出是正确的。Windows 10 Pro x64(两者)唯一的区别是工作中的一个安装了java 1.8.0_171

JAVA_HOME=C:\JAVA\jdk1.8.0_201HADOOP_HOME=C:\HADOOP-2.9.2

路径=%JAVA_HOME%\bin;C: \hadoop-2.9.2\bin

我的配置文件:/etc/hadop/core-site.xml

<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>

/etc/hadop/mapred-site.xml

<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>

/etc/hdop/hdfs-site.xml

<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>C:hadoop-2.9.2datanamenode</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>C:hadoop-2.9.2datadatanode</value>
</property>
</configuration>

/etc/hadop/yarn-site.xml

<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>  
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration>

已解决。它是非拉丁字符的默认用户名,扰乱了节点管理器。使用whoami命令进行检查,结果发现默认用户名是"????">

相关内容

  • 没有找到相关文章

最新更新