我在Ubuntu 14.04上,安装了Hadoop 2.5.1
我的jps
命令提供了所有资源信息,所以我知道hadoop运行正常!
我尝试从这里在分布式系统上运行MapReduce示例
但在某个时间点后,作业冻结
hduser@stratageeks-DL-E35LM1-R2:/usr/local/hadoop$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.1.jar grep input output 'dfs[a-z.]+'
14/09/24 00:13:02 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/09/24 00:13:14 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
14/09/24 00:13:20 WARN mapreduce.JobSubmitter: No job jar file set. User classes may not be found. See Job or Job#setJar(String).
14/09/24 00:13:21 INFO input.FileInputFormat: Total input paths to process : 30
14/09/24 00:13:25 INFO mapreduce.JobSubmitter: number of splits:30
14/09/24 00:13:30 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1411497776309_0001
14/09/24 00:13:33 INFO mapred.YARNRunner: Job jar is not present. Not adding any jar to the list of resources.
14/09/24 00:13:35 INFO impl.YarnClientImpl: Submitted application application_1411497776309_0001
14/09/24 00:13:35 INFO mapreduce.Job: The url to track the job: http://stratageeks-DL-E35LM1-R2:8088/proxy/application_1411497776309_0001/
14/09/24 00:13:35 INFO mapreduce.Job: Running job: job_1411497776309_0001
我甚至将我在yarn-site.xml中的内存使用量增加到4096,但仍然冻结
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>4096</value>
<description>Physical memory, in MB, to be made available to running containers</description>
</property>
还有什么建议吗。。?
是否检查了所有日志中的错误?还有userlogs
文件夹下的作业特定日志(尤其是syslog)?错误文件通常提供有关实际问题的良好线索。
我在yarn-site.xml
中修复了它
由于某种原因,其中一个属性不接受.
,因此用_
替换
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
最初我的价值是
<value>mapreduce_shuffle</value>