我正在apache-pig中运行一个group-by子句,它正在创建map reduce作业,该作业在完成1/3后失败。不管怎样,我可以解决这个问题吗?因为日志没有给出任何失败的原因。我在看下面的任何一个
1.找到确切错误的方法(即内存错误、数据类型错误等)
2.任何让日志更详细的方法都可以在屏幕上写更多的错误消息。
2016-04-03 22:59:40,252 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_1459707650021_0003
2016-04-03 22:59:40,252 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases a,b
2016-04-03 22:59:40,252 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: a[1,4],b[2,4] C: R:
2016-04-03 22:59:40,258 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
2016-04-03 22:59:40,258 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1459707650021_0003]
2016-04-03 23:00:10,428 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 33% complete
2016-04-03 23:00:10,428 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1459707650021_0003]
2016-04-03 23:00:15,435 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
2016-04-03 23:00:15,435 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_1459707650021_0003 has failed! Stop running all dependent jobs
2016-04-03 23:00:15,435 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
2016-04-03 23:00:15,508 [main] INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://compute.internal:8188/ws/v1/timeline/
2016-04-03 23:00:15,511 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at compute.internal/8050
2016-04-03 23:00:15,515 [main] INFO org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=FAILED. Redirecting to job history server
提交到map reduce框架的每个作业都有一个JobId。复制该JobId,粘贴并在你的工作跟踪器中搜索。例如,在CDH3中,他们在浏览器中通过集群cloudera设置了作业跟踪器,在那里你可以看到所有作业的详细信息。