我正在构建一个工具,它要求我在Hive SemanticAnalyzer中运行MapReduce作业。当我运行我的自定义Java程序来处理启动Hive驱动程序时,MapReduce作业运行良好。但是,当我尝试通过修改HiveServer2和Hive JDBC(可以很好地连接到我修改的服务器)来执行钩子时,MapReduce作业失败并显示以下错误:
Application application_1395851979242_0009 failed 2 times due to AM Container for appattempt_1395851979242_0009_000002 exited with exitCode: 1 due to: Exception from container-launch:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
at org.apache.hadoop.util.Shell.run(Shell.java:379)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
有没有人遇到过类似的 YARN 错误?
我使用了命令:
yarn logs --applicationId myAppId
以检索应用程序容器日志。
在那之后,我发现我少了一些罐子。由于我正在处理的应用程序的性质(MR Jobs从Thrift的Hive Hook启动),我不得不编辑MapReduce Job配置以将jar包含在"tmpjars"设置中。