Oozie java-action does not include core-site.xml

Question

当在新安装的Hadoop HDP 2.2.2.4上运行Oozie java操作时，例如试图访问hdfs，它访问了错误的文件系统:java.lang.IllegalArgumentException: Wrong FS: hdfs:/tmp/text.txt, expected: file:///

可以通过在Oozie动作中包含core-site.xml来修复:

<file>hdfs:/path-to-core-site.xml-on-hdfs</file>

但是原因是什么，正确的解决办法是什么?

Answer 1

core-site.xml不包含在java-action的类路径中的原因是属性mapreduce.application。类路径指向错误的目录:

<snip>/etc/hadoop/conf/secure

它应该指向

<snip>/etc/hadoop/conf

。E，完整的属性应该是这样的，在mapred-site.xml:

<property>
  <name>mapreduce.application.classpath</name>
  <value>$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:/usr/hdp/${hdp.version}/hadoop/lib/hadoop-lzo-0.6.0.${hdp.version}.jar:/etc/hadoop/conf</value>
</property>

Answer 2

这些文件包含在hadoop类路径中，据我所知，自HDP 2.2以来，您需要添加

// loading action conf prepared by Oozie
Configuration actionConf = new Configuration(false);
actionConf.addResource(new Path("file:///", System.getProperty("oozie.action.conf.xml")));

使用*-site.xml，您可以在oozie文档中获得详细信息https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html a3.2.7_Java_Action