我在各种博客上都提到了Sqoop作业所需的选项文件应该放在HDFS位置内的事实。
我确实将文件放置在HDFS位置，并在文件参数中提供了该文件。
尽管如此，它还是抛出FileNotFound异常。
oozie工作流文件如下所示-
<workflow-app name="TestFlow" xmlns="uri:oozie:workflow:0.4">
<start to="TableImport"/>
<action name="TableImport">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<prepare>
<delete path="${nameNode}/path/to/place/file"/>
</prepare>
<arg>--options-file </arg>
<arg>/tmp/sqoopoptions.params</arg>
<arg>--target-dir</arg>
<arg>/path/to/place/file</arg>
<arg>--query</arg>
<arg>select employee_id, employeename, DEPT_ID from employee where $CONDITIONS</arg>
<arg>--split-by</arg>
<arg>DEPT_ID</arg>
<file>/tmp/sqoopoptions.params#sqoopoptions.params</file>
</sqoop>
<ok to="end"/>
<error to="kill"/>
</action>
<kill name="kill">
<message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
选项文件存在于HDFS位置中
/tmp/sqoopoptions.params
我可以从Hue查看该文件，这意味着该文件位于HDFS位置！。
我使用Hortonworks 2.1分发
异常stakctrace是
4046 [main] ERROR org.apache.sqoop.Sqoop - Error while expanding arguments
java.lang.Exception: Unable to read options file: /tmp/sqoopoptions.params
at org.apache.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:102)
at com.cloudera.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:33)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:203)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231)
at org.apache.sqoop.Sqoop.main(Sqoop.java:240)
at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:206)
at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:174)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:38)
at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:45)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:225)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.io.FileNotFoundException: /tmp/sqoopoptions.params (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:146)
at java.io.FileReader.<init>(FileReader.java:72)
at org.apache.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:70)
更新：
我试图将选项文件放在oozie服务器运行的本地文件系统中。它有效。当我查看代码时，它似乎也是正确的。org.apache.scoop.util.OptionsFileUtil的相关部分显示在下方
public static String[] expandArguments(String[] args) throws Exception {
List<String> options = new ArrayList<String>();
for (int i = 0; i < args.length; i++) {
if (args[i].equals(Sqoop.SQOOP_OPTIONS_FILE_SPECIFIER)) {
if (i == args.length - 1) {
throw new Exception("Missing options file");
}
String fileName = args[++i];
File optionsFile = new File(fileName);
BufferedReader reader = null;
StringBuilder buffer = new StringBuilder();
try {
**reader = new BufferedReader(new FileReader(optionsFile));**
String nextLine = null;
while ((nextLine = reader.readLine()) != null) {
现在，这适用于本地文件配置。这意味着，如果我把文件放在HDFS中，它就不会工作。除非，如果路径给定为hdfs://namdnode/path/to/file。这就是预期的工作方式吗？
我将参数赋给了｛nameNode｝/tmp/scoopoptions.params。它引发了以下异常。如果我仔细地注意到，/正在引发问题。它从异常堆栈跟踪中清除。其中一个斜线在传递到FileReader 时消失
java.lang.Exception: Unable to read options file: hdfs://ipofnamednode:port/tmp/sqoopoptions.params
at org.apache.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:102)
at com.cloudera.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:33)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:203)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231)
at org.apache.sqoop.Sqoop.main(Sqoop.java:240)
at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:206)
at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:174)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:38)
at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:45)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:225)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
**Caused by: java.io.FileNotFoundException: hdfs:/ipofnamednode:port/tmp/sqoopoptions.params (No such file or directory)**
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:146)
at java.io.FileReader.<init>(FileReader.java:72)
at org.apache.sqoop.util.OptionsFileUtil.expandArguments(OptionsFileUtil.java:70)
... 21 more
如stacktrace中所示，其中一个斜杠由于转义字符而被删除。
您需要在<file>属性中的文件路径前面加上namenode，例如：
<file>hdfs://tmp/sqoopoptions.params#sqoopoptions.params</file>
尝试将符号链接名称作为<arg>，并将namenode添加到<file>标记的开头：
...
<arg>--options-file </arg>
<arg>sqoopoptions.params</arg>
...
<file>${namenode}/tmp/sqoopoptions.params#sqoopoptions.params</file>
...