使用色调中的oozie工作流程执行MapReduce作业，给出错误的输出

Question

我正在尝试使用色调中的oozie工作流程执行MapReduce作业。当我提交作业时，oozie 成功执行，但我没有得到预期的输出。似乎映射器或化简器从未调用过。这是我的工作流程.xml：

<workflow-app name="wordCount" xmlns="uri:oozie:workflow:0.4">
    <start to="wordcount"/>
    <action name="wordcount">
        <map-reduce>
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>mapred.input.dir</name>
                    <value>/user/root/jane/inputPath</value>
                </property>
                <property>
                    <name>mapred.output.dir</name>
                    <value>/user/root/jane/outputPath17</value>
                </property>
                <property>
                    <name>mapred.mapper.class</name>
                    <value>MapReduceGenerateReports.Map</value>
                </property>
                <property>
                    <name>mapred.reducer.class</name>
                    <value>MapReduceGenerateReports.Reduce</value>
                </property>
                <property>
                    <name>mapred.mapper.new-api</name>
                    <value>true</value>
                </property>
                <property>
                    <name>mapred.reducer.new-api</name>
                    <value>true</value>
                </property>
            </configuration>
        </map-reduce>
        <ok to="end"/>
        <error to="kill"/>
    </action>
    <kill name="kill">
        <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

谁能说出问题出在哪里？

我的新工作流程.xml ：

<workflow-app name="wordCount" xmlns="uri:oozie:workflow:0.4">
    <start to="wordcount"/>
    <action name="wordcount">
        <map-reduce>
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>mapred.input.dir</name>
                    <value>/user/root/jane/inputPath</value>
                </property>
                <property>
                    <name>mapred.output.dir</name>
                    <value>/user/root/jane/outputPath3</value>
                </property>
                <property>
                    <name>mapred.mapper.new-api</name>
                    <value>true</value>
                </property>
                <property>
                    <name>mapred.reducer.new-api</name>
                    <value>true</value>
                </property>
                <property>
                    <name>mapreduce.map.class</name>
                    <value>MapReduceGenerateReports$Map</value>
                </property>
                <property>
                    <name>mapreduce.reduce.class</name>
                    <value>MapReduceGenerateReports$Reduce</value>
                </property>
                <property>
                    <name> mapred.output.key.class</name>
                    <value>org.apache.hadoop.io.LongWritable</value>
                </property>
                <property>
                    <name>mapred.output.value.class</name>
                    <value>org.apache.hadoop.io.Text</value>
                </property>
            </configuration>
        </map-reduce>
        <ok to="end"/>
        <error to="kill"/>
    </action>
    <kill name="kill">
        <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

作业跟踪器日志：

1）

Kind    % Complete  Num Tasks   Pending Running Complete    Killed  Failed/Killed
Task Attempts
map 100.00%
1   0   0   1   0   0 / 0
reduce  100.00%
0   0   0   0   0   0 / 0

2）

   Kind Total Tasks(successful+failed+killed)   Successful tasks    Failed tasks    Killed tasks    Start Time  Finish Time
    Setup    1   1   0   0  5-Apr-2014 18:36:22 5-Apr-2014 18:36:23 (1sec)
    Map  1   1   0   0  5-Apr-2014 18:33:27 5-Apr-2014 18:33:33 (5sec)
    Reduce   0   0   0   0      
    Cleanup  1   1   0   0  5-Apr-2014 18:33:33 5-Apr-2014 18:33:37 (4sec)

Answer 1

在此处查看使用新 API 的说明

但是，如果你真的需要运行在Oozie中使用20 API编写的MapReduce作业，以下是你需要在工作流程中进行的更改.xml。

1. 将mapred.mapper.class更改为mapreduce.map.class
2. 将 mapred.reducer.class 更改为 mapreduce.reduce.class
3. 添加 mapred.output.key.class
4. 添加 mapred.output.value.class
5. 并且，将以下属性包含在 MR 操作配置中