Pig拉丁文区分Map或Reduce查询

Question

我有以下数据示例：

AGE,EDU,SEX,SALARY
67,10th,Male,<=50K
17,10th,Female,<=50K
40,Assoc-voc,Male,>50K
35,Assoc-voc,Male,<=50K
57,Assoc-voc,Male,<=50K
49,Assoc-voc,Male,>50K
42,Bachelors,Male,>50K
30,Bachelors,Male,>50K
23,Bachelors,Female,<=50K

==============================================

我创建了以下Pig Latin/hadoop脚本：

sensitive = LOAD '/mdsba' using PigStorage(',') as (AGE,EDU,SEX,SALARY);
    *--Filtered the data by the salary
    Data_filter1 = FILTER sensitive by (SALARY matches '<=50K');
    Data_filter2 = FILTER sensitive by (SALARY matches '>50K');
    --group both filters
    B= foreach(group Data_filter1 by(AGE,EDU,SEX)) 
    generate Data_filter1;
    C= foreach(group Data_filter2 by(AGE,EDU,SEX)) 
    generate Data_filter2;
    Dump B ;
    Dump C ;

=============================================

是否有任何方法可以确定查询B、C、Data_filter1或Data_filter2是否在Map或Reduce进程上运行。由于以下报告是在作业结束时生成的：

Elapsed: 35sec  
Diagnostics: 
 Average Map Time: 12sec  
 Average Shuffle Time: 10sec  
 Average Merge Time: 0sec  
 Average Reduce Time: 2sec 

非常感谢

Answer 1

是的，当你启动作业时，你会看到一个字符串

 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: Alias1[73,14] C: Alias2[20, 9] R: Alias3[90, 78]

M代表映射器，C代表组合器，R代表归约器。但在一般情况下，您的查询可能同时在映射器和还原器上运行