编写下面的mapreduce代码是为了得到以下输出:
"普林斯顿航空公司"
"优先包机"
"优先航空运输"优先航空公司"
"私人飞机探险" "私人飞机管理"
法典:
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
public class UScountry {
public static class mymapper extends Mapper<Object,Text,Text,Text>{
protected void map(Object key,Text value,Context context) throws IOException, InterruptedException{
String data[]=value.toString().split(",");
CharSequence c="united states";
if(data[6].toLowerCase().contains(c)){
context.write(new Text(data[1]),new Text("") );
}
}
}
public static void main(String args[]) throws IOException, ClassNotFoundException, InterruptedException
{
Configuration conf=new Configuration();
Job job=Job.getInstance(conf,"US");
job.setJarByClass(UScountry.class);
job.setMapperClass(mymapper.class);
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(Text.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
job.waitForCompletion(true);
}
}
但是代码给出的输出是:
"普林斯顿航空公司" 0
"优先包机" 0
"优先空运" 0
"优先航空公司" 0
"私人飞机探险" 0
"私人飞机管理" 0
请帮我找出我哪里出错了!
输入文件为:
1,"特权样式 L",\N,","PVG","PRIVILEGE","西班牙","N"
2,"普林斯顿航空公司",\N,","PCN","普林斯顿","美联航 状态","N"
3,"优先包机",\N,","撬","优先航空","美联航 状态","N"
4,"优先空运",\N,","PAT","PAT","美国","N"
5,"优先航空公司",\N,","BCK","银行支票","美联航 状态","N"
6,"Privatair",\N,","PTI","PRIVATAIR","Switzerland","Y"
7,"私人飞机探险",\N,","PJE","皮杰","美国","N"
8,"私人飞机管理",\N,","PJA","私人航班","美联航 状态","N"
9,"私人之翼",\N,"8W","PWF","私人 翅膀","德国","N"
您可以为输出值类设置NullWritable.class
。
job.setOutputValueClass(NullWritable.class);
这将抑制减速器输出的值部分,即输出中的 0。
尝试更改:
context.write(new Text(data[1]),new Text("") );
对此:
context.write(new Text(data[1]));