我有这样的代码:
public void reduce(Text key, Iterable<Text> values, Context context)
throws IOException, InterruptedException
{
String name = null;
String sid = null;
String predicate = null;
String oid = null;
String id = null;
String outKey = null;
String outVal = null;
LinkedList<Text> valuesList = new LinkedList<Text>();
Iterator<Text> ite = values.iterator();
while(ite.hasNext()) {
Text t = ite.next();
String[] entities = t.toString().split("#-#-#-#");
if(entities[entities.length-1].equalsIgnoreCase("topic_name"))
{
name = entities[0];
}
valuesList.add(t);
}
Iterator<Text> ite2 = valuesList.iterator();
while(ite2.hasNext()) {
Text t2 = ite2.next();
String[] entities = t2.toString().split("#-#-#-#");
if(!entities[entities.length-1].contains("topic_name"))
{
if(name!=null) {
outKey = entities[0]+"t"+entities[1]+"t"+name;
}
else
{
outKey = entities[0]+"t"+entities[1]+"t"+key.toString();
}
context.write(new Text(outKey), null);
}
}
}
我看到当我再次遍历值时,它总是取缓存副本中的最后一个值。
第一个迭代器实际上总是返回相同的Text
对象,它只是在每次调用之前用不同的String填充它。这样做是为了节省实例化对象的时间。所以你实际上是在构建一个List<Text>
,包含同一物体的许多副本。为了解决这个问题,你应该将值保存到包含实际"未装箱"值的List<String>
中。这样的:
LinkedList<String> valuesList = new LinkedList<String>();
Iterator<Text> ite = values.iterator();
while(ite.hasNext()) {
Text t = ite.next();
String[] entities = t.toString().split("#-#-#-#");
if(entities[entities.length-1].equalsIgnoreCase("topic_name"))
{
name = entities[0];
}
valuesList.add(t.toString());
}
Iterator<String> ite2 = valuesList.iterator();
while(ite2.hasNext()) {
String t2 = ite2.next();
String[] entities = t2.split("#-#-#-#");
if(!entities[entities.length-1].contains("topic_name"))
{
if(name!=null) {
outKey = entities[0]+"t"+entities[1]+"t"+name;
}
else
{
outKey = entities[0]+"t"+entities[1]+"t"+key.toString();
}
context.write(new Text(outKey), null);
}
}