MongoDB，计算mapreduce后的结果

Question

>我有mapreduce函数来分组和计算一些字段

map = %Q{
  function() {
    emit({ip: this.ip, campaign_id: this.campaign_id}, {total_count: 1, total_commission: this.commission, uniq_commission: 0});
  }
}
reduce = %Q{
  function(key, values) {   
    var total_commission = 0; 
    values.forEach(function(value) {
      total_commission += value.total_commission; // to sum 
    });        
    return {total_count: values.length, total_commission: total_commission, uniq_commission: values[0]["total_commission"]};
  }
}

输出如下：

[
{"_id":{"ip":"39.49.65.101","campaign_id":{"$oid":"548121156e6f62101f000000"}},"value":   {"total_count":1.0,"total_commission":1.0,"uniq_commission":0.0}},
{"_id":{"ip":"39.49.65.102","campaign_id":{"$oid":"548121156e6f62101f000000"}},"value":{"total_count":5.0,"total_commission":1.0,"uniq_commission":0.0}},
{"_id":{"ip":"39.49.65.103","campaign_id":{"$oid":"548121156e6f62101f000000"}},"value":{"total_count":3.0,"total_commission":1.0,"uniq_commission":0.0}},
...
]

我想这样计算：

result = {}
obj.map_reduce(map, reduce).out(inline: true).each do |r|
   result["total_count"] += r["value"]["total_count"]
   result["total_commission"] += r["value"]["total_commission"]
   result["uniq_count"] += 1 
   result["uniq_commission"] += r["value"]["uniq_commission"]
end

这个函数需要很长时间才能用大数据库计算50.000 记录 ~ 40s

这种情况有什么解决方案吗？我希望我可以访问一些全局 obj 来处理这个问题 ^_^

编辑：数据示例：

<Transaction _id: 5491a66e6e6f620ff4f74d00, commission: 1.0, total_cost: 200.0, order_id: "Order-1418831470", ip: "11.11.11.11", referer_url: nil, campaign_id: BSON::ObjectId('548121156e6f62101f000000'), affiliate_id: BSON::ObjectId('5472dea46e6f62236c060000'), rawclick_id: BSON::ObjectId('5491a66e6e6f620ff4f64d00'), _keywords: ["123213", "abc", "company", "last", "name", "organization"]> 

<Transaction _id: 5491a66e6e6f620ff4f74d11, commission: 1.0, total_cost: 200.0, order_id: "Order-1418831470", ip: "11.11.11.11", referer_url: nil, campaign_id: BSON::ObjectId('548121156e6f62101f000000'), affiliate_id: BSON::ObjectId('5472dea46e6f62236c060000'), rawclick_id: BSON::ObjectId('5491a66e6e6f620ff4f64d00'), _keywords: ["123213", "abc", "company", "last", "name", "organization"]> 

<Transaction _id: 5491a66e6e6f620ff4f74d22, commission: 1.0, total_cost: 200.0, order_id: "Order-1418831470", ip: "22.22.22.22", referer_url: nil, campaign_id: BSON::ObjectId('548121156e6f62101f000000'), affiliate_id: BSON::ObjectId('5472dea46e6f62236c060000'), rawclick_id: BSON::ObjectId('5491a66e6e6f620ff4f64d00'), _keywords: ["123213", "abc", "company", "last", "name", "organization"]> 

<Transaction _id: 5491a66e6e6f620ff4f74d33, commission: 1.0, total_cost: 200.0, order_id: "Order-1418831470", ip: "33.33.33.33", referer_url: nil, campaign_id: BSON::ObjectId('548121156e6f62101f000000'), affiliate_id: BSON::ObjectId('5472dea46e6f62236c060000'), rawclick_id: BSON::ObjectId('5491a66e6e6f620ff4f64d00'), _keywords: ["123213", "abc", "company", "last", "name", "organization"]> 

<Transaction _id: 5491a66e6e6f620ff4f74d44, commission: 1.0, total_cost: 200.0, order_id: "Order-1418831470", ip: "96.9.32.51", referer_url: nil, campaign_id: BSON::ObjectId('548121156e6f62101f000000'), affiliate_id: BSON::ObjectId('5472dea46e6f62236c060000'), rawclick_id: BSON::ObjectId('5491a66e6e6f620ff4f64d00'), _keywords: ["123213", "abc", "company", "last", "name", "organization"]> 

Answer 1

如果我

正确阅读了您的地图/reduce，您所做的只是计算按ip和campaign_id分组的文档的数量和总佣金。我不确定uniq_commission的目的是什么，因为在您的map/reduce中，它的最终值取决于MongoDB如何执行map/reduce，在其他代码中，它始终为零，因为它汇总的值似乎都设置为零。- 如果你能解释它的目的，我会尝试修改我的答案以考虑到它。不要使用映射/减少。聚合可以完成此计算，并且比map/reduce更快：

db.test.aggregate([
    { "$group" : { "_id" : { "ip" : "$ip", "campaign_id" : "$campaign_id" },
                   "total_count" : { "$sum" : 1 },
                   "total_commission" : { "$sum" : "$commission" }
    } }
])