比较列表中的对象，以识别具有某些相同键/值对的对象和不具有相同键/值对的对象

Question

使用Python，我如何在列表中找到共享某些键/值对的对象，然后创建两个单独的列表-一个用于共享这些键/值对的对象，另一个用于不共享这些键/值对的对象?

例如，使用下面的简单列表-

[
{
"id": "111",
"host": "aaa",
"path": "/b/c/d"
},
{
"id": "222",
"host": "bbb",
"path": "/x/y/z"
},
{
"id": "333",
"host": "aaa",
"path": "/b/c/d"
},
{
"id": "444",
"host": "aaa",
"path": "/b/c/d"
}
]

我想以两个列表结束-

• 具有重复host和path的对象。

  [
  {
  "host": "aaa",
  "path": "/b/c/d"
  "ids": [
  "111",
  "333",
  "444",
  }
  ]
  

• 没有重复host和path的对象。

  [
  {
  "id": "222",
  "host": "bbb",
  "path": "/x/y/z"
  }
  ]
  

到目前为止，我最好的尝试已经产生了两个列表，但是原始列表中的所有对象都被添加到dups_list中，无论它们是否实际上是重复的。

请注意，我曾尝试在第二个for语句中使用main_list的deepcopy，但产生了完全相同的结果。

>>> import jsonpickle
>>> main_list = list((dict(Id="111",host="aaa",path="/b/c/d"),dict(Id="222",host="bbb",path="/x/y/z"),dict(Id="333",host="aaa",path="/b/c/d"),dict(Id="444",host="aaa",path="/b/c/d")))
>>> dups_list = list()
>>> non_dups_list = list()
>>> for o in main_list:
...     is_duplicate = False
...     for o2 in main_list:
...         if o2['host'] == o['host'] and o2['path'] == o['path']:
...             is_duplicate = True
...             break
...     if is_duplicate:
...         dups_list.append(o)
...     else:
...         non_dups_list.append(o)
... 
>>> print(jsonpickle.encode(non_dups_list, indent=4))
[]
>>> print(jsonpickle.encode(dups_list, indent=4))
[
{
"Id": "111",
"host": "aaa",
"path": "/b/c/d"
},
{
"Id": "222",
"host": "bbb",
"path": "/x/y/z"
},
{
"Id": "333",
"host": "aaa",
"path": "/b/c/d"
},
{
"Id": "444",
"host": "aaa",
"path": "/b/c/d"
}
]

Answer 1

我将使用defaultdict与dict/listcomps:

from collections import defaultdict
g = defaultdict(list)
for obj in list_objs:
g[(obj["host"], obj["path"])].append(obj["id"])
dups = [{"host": k[0], "path": k[1], "ids": v} for k, v in g.items() if len(v) > 1]
uniqs = [obj for obj in list_objs if (obj["host"], obj["path"])
not in [k for k, v in g.items() if len(v) > 1]]
#12.2 µs ± 316 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)

输出:

>>> print(dups)
#[{'host': 'aaa', 'path': '/b/c/d', 'ids': ['111', '333', '444']}]
>>> print(uniqs)
#[{'id': '222', 'host': 'bbb', 'path': '/x/y/z'}]

Answer 2

我建议使用itertools。

from itertools import groupby
def get_key(d):
# Define a custom key function to group by multiple keys
return d['host'], d['path']
data = [...your data here]
grouped_data = []
for k, g in groupby(sorted(data, key=get_key), key=get_key):
grouped_data.append({'host': k[0], 'path': k[1], 'ids': [i['id'] for i in list(g)]})