如何从字典中删除所有重复的值(在相同或不同的键下)?

Question

我有一个像这样的字典:

{"A": ["a", "b", "c"],
"B": ["a", "d", "f"],
"C": ["i", "i", "j"]}

我想对它进行如下变换:

{"A": ["b", "c"],
"B": ["d", "f"],
"C": ["j"]}

也就是说，所有重复的值都被删除，无论它们出现在同一个键中还是不同的键中。如何有效地实现它?

Answer 1

此代码将删除所有重复项:

from collections import Counter
def remove_all_dupes(d):
c = Counter()
for v in d.values():
c.update(v)
for k,v in d.items():
d[k] = [item for item in v if c[item] == 1]
return d
d = {"A": ["a", "b", "c"],
"B": ["a", "d", "f"],
"C": ["i", "i", "j"]}
print(d)
remove_all_dupes(d)
print(d)

按要求输出。

我希望这是相当有效的，即O(n)，因为它只循环遍历所有值两次，并且查找以消除重复项应该是O(1)。

Answer 2

试试这个:

from itertools import chain
d = {"A": ["a", "b", "c"],
"B": ["a", "d", "f"],
"C": ["i", "i", "j"]}
new_dic = {}
values = list(chain(*d.values()))
for key, value in d.items():
new_dic[key] = [x for x in value if values.count(x) == 1]

print(new_dic)

使用字典推导式:

new_dic = {key: [x for x in value if list(chain(*d.values())).count(x)==1] for key, value in d.items()}

输出:

{"A": ["b", "c"], "B": ["d", "f"], "C": ["j"]}

Answer 3

这段代码没有使用任何导入。

data = {"A": ["a", "b", "c"], "B": ["a", "d", "f"], "C": ["i", "i", "j"]}
# vals = []
# for ky in data.keys():
#     vals = vals + data[ky]
vals = sum(data.values(), [])    
dups = set([val for val in vals if vals.count(val) > 1])
data_deduped = {
ky: [val for val in data[ky] if not val in dups] for ky in data.keys()
}
print(data_deduped)

Sample Output
{'A': ['b', 'c'], 'B': ['d', 'f'], 'C': ['j']}

Answer 4

您可以使用pip install more-itertools和flatten来快速创建所有值的列表，并将它们与collections中的Counter一起计数。

from more_itertools import flatten
from collections import Counter
def remove_duplicates(d):
all_values = list(flatten(d.values()))
count = Counter(all_values)
filtered_dict = {key: [v for v in value if count[v] == 1] for key, value in d.items()}
return filtered_dict

Answer 5

from collections import Counter
d_input = {"A": ["a", "b", "c"], "B": ["a", "d", "f"], "C": ["i", "i", "j"]}
d_correct_answer = {"A": ["b", "c"], "B": ["d", "f"], "C": ["j"]}
# first make a giant list of all lists in dictionary
giant_list = []
for key in d_input:
partial_list = d_input[key]
# build a big list of all items
giant_list.extend(partial_list)
# use counter to find items
counted = Counter(giant_list)
# now build remove list
remove_list = []
for key in counted:
if counted[key] > 1:
remove_list.append(key)
# now loop and remove by only adding proper results to new dict
result = {}
for key in d_input:
partial_list = d_input[key]
new_partial_list = []
for item in partial_list:
if item not in remove_list:
new_partial_list.append(item)
result[key] = new_partial_list
print(f'it is {result == d_correct_answer} that this code works')