How can I remove duplicate dict in list in Python?

+1 vote
I have a list of dicts, and I'd like to remove the dicts with identical key and value pairs.

For this list: [{'a': 123}, {'b': 123}, {'a': 123}]

I'd like to return this: [{'a': 123}, {'b': 123}]
Nov 19, 2018 in Python by Anirudh
• 2,070 points
1,616 views

2 answers to this question.

+1 vote

You can preserve the Order, then you can do:

from collections import OrderedDict
print OrderedDict((frozenset(item.items()),item) for item in data).values()
# [{'a': 123, 'b': 1234}, {'a': 3222, 'b': 1234}]

If the order doesn't matter, then you can do:

print {frozenset(item.items()):item for item in data}.values()
# [{'a': 3222, 'b': 1234}, {'a': 123, 'b': 1234}]
answered Nov 19, 2018 by Nymeria
• 3,520 points
+2 votes
[dict(t) for t in {tuple(d.items()) for d in l}]

or

a = [{'a': 123}, {'b': 123}, {'a': 123}]
b = []
for i in range(0, len(a)):
    if a[i] not in a[i+1:]:
        b.append(a[i])
answered Aug 15 by anonymous

edited Aug 20 by Kalgi
This is so much easier than @Nymeria's answer. The logic is clear and understandable. Thanks!

Related Questions In Python

0 votes
1 answer

How can I convert a list of dictionaries from a CSV into a JSON object in Python?

You could try using the AST module. ...READ MORE

answered Apr 17, 2018 in Python by anonymous
795 views
0 votes
1 answer

How can I get the count of a list in Python?

len() >>> mylist=[] >>> print len(mylist) 0 READ MORE

answered Dec 12, 2018 in Python by charlie_brown
• 7,720 points
96 views
+1 vote
1 answer

How can I reverse list in Python?

Reversing a list is a commonly used ...READ MORE

answered May 13 in Python by Taj
• 1,040 points
51 views
0 votes
1 answer
0 votes
1 answer

How do I remove an element from a list by index in Python?

You can use the pop() method to ...READ MORE

answered Jun 21 in Python by Nisa
• 1,090 points
253 views
+1 vote
2 answers

how can i count the items in a list?

Syntax :            list. count(value) Code: colors = ['red', 'green', ...READ MORE

answered Jul 6 in Python by Neha
• 330 points

edited Jul 8 by Kalgi 346 views