Removing duplicates in lists

0 votes

Pretty much I need to write a program to check if a list has any duplicates and if it does it removes them and returns a new list with the items that weren't duplicated/removed. This is what I have but to be honest I do not know what to do.

def remove_duplicates():
    t = ['a', 'b', 'c', 'd']
    t2 = ['a', 'c', 'd']
    for t in t2:
    return t
Feb 10, 2022 in Python by surbhi
• 3,810 points

2 answers to this question.

0 votes
Using a set is a typical way to create a one-of-a-kind collection of goods. Sets are groups of different things that are not arranged in any particular order. You can create a set from any iterable by using the built-in set() function. You can also give the set to the list() function if you require a real list later.

The example below should cover whatever you're wanting to accomplish:

>>> t = [1, 2, 3, 1, 2, 5, 6, 7, 8]
>>> t
[1, 2, 3, 1, 2, 5, 6, 7, 8]
>>> list(set(t))
[1, 2, 3, 5, 6, 7, 8]
>>> s = [1, 2, 3]
>>> list(set(t) - set(s))
[8, 5, 6, 7]

The original order is not retained, as you can see in the example result. As previously stated, sets are unsorted collections, hence the order is lost. When you convert a set back to a list, you get an arbitrary order.

Keeping the peace
If order is crucial to you, you'll need to utilize a different method. Using OrderedDict to retain the order of keys during insertion is a standard approach for this:

>>> from collections import OrderedDict
>>> list(OrderedDict.fromkeys(t))
[1, 2, 3, 5, 6, 7, 8]

Starting with Python 3.7, the built-in dictionary is guaranteed to keep the insertion order, thus if you're using Python 3.7 or later (or CPython 3.6), you can use it directly:

>>> list(dict.fromkeys(t))
[1, 2, 3, 5, 6, 7, 8]

It's worth noting that the overhead of first building a dictionary and then creating a list from it could be significant. You're frequently better off utilizing a set if you don't need to keep the order, especially because it provides you a lot more operations to deal with.

Finally, your items must be hashable for both the set and the OrderedDict/dict solutions to work. This usually implies that they must remain unchangeable. If you need to work with items that aren't hashable (like list objects), you'll have to take a more time-consuming technique, in which you'll have to compare each item to every other item in a nested loop.
answered Feb 10, 2022 by CoolCoder
• 4,360 points
0 votes
list(set(t + t2))
answered Feb 13, 2022 by anonymous

Related Questions In Python

0 votes
1 answer

Removing duplicates in lists

To remove duplicates from List use, set ...READ MORE

answered Feb 10, 2022 in Python by Nandini
• 5,480 points
0 votes
0 answers
0 votes
1 answer

How can I iterate through two lists in Parallel

You have to use the zip function ...READ MORE

answered Apr 17, 2018 in Python by anonymous
0 votes
1 answer

Copying lists in python

Assigning one list to another one simply ...READ MORE

answered Apr 30, 2018 in Python by Nietzsche's daemon
• 4,260 points
0 votes
2 answers
+1 vote
2 answers

how can i count the items in a list?

Syntax :            list. count(value) Code: colors = ['red', 'green', ...READ MORE

answered Jul 7, 2019 in Python by Neha
• 330 points

edited Jul 8, 2019 by Kalgi 3,304 views
0 votes
1 answer
0 votes
1 answer

When to use "while" or "for" in Python

Yes, there is a significant distinction between ...READ MORE

answered Feb 9, 2022 in Python by CoolCoder
• 4,360 points
0 votes
1 answer

Sum a list of numbers in Python

So you want (element 0 + element ...READ MORE

answered Feb 14, 2022 in Python by CoolCoder
• 4,360 points
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP