As we all know, the list in Python is easy to use, and very playful but we always have one question about the list in Python How to remove duplicates from a list in Python? So, let's discover different ways to remove duplicates in the Python list.

What are duplicates in the list?

As we know, Lists are mutable data types and hence can have more than one element with the same value. These elements are duplicates. Moreover, these duplicates must be removed for programming in python.

We have several methods to remove the duplicates from the list python. Like converting a list temporarily into a dictionary.

Algorithm to remove duplicates from list python

There are different ways to remove the duplicates from the Python array but most of them work on the same algorithm:

• Input the list.
• Take elements index-wise.
• Check whether elements have more than one occurrence If true then remove all other occurrences. If False then moves to the next index.
• Then return a new list without duplicates.
• Stop

Ways to remove duplicates from list Python

Let's see different methods to remove duplicates from list Python:

1. Removing duplicates from a list using Dict function

Duplicates from a list can be removed from the list using the dict method. OrderedDict is an in-built function that is imported from the collection module.

from collections import OrderedDict
lst1=[1, 4, 8, 2, 1, 8, 3, 5, 3, 7]
new_list= list(OrderedDict.fromkeys(lst1))
print(new_list)
#Output
[1, 4, 8, 2, 3, 5, 7]

2. Using Naive method using Temporary List

In this method, the elements of the list are traversed and the first occurrence of every element is stored in the temporary list. Hence, duplicates are removed as the only first occurrence of the element is stored using the list method append().

list1 = [1, 2, 3, 1, 2, 4, 5, 6, 3, 2]
print("The original list is ", list1)
temp = []

for x in list1:
    if x not in temp:
        temp.append(x)

list1 = temp
print("List After removing duplicates ", list1)
#Output
The original list is [1, 2, 3, 1, 2, 4, 5, 6, 3, 2]
List After removing duplicates [1, 2, 3, 4, 5, 6]

3. Using set() function

We can remove duplicates from a list using an inbuilt function called set(). The set() always returns distinct elements. Therefore, we use the set() for removing duplicates.

list1 = [1,1,2,3,2,2,4,5,6,7,4]
new_list = set(list1)
print(list(new_list))
#Output
[1, 2, 3, 4, 5, 6, 7]

4. Remove duplicates using List Comprehension

List comprehensions are Python functions that are used for creating new sequences (such as lists, tuples, etc.) using previously created sequences. This makes code more efficient and easy to understand. List comprehension is used to remove duplicates from an array.

list1 = [1, 2, 5, 2, 3, 1, 4, 5, 1, 2, 6, 4]
new_list = []
[new_list.append(n) for n in list1 if n not in new_list]
print(new_list)
#Output
[1, 2, 3, 4, 5, 6]

5. Removing duplicates in a list using unique() method from Panda Module

In the panda module, there is a unique() method that returns only unique elements from the given list.

import pandas as pd
list1 = [1, 2, 5, 3, 1, 4, 3, 5, 1, 2, 6] 
new_List = pd.unique(list1).tolist() 
print(new_List)
#Output
[1, 2, 3, 4, 5, 6]

6. Using enumerate() and List comprehension

Enumerate can also be used for removing duplicates when used with a list comprehension.

list1 = [1, 5, 2, 2, 3, 1, 3, 4, 5, 1, 2, 6] 
new_list = [i for j, i in enumerate(list1) if i not in list1[: j] ] 
print(list(new_list))
#Output
[1, 2, 3, 4, 5, 6]

Conclusion

In conclusion, now, you may know "how to remove duplicates from a list in Python?". There are different ways but the count() method is the best in accordance with the programming efficiency of the computer.

Try the codes by yourself!

Check out the original article at:

More content at plainenglish.io. Sign up for our free weekly newsletter. Get exclusive access to writing opportunities and advice in our community Discord.