How do I find the duplicates in a list and create another list with them?

To remove duplicates use set(a). To print duplicates, something like:

a = [1,2,3,2,1,5,6,5,5,5]

import collections
print([item for item, count in collections.Counter(a).items() if count > 1])

## [1, 2, 5]

Note that Counter is not particularly efficient (timings) and probably overkill here. set will perform better. This code computes a list of unique elements in the source order:

seen = set()
uniq = []
for x in a:
    if x not in seen:
        uniq.append(x)
        seen.add(x)

or, more concisely:

seen = set()
uniq = [x for x in a if x not in seen and not seen.add(x)]

I don’t recommend the latter style, because it is not obvious what not seen.add(x) is doing (the set add() method always returns None, hence the need for not).

To compute the list of duplicated elements without libraries:

seen = set()
dupes = []

for x in a:
    if x in seen:
        dupes.append(x)
    else:
        seen.add(x)

or, more concisely:

seen = set()
dupes = [x for x in a if x in seen or seen.add(x)]

If list elements are not hashable, you cannot use sets/dicts and have to resort to a quadratic time solution (compare each with each). For example:

a = [[1], [2], [3], [1], [5], [3]]

no_dupes = [x for n, x in enumerate(a) if x not in a[:n]]
print no_dupes # [[1], [2], [3], [5]]

dupes = [x for n, x in enumerate(a) if x in a[:n]]
print dupes # [[1], [3]]

Related Posts:

  1. Removing duplicates in lists
  2. How do I find the duplicates in a list and create another list with them?
  3. How do I check if there are duplicates in a flat list?
  4. Difference between del, remove, and pop on lists
  5. How can I reverse a list in Python?
  6. Python – TypeError: ‘int’ object is not iterable
  7. TypeError: list indices must be integers or slices, not str
  8. IndexError: list index out of range and python
  9. Python – TypeError: ‘int’ object is not iterable
  10. Finding the index of an item in a list
  11. Finding the index of an item in a list
  12. Accessing the index in ‘for’ loops?
  13. What does the list() function do in Python?
  14. List changes unexpectedly after assignment. Why is this and how can I prevent it?
  15. Why does this iterative list-growing code give IndexError: list assignment index out of range?
  16. TypeError: ‘float’ object is not subscriptable
  17. Is there a short contains function for lists?
  18. appending list but error ‘NoneType’ object has no attribute ‘append’
  19. Check if something is (not) in a list in Python
  20. Python: finding an element in a list [duplicate]
  21. How to avoid “RuntimeError: dictionary changed size during iteration” error?
  22. Writing a list to a file with Python
  23. How do I concatenate two lists in Python?
  24. if else in a list comprehension
  25. How can I randomly select an item from a list?
  26. Fastest way to check if a value exists in a list
  27. How can I count the occurrences of a list item?
  28. TypeError: ‘list’ object cannot be interpreted as an integer
  29. Finding median of list in Python
  30. Get list from pandas dataframe column or row?
  31. Finding and replacing elements in a list
  32. ValueError: max() arg is an empty sequence
  33. Remove list from list in Python
  34. python – if not in list
  35. How can I compare two lists in python and return matches
  36. Replace values in list using Python
  37. Append integer to beginning of list in Python
  38. List comprehension on a nested list?
  39. Union of two lists in Python
  40. Python: Array v. List
  41. Get unique values from a list in python
  42. ‘list’ object has no attribute ‘shape’
  43. Writing a list to a file with Python
  44. How do I check if a list is empty?
  46. Understanding slice notation
  47. Add list to set?
  48. What’s the idiomatic syntax for prepending to a short python list?
  49. TypeError: unsupported operand type(s) for -: ‘list’ and ‘list’
  50. Converting Dictionary to List?
  51. Python Sets vs Lists
  52. AttributeError: ‘list’ object has no attribute ‘replace’ when trying to remove character
  53. How to deep copy a list?
  54. Python return list from function
  55. How to check if a string is a substring of items in a list of strings?
  56. Whats the difference between ‘for x in list:’ and ‘for x in list[:]:’
  57. ‘list’ object has no attribute ‘shape’
  58. How do I check if a list is empty?
  59. IndexError: list index out of range and python
  60. Pop index out of range
  61. Print list without brackets in a single row
  62. How to append multiple values to a list in Python
  63. How to iterate through two lists in parallel?
  64. How to remove \n from a list element?
  65. How to read a text file into a list or an array with Python
  66. Map to List error: Series object not callable
  67. how to create a list of lists
  68. Python assigning multiple variables to same value? list behavior
  69. How can I create an array/list of dictionaries in python?
  70. How to sort a list/tuple of lists/tuples by the element at a given index?
  71. Append a tuple to a list – what’s the difference between two ways?
  72. Defining lists as global variables in Python
  73. How can I remove all instances of an element from a list in Python?
  74. Create an empty list in Python with certain size
  75. How to multiply all integers inside list
  76. How to concatenate items in a list to a single string?
  77. What’s the difference between lists enclosed by square brackets and parentheses in Python?
  78. Python 2: AttributeError: ‘list’ object has no attribute ‘strip’
  79. How to make a set of lists
  80. Appending to a list gives ‘int’ object has no attribute ‘append’
  81. Python data structure sort list alphabetically
  82. TypeError: unsupported format string passed to list.__format__
  83. How to remove square brackets from list in Python?
  84. How to create a numpy array of lists?
  85. Logical indexing with lists
  86. How does str(list) work?
  87. Element-wise addition of 2 lists?
  88. creating a reverse method for a python list from scratch
  89. Turn a single number into single digits Python
  90. How to create a list of objects?
  91. Python cant convert ‘list’ object to str error [closed]
  92. In Python, is there an elegant way to print a list in a custom format without explicit looping?
  93. Circular list iterator in Python
  94. Determine if 2 lists have the same elements, regardless of order? [duplicate]
  95. Python copy files to a new directory and rename if file name already exists
  96. Building a list inside a list in python
  97. Writing Python lists to columns in csv
  98. Create a list with initial capacity in Python
  99. Python initializing a list of lists
  100. Does Python have an immutable list?

Leave a Comment