www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Need help with Removing Duplicates from Array

reply sonal13 <singhalishaas60 gmail.com> writes:
Hey everyone,

I've been encountering some trouble while trying to remove 
duplicates from an array, and I was hoping someone could help me 
out. I came across a tutorial that explains how to remove 
duplicates from an array. Here's the [Remove Duplicates from 
Array 
Tutorial](https://www.interviewbit.com/blog/remove-duplicates-from-array/).

I followed the steps outlined in the tutorial, but I seem to be 
getting unexpected results. Instead of removing duplicates, the 
array remains unchanged. I'm not sure what I'm doing wrong here.

Here's the code snippet I'm using:

```python
def remove_duplicates(arr):
     unique_array = list(set(arr))
     return unique_array
```


I've tried this code with different arrays, but the duplicates 
are not getting removed. Can someone please help me understand 
what I'm missing or suggest an alternative approach to achieve 
this?

I would greatly appreciate any guidance or advice on this issue. 
If you have any other resources or tutorials that provide a 
solution to removing duplicates from an array, please feel free 
to share them as well.

Thank you all in advance.
Jun 27 2023
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 6/27/23 8:13 AM, sonal13 wrote:

 Hey everyone,
 
 I've been encountering some trouble while trying to remove duplicates 
 from an array, and I was hoping someone could help me out. I came across 
 a tutorial that explains how to remove duplicates from an array. Here's 
 the [Remove Duplicates from Array 
 Tutorial](https://www.interviewbit.com/blog/remove-duplicates-from-array/).
 
 I followed the steps outlined in the tutorial, but I seem to be getting 
 unexpected results. Instead of removing duplicates, the array remains 
 unchanged. I'm not sure what I'm doing wrong here.
 
 Here's the code snippet I'm using:
 
 ```python
 def remove_duplicates(arr):
      unique_array = list(set(arr))
      return unique_array
 ```


This is a D forum, not python.

So while you might find help here, it likely will be in the form of D code.

This also is not the learn forum, please use that if you have further 
questions about D.

How I'd do it:

```d
arr = arr.sort.uniq.array;
```

-Steve
Jun 27 2023
parent reply Sergey <kornburn yandex.ru> writes:
On Tuesday, 27 June 2023 at 12:31:27 UTC, Steven Schveighoffer 
wrote:

 On 6/27/23 8:13 AM, sonal13 wrote:

 Hey everyone,
 


 This also is not the learn forum, please use that if you have 
 further questions about D.
 -Steve


To make this post more “general”, we could complain a bit, that D 
doesn’t have “set” data structure :) we could emulate it with AA 
- but it is meh ..

And correct me if I’m wrong asymptotically usage of set should be 
more effective than sorting and then removing duplicates :P
Jun 27 2023
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 6/27/23 8:45 AM, Sergey wrote:

 On Tuesday, 27 June 2023 at 12:31:27 UTC, Steven Schveighoffer wrote:

 On 6/27/23 8:13 AM, sonal13 wrote:

 Hey everyone,


 This also is not the learn forum, please use that if you have further 
 questions about D.


 
 To make this post more “general”, we could complain a bit, that D 
 doesn’t have “set” data structure :) we could emulate it with AA - but 
 it is meh ..


std.container.rbtree.RedBlackTree is a set.


 
 And correct me if I’m wrong asymptotically usage of set should be more 
 effective than sorting and then removing duplicates :P


Set insertion is likely more expensive than sorting -- it uses 
allocations for every insertion. Sorting is done in-place. In fact, I 
could have avoided the extra allocation of a new array as well, but it's 
easier to just write `.array`.

Algorithmically, if you use a hashset, the complexity is supposed to be 
O(1) for insertion, but that is best-case. And quicksort is pretty fast 
too, and likely has better cache coherency.

I'd say it's something you should measure for the use cases you are 
thinking. Either one may be faster in certain cases.

-Steve
Jun 27 2023