Remove Duplicates From JavaScript Array
In this tutorial, you will learn how to remove duplicates from JavaScript array. We will look at various different ways to remove duplicates from the array with examples.
We will discuss 5 best methods to remove duplicates from the array and at the end, we will compare the performance of all the methods, so you can choose the best method for you.
- Using Set() Constructor
- Using filter() Method
- Using reduce() Method
- Using map() Method
- Performance Of Different Methods
- Conclusion
Table Of Contents
1. Using Set() Constructor
A set is a collection of unique values in JavaScript. A value in a set occurs only once.
This can be quickest and easiest way to remove duplicates from the array. You just have to convert the array to a set and then convert the set back to an array.
Pass the array to the Set() constructor and it will remove the duplicate values from the array.
Example
var arr = [1, 2, 1, 4, 3, 3, 1, 2, 4];
// create a set from array
var unique = new Set(arr);
// convert set to array
var uniqueArray = Array.from(unique);
console.log(uniqueArray); // [1, 2, 4, 3]The duplicates from the array are removed when a set is created. You have to convert set to array to get an array.
Set includes the first unique element and removes the next duplicate elements. The first unique '4' came before the first unique '3' so it includes '4' before '3' in the set.
2. Using filter() Method
The filter() method is used to remove any element from the array for which the return value of the function is false.
To remove duplicates from the array, create a function that returns true only for the first element and false for the rest of the elements.
You can use the indexOf() method here. It returns the index of the first occurrence of the element in the array.
Example
var arr = [1, 2, 1, 4, 3, 3, 1, 2, 4];
// using filter method to remove duplicates
var unique = arr.filter(function(element, index) {
return arr.indexOf(element) === index;
});
console.log(unique);Code Explanation: We have access to the index of the current element of the array within the filter function. We can use this index to check if the element is the first occurrence or not using the indexOf() method. If it is the first occurrence, it will return true and if it is not the first occurrence, it will return false.
3. Using reduce() Method
The reduce() method is used to reduce the array to a single value by executing a reducer function on each element of the array, resulting in a single output value.
We can use this to reduce the given array in an array that has no duplicate values.
The method takes an initial value to work with, so we give an empty array [] as the initial value.
Syntax:
array.reduce(function(accumulator, currentValue), initialValue)
array.reduce(function(accumulator, currentValue, currentIndex, array), initialValue)We are going to use only 2 arguments in the reducer function: accumulator and currentValue.
- The accumulator is the value returned by the last execution of the reducer function.
- currentValue is the current element of the array.
3.1. reduce() Method with indexOf() Method
Start with an empty array and start adding the current element to the array if it is not already present in the array. Then return the array.
Example
var arr = [1, 2, 1, 4, 3, 3, 1, 2, 4];
// using reduce method to remove duplicates
var unique = arr.reduce(function(acc, current) {
if (acc.indexOf(current) === -1) {
acc.push(current);
}
return acc;
}, []);
console.log(unique);Code Explanation: Execution starts with an empty array as the initial value. Then we check if the accumulator (initialValue) contains the current element. If it does not contain the current element, we add it to the accumulator array. Then we return the accumulator array. Then the same process is repeated for all the elements in the array.
If it contains the current element, we do not add it to the accumulator array. Then we return the accumulator array.
3.2. reduce() Method with includes() Method
Start with an empty array and start adding the current element to the array if it is not already present in the array. Then return the array.
Example
var arr = [1, 2, 1, 4, 3, 3, 1, 2, 4];
// using reduce method to remove duplicates
var unique = arr.reduce(function(acc, current) {
return acc.includes(current) ? acc : [...acc, current];
}, []);
console.log(unique);Code Explanation: This is the same as above but we are using the includes() method to check if the accumulator array contains the current element. The rest we are doing is the same as above.
4. Using map() Method
This is not the very best way to remove duplicates from an array but eventually, it can be used.
We can do a very simple thing we can map the array and return the item if it is not present in the array and if it is present in the array return null after that, we filter the array and remove the null values.
Example
var arr = [1, 2, 1, 4, 3, 3, 1, 2, 4];
// using map method to remove duplicates
var unique = arr.map((item, index) => {
return arr.indexOf(item) === index ? item : null;
}).filter(item => item !== null);
console.log(unique);Performance Comparison All Discussed Methods
We can compare the performance of all the methods discussed above.
The graph below shows the performance of all the methods based on the number of operations and time taken to execute.
The method is executed for 100, 1000, 10000, 100000, 1000000, and 10000000 operations, and the time taken to execute is displayed in the graph.
Here is the program to compare the performance of all the methods discussed above.
Each method can be executed for a different number of operations. Since each execution gives a different result, so we ran each method 10 times for each number of operations.
Example
// remove duplicates from array
let arr = [1, 2, 1, 4, 3, 3, 1, 2, 4];
var operation = 1000, repeat = 10;
var avg = 0, sum = 0;
var start, end;
// check the performance of each method
// method 1
for (let i = 0; i < repeat; i++) {
start = new Date().getTime();
for (let i = 0; i < operation; i++) {
let unique = [...new Set(arr)];
}
end = new Date().getTime();
sum += end - start;
}
avg = sum / repeat;
console.log(`Method 1: ${avg}ms`);
// method 2
for (let i = 0; i < repeat; i++) {
start = new Date().getTime();
for (let i = 0; i < operation; i++) {
let unique = arr.filter((item, index) => {
return arr.indexOf(item) === index;
});
}
end = new Date().getTime();
sum += end - start;
}
avg = sum / repeat;
console.log(`Method 2: ${avg}ms`);
// method 3.1
for (let i = 0; i < repeat; i++) {
start = new Date().getTime();
for (let i = 0; i < operation; i++) {
let unique = arr.reduce((acc, curr) => {
if (acc.indexOf(curr) === -1) {
acc.push(curr);
}
return acc;
}, []);
}
end = new Date().getTime();
sum += end - start;
}
avg = sum / repeat;
console.log(`Method 3.1: ${avg}ms`);
// method 3.2
for (let i = 0; i < repeat; i++) {
start = new Date().getTime();
for (let i = 0; i < operation; i++) {
let unique = arr.reduce((acc, curr) => {
return acc.includes(curr) ? acc : [...acc, curr];
}, []);
}
end = new Date().getTime();
sum += end - start;
}
avg = sum / repeat;
console.log(`Method 3.2: ${avg}ms`);
// method 4
for (let i = 0; i < repeat; i++) {
start = new Date().getTime();
for (let i = 0; i < operation; i++) {
let unique = arr.map((item, index) => {
return arr.indexOf(item) === index ? item : null;
}).filter(item => item !== null);
}
end = new Date().getTime();
sum += end - start;
}
avg = sum / repeat;
console.log(`Method 4: ${avg}ms`);Conclusion
We have seen 5 different methods of how javascript remove duplicates from array. We have also compared the performance of all the methods.
From the Comparison of all the methods, we can see that the Set() method is the best method to remove duplicates from the array. It is the fastest method and it is also the most efficient method.