As is often the case, sometimes you only want unique values in your array.

There are several ways to go about this in JavaScript depending how deep you want to go.

Method 1: The IE11 way, `Array.filter()`

Unfortunately, without polyfills, the code is little verbose.

Due to the lack of features in "old" JS, we have to double iterate over the array. Once over the entire array to remove any dupes and within each iteration we need to loop over the loop again and figure out if the element already exists up to that point in the array, if it does then we need to return `false` to the filter function to remove the element.

    var myArr = [1, 2, 3, 4, 4, 5, 6, 7];

    var deduped =  myArr.filter(function (el, i, arr) {
        var hasDupes = arr
            .slice(0, i)
            .reduce(function (acc, rEl) {
                if (rEl === el) return false
                return acc
            }, true)

        return hasDupes
    })
    // deduped = [1,2,3,4,5,6,7]

Method 2: The slightly more modern way (arrow funcs, > IE1)

Thanks to arrow functions, we can clean this up a little bit:

    const deduped =  myArr.filter((el, i, arr) => {
        const hasDupes = arr
            .slice(0, i)
            .reduce((acc, rEl) => {
                if (rEl === el) return false
                return acc
            }, true)

        return hasDupes
    })
    // deduped = [1,2,3,4,5,6,7]

Method 3: The modern way (ES6 `Set`)

Thanks to ES6 `Set` and we can reduce this code to just a single line:

    const deduped = Array.from(new Set(myArr))

Luckily Sets and assorted array methods can be easily polyfilled.

That's great, but you're not always going to get an array of primitive values...

So the likelihood of you having to dedupe an array of plain values is slim-ish. What's more likely is having to dedupe an array of objects, say a list of articles (objects) from a API response. However, any of the methods above are only going to remove duplicates by reference not by value eg it'll only class two elements as a duplicate if they are exactly the same object.

So, if you're getting your values from an API then the objects are unlikely to be the exact same by reference, only by value. For more info on this look here.

So how do we dedupe them? Well, in an ideal world each object (read article) should have a unique ID we can check. So then it becomes just a simple case of adding a `map()` stage to our comparison function and removing duplicated based on the values of a specific key.

const myArr = [
    {
        id: 1,
        thing: 'pies',
        another: 'splod',
    },
    {
        id: 2,
        thing: 'pies',
        another: 'mlel'
    },
    {
        id:3,
        thing: 'a small man fashioning a cayak from a log',
        another: 'The Magna Carta',
    },
    {
        id: 2,
        thing: 'pies',
        another: 'mlel'
    },
]

const createDedupeFilter = key => (el, i, arr) => {
    const hasDupes = arr
        .slice(0, i)
        .map(obj => obj[key])
        .reduce((acc, value) => {
            if (value === el[key]) return false
            return acc
        }, true)

    return hasDupes
}

const dedupeFilter = createDedupeFilter('id')

const deduped = myArr.filter(dedupeFilter)

Obviously the code above makes some pretty hefty assumptions:

Each element is as unique as it's id
the id field exists
there is no error checking.

However, you can hopefully grok the basic premise of how it works.

Hope this helps!

Any questions / suggestions / want to tell me I'm completely wrong? hit me up on twitter.

Getting Started with creative coding in Rust and Nannou.

101 Ways to dedupe an array in JS (well 4).

On the importance of building silly things for the hell of it.

Well, hello.