There are many examples of where the hive mind can I help you deliver an optimised solution. One example of this is the collaborative filtering algorithm.
The collaborative filtering algorithm uses the behaviour of others to understand how you are likely to behave. This data then allows it to make recommendations.
Many recommendation systems, including Netflix, have used the collaborative filtering example. To provide you with the best possible film recommendations, Netflix uses collaborative filtering to identify which films you will like.
In the system, the algorithms can predict behaviour based on the behaviour of other people who like some of the same items as you.
One or something about the collaborative filtering example is that it’s pretty easy to understand.
You don’t need to master any complicated maths to be able to implement collaborative filtering in python – or even in excel!
Unlike some of the other recommendation systems we have discussed in this blog, collaborative filtering is super easy to understand and even more comfortable to implement!
I think it’s time we dive into some examples of collaborative filtering!
Let’s go
Three Powerful Collaborative Filtering Examples
Now I’m going to share with you three collaborative filtering examples that you will recognize.
Netflix
Netflix uses a collaborative filtering algorithm to recommend content to users of their streaming services.
When you first sign up you are prompted to highlight whether you like or dislike certain popular films. This rating is then used to start the recommendation algorithm.
Dating Websites
One way that dating sites match potential lovers is by using collaborative filtering. These sites will analyze previous matches and compare the matches of people who are the most similar in profile to you.
Once they have an understanding of your profile, the dating site can make a recommendation based upon collaborative filtering algorithms.
YouTube
Similar to Netflix, YouTube is another collaborative filtering algorithm when this process is used to recommend content for users to stream.
In all of the above collaborative filtering examples, the critical thing that is common among them is that they are making personality matches.
By matching your behaviour to the behaviour of people with similar personalities, likes, and dislikes, the algorithm can quickly identify things that you will enjoy.
The more it learns about your specific likes and dislikes, the better it is at making recommendations.
So how does the collaborative filtering algorithm work? Let’s take a more in-depth look.
How the collaborative filtering algorithm works
You can think of the collaborative filtering a bit like getting nearest neighbours algorithms.
What I mean by this is that the collaborative filtering algorithm matches items based on how close they are within the space.
However, the collaborative filtering algorithm is a deep neural network.
I will share the mathematics a little bit later. For now, you can think of it as one of those quiz questions where you have to map the first part (a) of a problem, to the correct answer (b)
To understand the best possible answer brackets B) for person A, you can look at how people similar to a person may have previously answered B.
Ready to get started with Machine Learning Algorithms? Try the FREE Bootcamp
Making a recommendation
Let’s take the film recommendations, for example.
Imagine you are trying to predict whether the person 1 would like film B.
You know that you have data on other people who have made a recommendation as to whether they like or dislike film B.
You can use this data and data on how the other people have a reacted to films that person 1 liked, to predict as to whether the person 1 will like film B.
This theory is the basis of the collaborative filtering algorithm.
Why you need to add bias into the collaborative filtering algorithm
One thing to remember is that some films are more popular than others. In addition to this, some people are more likely to like films than others.
Knowing these two facts, you then apply different weights and biases to the people and films that you are assessing to account for this preference.
Adding these biases allows you to make much more accurate predictions depending on the specific characteristics of the film or person that you are looking to recommend.
I think it’s time we take a look at the maths.
Below is an example of the collaborative filtering algorithm.
I have annotated it so you can see each of the points that I described above.
In this collaborative filtering example, you will see how the different biases apply to the various films. For example, you can say that the film Forest Gump has a higher bias associated with it. This bias is because commonly people will like this film even if they dislike other films.
That’s because Forest Gump is an excellent film!
It is essential in collaborative filtering, to understand the bias and the weights for the dataset so that you can make the best possible predictions.
Finally, I think it’s time to take a look at implementing a collaborative filtering example in python.
Collaborative filtering is an excellent skill for you to be able to use as it implemented in industry.
Let’s get started!
How to implement collaborative filtering in python
In this segment, I am going to show you how to implement the collaborative filtering algorithm in python.
To do this, we are going to use the Fastai library and Pytorch.
The code that you will implement is straightforward; in fact, we can do it in three steps.
Step one of the collaborative filtering algorithm in python.
The first thing you do is get the data ready and train the algorithm
In the below code you can see how the data is set up, creating a collaborative filtering databunch using fastai and creating the learner.
Once you know the learning rate you can train the algorithm.
The next code snippets are taken from the fast.AI collaborative filtering notebook.
Step 2: Understand Bias
Next step is to look at your results and see what they tell you about preferences. First, you can look at the rankings of different elements.
Then, you want to understand the bias assigned to different elements.
Step 3: Understand Weights
Finally, it is time to understand how the different elements have been weighted by the collaborative filtering algorithm.
Then you can use the data to create cool plots like this:
Now it’s your time to get practising!
Three fun datasets to practice collaborative filtering examples
These are three of my favourite collaborative filtering example data sets that I have found online.
The benefit of these datasets when practising collaborative filtering in python is that they link to applications in the real world.
It is important because it shows employers that you have a good understanding of how to use these kinds of algorithms in an industry applicable way.
And that’s it now you have three collaborative filtering examples to practice on.
What have we learned today?
I hope you have enjoyed learning all about the collaborative filtering algorithm.
To summarise what we have discussed:
- the collaborative filtering algorithm is a robust neural network used in recommendation systems
- there are many collaborative filtering examples in the real world
- implementing collaborative filtering in python is simple!
I hope you’ve enjoyed this tutorial and can’t wait to hear what you do with what you’ve learned.
Recommendation systems are always entertaining but the real fun comes in using them to predict/recommend classes for unusual edge cases that are sometimes weird but spooky accurate! 😜