Cosine similarity is a metric used to measure the similarity between two vectors in a multidimensional space. It is widely used in various fields including natural language processing, information retrieval, and machine learning.
I’ll be sure to break it down for you.
To make it simple consider two documents. If one document mentions Tesla many times but does not mention Ford, then we can infer that the document is about Tesla and vice versa.
Consider these two documents.
How can we tell the similarity of these documents?
The first document is about Tesla and the second one is about Ford. Now count the keyword in each document, for this various methods like Regex are used.
Now you can just plot the vector in the graph.
Consider the angle between these vectors. The cosine value of this angle will give the similarity between the documents. Cosine similarity produces a value between -1 and 1
Similarity increases as the value moves towards 1.
When the angle between vectors increases similarity decreases
Mathematically,
Given two vectors A and B, cosine similarity calculates the cosine of the angle between them. It’s a measure of orientation rather than magnitude.
The formula for cosine similarity between two vectors A and B is:
In text mining and natural language processing, cosine similarity is often used to compare the similarity between documents represented as vectors, where each dimension represents a term frequency or some other measure of word importance. It helps in tasks like document retrieval, clustering, and recommendation systems.
I hope you all now have a clear understanding of Cosine similarity and the reason why we study trigonometry in schools.
See you in the next one👋.
