Similarity of histograms: interpretation of cosine and jaccard similarities with "pdist2"

Question

Sim 2023-6-26

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1988173-similarity-of-histograms-interpretation-of-cosine-and-jaccard-similarities-with-pdist2

编辑： Sai Teja G 2023-10-10

I would like to assess the similarity between two "bin counts" (that I previously derived through the "histcounts" function), by using the "pdist2" function:

% input

bin_counts_a = [689 430 311 135 66 67 99 23 37 19 8 4 3 4 1 3 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1];

bin_counts_b = [569 402 200 166 262 90 50 16 33 12 6 35 49 4 12 8 8 2 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 1];

% visualise the two "bin counts" vectors as bars:

bar(1:length(bin_counts_a),[bin_counts_a;bin_counts_b])

% calculation of similarities

cosine_similarity = 1 - pdist2(bin_counts_a,bin_counts_b,'cosine')

cosine_similarity = 0.9547

jaccard_similarity = 1 - pdist2(bin_counts_a,bin_counts_b,'jaccard')

jaccard_similarity = 0.0769

If the cosine similarity is close to 1, which means the two vectors are similar, shouldn't the jaccard similarity be closer to 1 as well?

2 个评论
显示无隐藏无

Dyuman Joshi 2023-6-26

"If the cosine similarity is close to 1, which means the two vectors are similar, shouldn't the jaccard similarity be closer to 1 as well?"

No, because the similarities are defined differently. Cosine similarilty is not same as Jaccard similarity.

You can check out the definitions in the More About section of the pdist2 documentation page.

Sim 2023-7-12

thanks @Dyuman Joshi :-)

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Sai Teja G 2023-8-14

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1988173-similarity-of-histograms-interpretation-of-cosine-and-jaccard-similarities-with-pdist2#answer_1287292

编辑：Sai Teja G 2023-10-10

Hi Sim,

I see that you are comparing two vectors by using ‘cosine’ and ‘jaccard’ distances between them.

They are not the same, as Jaccard Similarity considers a set of unique word lengths, while cosine similarity considers the entire sentence vector, disregarding data duplication.

Please refer the following documentation for more information on distance metrics like ‘jaccard’ and ‘cosine’ for the function ‘pdist2()’ –

https://www.mathworks.com/help/stats/pdist2.html

Hope it helps!