Hi Nazila,
To calculate the joint entropy between vectors ‘x’ and ‘y’, you can refer to the steps given below:
- Compute the joint probability matrix ‘p’ for vector ‘x’ and vector ‘y’. If you don't have the probability matrix, You can use the MATLAB ‘histcounts2’ function, to generate the probability matrix.
[p, ~, ~] = histcounts2(X, Y, 'Normalization', 'probability');
- Flatten the probability matrix.
p= p(:);
- Remove the zero probabilities.
p(p==0)=[];
- Calculate the Entropy Using Shannon's Entropy Formula.
H = -sum(p .* log2(p));
To calculate the entropy of variable ‘x’ you can follow the approach described above.
For more information on entropy calculation, please follow the MATLAB community discussions.
- https://www.mathworks.com/matlabcentral/answers/436929-how-to-find-entropy-of-x
- https://www.mathworks.com/matlabcentral/answers/42142-entropy-of-a-signal
For more information on MATLAB ‘histcounts2’ function, please refer to the MATLAB documentation linked below.
Hope this helps.
