Implement the "total variation distance" (TVD) in Matlab

Question

Sim 2023-7-3

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1991183-implement-the-total-variation-distance-tvd-in-matlab

编辑： Bruno Luong 2023-8-4

采纳的回答： Bruno Luong

在 MATLAB Online 中打开

I am trying to implement the Total variation distance of probability measures (TVD) in Matlab.

Would it be correct to use the max function, in order to calculate the "supremum" of the TVD equation (here below)?

My attempt:

% Input
A =[     0.444643925792938         0.258402203856749
         0.224416517055655         0.309641873278237
        0.0730101735487732         0.148209366391185
        0.0825852782764812        0.0848484848484849
        0.0867743865948534        0.0727272727272727
        0.0550568521843208        0.0440771349862259
       0.00718132854578097        0.0121212121212121
       0.00418910831837223        0.0336088154269972
       0.00478755236385398        0.0269972451790634
       0.00359066427289048       0.00110192837465565
       0.00538599640933573       0.00220385674931129
      0.000598444045481747                         0
       0.00299222022740874       0.00165289256198347
                         0                         0
       0.00119688809096349      0.000550964187327824
                         0      0.000550964187327824
       0.00119688809096349      0.000550964187327824
                         0      0.000550964187327824
                         0      0.000550964187327824
      0.000598444045481747                         0
      0.000598444045481747                         0
                         0                         0
                         0      0.000550964187327824
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0      0.000550964187327824
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
                         0                         0
       0.00119688809096349      0.000550964187327824];
P   = A(:,1);
Q   = A(:,2);
% Total variation distance (of probability measures)
d = max(abs(P-Q))
d = 0.1862

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Bruno Luong 2023-8-4

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1991183-implement-the-total-variation-distance-tvd-in-matlab#answer_1283367

编辑：Bruno Luong 2023-8-4

在 MATLAB Online 中打开

Supremum is very often implemented by max, since one can only list or compute a finite set on computer.

However your formula d = max(abs(P-Q)) is not correct to compute TVD.

According to this wiki page; correct formula is given bellow "When Ω is countable"

d = 0.5 * norm(P-Q,1)

or

d = 0.5 * sum(abs(P-Q));

8 个评论
显示 6更早的评论隐藏 6更早的评论

Sim 2023-8-4

编辑：Sim 2023-8-4

在 MATLAB Online 中打开

OK... so, basically, what I wrote initially i.e.

d = max(abs(P-Q))

was not fully correct, right?

I tried to compare all your code to what I wrote initially, and there is a small difference between what you did and what I wrote initially:

% Generate random test discrete probability density (pdf) P and Q
n = 5;
P=rand(1,n); P=P/sum(P);
Q=rand(1,n); Q=Q/sum(Q);
% Compute TVD using definition
n = length(P);
b = logical(dec2bin(0:2^n-1)-'0');
d = zeros(1,size(b,1));
for k=1:size(b,1)
    bk = b(k,:);
    Pa = sum(P(bk));
    Qa = sum(Q(bk));
    dPaQa = abs(Pa-Qa);
    d(k)= dPaQa;
end
dPQ = max(d)                 % <-- (1) First equation for TVD (from Wikipedia's Definition)
dPQ = 0.4406
dFormula = 0.5 * norm(P-Q,1) % <-- (2) Second equation for TVD (from Wikiperida's Properties)
dFormula = 0.4406
d_Sim = max(abs(P-Q))        % <-- what I wrote initially
d_Sim = 0.2920

Final message to future readers: What I wrote initially is not correct. Please use the @Bruno Luong's code! :-)

Bruno Luong 2023-8-4

编辑：Bruno Luong 2023-8-4

在 MATLAB Online 中打开

Don't use the brute force implementation of the initial definition for any discrete pdf with more than 20 values (n = cardinal of Omega), rather use

dFormula = 0.5 * norm(P-Q,1)

The for-loop I made is just to illustrate the correctness of the formula. Just like no-one would computes the determinant of matrix 30 x 30 using Leibniz formula.

Sim 2023-8-4

在 MATLAB Online 中打开

Ah ok..great..!! Many many thanks!

Then, I will use:

dFormula = 0.5 * norm(P-Q,1)

请先登录，再进行评论。

Answer 2

Debadipto 2023-8-4

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1991183-implement-the-total-variation-distance-tvd-in-matlab#answer_1283242

Hi Sim,

Upon searching, I found the exact question being asked on stackoverflow (I'm assuming it was posted by you only), where somebody has already answered the question. I am attaching the link to that answer for future reference:

max - Implement the "Total variation distance of probability measures" in Matlab - Stack Overflow