Is there a way to automatically calculate the batch mean standard error of the training perplexity of fitlda?

Question

Stephen Bruestle 2019-1-20

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/440581-is-there-a-way-to-automatically-calculate-the-batch-mean-standard-error-of-the-training-perplexity-o

评论： Wei Wong 2019-7-26

It is standard practice to calculate the standard error of the training perplexity from an algorithm like fitlda using the method of batch means. See Jones, et al (2006) for detail.

I can do this calculation by hand using fitlda's verbose output. However, the verbose output only gives the training perplexity out to 3 digits. It would be much easier (for me) and more accurate if there were a way to do this automatically.

Is there a way to automatically calculate the batch mean standard error of the training perplexity of fitlda?

Reference:

Jones, Galin L, Murali Haran, Brian S Caffo, and Ronald Neath, "Fixed-width output analysis for Markov chain Monte Carlo," Journal of the American Statistical Association, 2006, 101 (476), 1537-1547.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Stephen Bruestle 2019-1-24

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/440581-is-there-a-way-to-automatically-calculate-the-batch-mean-standard-error-of-the-training-perplexity-o#answer_357842

在 MATLAB Online 中打开

I answered this question myself.

While I have found no precise way to calculate batch mean standard error, I have found that you can get precise measurements of perplexity in "mdl.FitInfo.History.Perplexity"

Therefore to calculate batch mean standard error:

% Input Initial Burn
burn = 100;
% Calculate Batch Size (b), number of batches (a), and true burn (whatever is left from a*b)
Matrix = mdl.FitInfo.History.Perplexity;
[m,n] = size(Matrix);
b = floor(sqrt(m-burn));
a = floor((m-burn)/b);
burn = m - a*b;
% Burn initial values
Matrix(1:burn,:)=[];
% Create Groups (G)
G = [];
for t=1:a
  temp=t*ones(b,1);
  G=[G;temp];
end
% Calculate Batch Means
Y = splitapply(@mean,Matrix,G);
% Calculate Batch Mean Standard Error (stdev)
Ybar = mean(Y) * ones(a,1)
Err = Y-Ybar;
VAR = b/(a-1) * sum(Err.^2);
stdev = sqrt(VAR);