Does the text analytics toolbox allow users to test out-of-sample perplexity with LDA?

1 次查看（过去 30 天）
Stephen Bruestle
Stephen Bruestle 2018-10-15
评论： Stephen Bruestle 2018-11-30
I want to create two samples from my data: one for training and one for testing. Then I want to fit the LDA model using the training sample. Then I want to test the preplexity of the test sample using the fitted model. Is this possible with the text analytics toolbox?

请先登录，再回答此问题。

采纳的回答

Christopher Creutzig
The second output of logp gives you the perplexity.
txt = extractFileText('sonnets.txt');
sonnets = split(txt,[newline newline]);
sonnets = sonnets(5:2:end);
td = tokenizedDocument(sonnets);
bow = bagOfWords(td(1:50));
mdl = fitlda(bow,5,'Verbose',0);
[~,perpl] = logp(mdl, encode(bow,td(51:53)))
% perpl = 337.4999

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Text Analytics Toolbox 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by