How do I create a term - frequency matrix that runs fast

Question

Ryan 2013-9-5

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/86457-how-do-i-create-a-term-frequency-matrix-that-runs-fast

Hello everyone! I am trying to create a term frequency matrix for a TF-IDF program. I have the code written but it runs extremely slow. My code works by finding the unique words in all of the documents, say for example

A = {'dog','cat','mouse'}

and for example two documents,

D = {'dog','cat','cat'; 'cat','mouse','mouse'}.

The code I have is:

for k = 1:n
  for j = 1:m
      seq_sum(k,j) = sum(ismember(D{k,:},A{j}));
  end
end

The output of the above example would be a matrix that looks like

seq_sum = [1 2 0 ; 0 1 2];

where k is the row size of the cell array D and j is the column size of A. I also have this written in parallel but I don't want to have to rely on parallel computing. Any help would be greatly appreciated! Oh and I guess my question is how can I improve this to run faster?

2 个评论
显示无隐藏无

Muthu Annamalai 2013-9-5

In MATLAB 13b, the new datatype 'categorical' is designed to solve this problem.

Ryan 2013-9-5

Thank you Muthu I am however working with 12b :(. Im sure I can get a copy of 13b though. Thank you for your comment.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Azzi Abdelmalek 2013-9-5

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/86457-how-do-i-create-a-term-frequency-matrix-that-runs-fast#answer_95942

编辑：Azzi Abdelmalek 2013-9-5

在 MATLAB Online 中打开

A = {'dog','cat','mouse'};
D = {'dog','cat','cat'; 'cat','mouse','mouse'};
out=zeros(size(D));
for k=1:numel(A)
  idx=ismember(D,A(k));
  out(:,k)=sum(idx,2);
end
disp(out)

%or

out=cell2mat(arrayfun(@(x) sum(ismember(D,A(x)),2),1:numel(A),'un',0))

3 个评论
显示 1更早的评论隐藏 1更早的评论

Ryan 2013-9-5

The arrayfun also works much faster than the for loop. Just in case anyone sees this and is interested. Thanks again!

Ryan 2013-9-5

Azzi I have one more question. I forgot in my code that D may not have the same column size so it could look like:

D = {1x3 ; 1x5}

Is there still a way to use the arrayfun as a double for loop? Or possible make it so D has the same column dimension so I can apply what you suggested?

请先登录，再进行评论。