How can I speed up my script? I'm using loops and contains function and I have to screen 14K X 167K variables

Question

Alan Cesar Pilon Miro 2018-4-24

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/397056-how-can-i-speed-up-my-script-i-m-using-loops-and-contains-function-and-i-have-to-screen-14k-x-167k

评论： Guillaume 2018-4-25

Hello, I'm doing text mining in attempt to organize my data. I have a table with more then 200K rows and 12 columns and I want extract some information from one of columns. Indeed, I'm looking for names that match with my reference table (approx. 14K names). For that, I'm using contains function. For make this search, I'm using two loops. First to lock one of 14K names and second for look for this name in the 200K rows. This takes a very long time. Could help to speed up my script? Thanks

Here I show you the code:

if true
  % code
1st loop (reference name table)
for k=2:14045;
    clear test
    clear Genr
    test=DNP(k,10);
    virgula=',';
    Space= ' ';
    Genr=Space+test+Space;
 Second loop (my raw table with more than 200K rows)
    for i=15001:16000;
        clear Presence
        clear A
        clear B
        clear C
        BiolSource=DNP(i,3);
        Presence=contains(BiolSource, Genr, 'IgnoreCase',true);
            if Presence ==1;
                A=DNP(i,13);
                B=DNP(k,11);
                    DNP(i,13)=A+virgula+Space+test+Space+B;
                    C=DNP(i,13);
                     DNP(i,13)=erase(C,"0, ");
            end
    end
end

5 个评论
显示 3更早的评论隐藏 3更早的评论

Guillaume 2018-4-24

编辑：Guillaume 2018-4-24

So, to be clear, you want to identify in column C of Ask3 which term is the genera. All possible genera are stored in column A of Ask2?

Assumption: there is always one and only one genera in column C.

Alan Cesar Pilon Miro 2018-4-24

Yes, I want to find one of genera present in A ASk2 in Column C. Can there is more than one genus by row of column C

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Guillaume 2018-4-24

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/397056-how-can-i-speed-up-my-script-i-m-using-loops-and-contains-function-and-i-have-to-screen-14k-x-167k#answer_316957

在 MATLAB Online 中打开

reference = readtable('MATLAB ASk2.xls', 'ReadVariableNames', false, 'Range', 'A:B');
raw = readtable('MATLAB ASk3.xls');
genera = lower(reference.Var1);  %convert everything to lower case for easier comparison
matched_genera = rowfun(@(t) {genera(ismember(genera, strsplit(lower(t))))}, raw, 'InputVariables', 'Text', 'ExtractCellContent', true, 'OutputVariableNames', 'matched');

Each row of matched_genera is a cell array contain 0, 1 or more of the genera found in 'Text' (case insensitive). You can concatenate that with the original table if you wish:

newraw = [raw, matched_genera]

2 个评论
显示无隐藏无

Alan Cesar Pilon Miro 2018-4-24

Could you include column 2 (family) from the reference table as well in the newraw?

Guillaume 2018-4-25

在 MATLAB Online 中打开

The easiest way to do that is to create a separate m file for the rowfun function:

In its own match_raw.m file:

function [matched_genera, family] = match_raw(raw_row, genera_lower, family)
     ismatch = ismember(genera_lower, strsplit(lower(raw_row)));
     matched_genera = {genera_lower(ismatch)};
     family = {family(ismatch)};
end

The rowfun call then becomes:

matches = rowfun(@(t) match_raw(t, genera, reference.Var2), raw, 'InputVariables', 'Text', 'ExtractCellContent', true, 'OutputVariableNames', {'matched_genera', 'family'})

请先登录，再进行评论。

How can I speed up my script? I'm using loops and contains function and I have to screen 14K X 167K variables

5 个评论
显示 3更早的评论隐藏 3更早的评论

采纳的回答

2 个评论
显示无隐藏无

更多回答（0 个）

另请参阅

类别

标签

Community Treasure Hunt

How can I speed up my script? I'm using loops and contains function and I have to screen 14K X 167K variables

5 个评论 显示 3更早的评论隐藏 3更早的评论

采纳的回答

2 个评论 显示 无隐藏 无

更多回答（0 个）

另请参阅

类别

标签

Community Treasure Hunt

5 个评论
显示 3更早的评论隐藏 3更早的评论

2 个评论
显示无隐藏无