find duplicated rows in matlab without for loop

Question

Mina Mino 2023-7-21

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1999193-find-duplicated-rows-in-matlab-without-for-loop

移动：Matt J 2023-7-21

Hello Friedns,

I have a very large matrix with 2 columns. I need to find the location of duplicated rows (the position of them) . However, I don't want to solve this problem with a for loop because I've tried it before (see the attached code) and it takes a long time. I'm looking for an alternative way to do this. I would be grateful if you could suggest me an idea.

Best,

Mina

x  = [File(:,1) File(:,2)];
Grid=unique(x,'rows');
for j=1:length(DD)
    idx=find(day_of_year==DD(j));
    File2=File(idx,:);
    for g=1:length(Grid)
        [index1]    = (ismember(File2(:,[1 2]),Grid(g,:),'rows'));
        idx2=find(index1==1);
        Total=[Total;Grid(g,1) Grid(g,2) DD(j) mean(File2(idx2,3)) mean(File2(idx2,4)) mean(File2(idx2,5)) mean(File2(idx2,6))];
    end
end

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Matt J 2023-7-21

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1999193-find-duplicated-rows-in-matlab-without-for-loop#answer_1276868

编辑：Matt J 2023-7-21

在 MATLAB Online 中打开

[~,I]=unique(x,'rows');
locations=setdiff(1:height(x),I) %locations of duplicate rows

2 个评论
显示无隐藏无

Mina Mino 2023-7-21

Hi Matt J,

Many thanks for your help! It seems that for each line your code is only finding the position of one of the duplicated lines. However, there are more than one duplicates for each row. How can I find all duplicate rows of each row?

Thanks in advance for your answer and time.

Matt J 2023-7-21

编辑：Matt J 2023-7-21

在 MATLAB Online 中打开

It seems that for each line your code is only finding the position of one of the duplicated lines.

I don't think so. It should return the indices of all rows that have been seen before. As you can see below, the final locations list includes all rows except for 1 and 3, which is where a new row is encountered.

x=[ 1 2;
    1 2;
    0 4;
    1 2;
    0 4
    0 4];
[~,I]=unique(x,'rows');
locations=setdiff(1:height(x),I) %locations of duplicate rows
locations = 1×4
     2     4     5     6

请先登录，再进行评论。

Answer 2

Walter Roberson 2023-7-21

2
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1999193-find-duplicated-rows-in-matlab-without-for-loop#answer_1276903

移动：Matt J 2023-7-21

在 MATLAB Online 中打开

The third output of unique gives the "group number" for each entry. There are different ways of handling that. one of ways is

[unique_rows, ~, ic] = unique(x,'rows');
appears_in_rows = accumarray(ic, (1:size(x,1)).', [], @(v) {v});
T = table(unique_rows, appears_in_rows);

This would create a table in which the first variable is each unique row, and the second variable is a cell array of row indices that are that unique row. The cell array will always have at least one entry, but might have more.