Checking repetition of random data

4 次查看（过去 30 天）

Student for ever 2018-1-4

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/375546-checking-repetition-of-random-data

编辑： Student for ever 2018-1-9

I heed your help please. I made a random data for example T1 = randn (1000,1); T2= randn (1000,1); .... T100=randn (1000,1); and I want check whether there is any repetition for T's if so then remove it. How can I do that ?? Thanks in advance :)

Regards, Ahmed

11 个评论
显示 9更早的评论隐藏 9更早的评论

Star Strider 2018-1-7

@Ahmed — See the documentation on rng (link), and more generally, the discussion on Generate Random Numbers That Are Repeatable (link).

Student for ever 2018-1-7

编辑：Jan 2018-1-7

Dear Jan,

I am new in matlab :), May my question is not clear, but your answer it is so close of what I want to do I think. I have 200 timeseries, which were come from parallel computation and I just want make sure that 200 are not repeated (what I mean, if I make plot for them they should give me different graphs). So, I put all the 200 timeseries as a matrix, it will be 200 column , then I just want check these columns not the same.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Jan 2018-1-4

2
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/375546-checking-repetition-of-random-data#answer_298661

编辑：Jan 2018-1-8

在 MATLAB Online 中打开

Do not create a list of variables called T1, T2, ... See https://www.mathworks.com/matlabcentral/answers/57445-faq-how-can-i-create-variables-a1-a2-a10-in-a-loop. Use a cell or multidimensional array instead.

I assume your problem is to have no repeated values inside each vector and between all vectors. Then you need 1000*100 different random numbers at first:

ready = false;
while ~ready
  Pool  = rand(1, 100000);
  ready = (length(unique(Pool)) == length(Pool));
end
T = reshape(Pool, 1000, 100);

Maybe this is faster:

ready = all(diff(sort(Pool)));

[EDITED] If all you want is to create a unique set of vectors, and randn was just an example to create test data for the forum:

[T, Idx] = unique(T, 'rows')

[EDITED] And for unique columns:

T = unique(T.', 'rows').'

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Student for ever 2018-1-8

Thank you so much

请先登录，再进行评论。

更多回答（2 个）

Birdman 2018-1-4

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/375546-checking-repetition-of-random-data#answer_298660

在 MATLAB Online 中打开

Firstly, generate random data as follows:

T=randn(1000,100);

Secondly, as Adam said, use unique function to check repetitions.

Tun=unique(T,'stable');

stable command helps to protect the initial order of values.

5 个评论
显示 3更早的评论隐藏 3更早的评论

Jan 2018-1-4

A "habit"? :-) I'd suggest to use time consuming methods only, if they are needed for the results.

Birdman 2018-1-4

It is needed for result, exactly.

请先登录，再进行评论。

John BG 2018-1-5

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/375546-checking-repetition-of-random-data#answer_298781

在 MATLAB Online 中打开

Hi Ahmed

this is John BG <mailto:jgb2012@sky.com jgb2012@sky.com>

so far, the supplied answers increase the probability to generate all-different, random Ts.

Each of the answers improves generation randomness, yet if you really want to make sure that all T sequences are different, once generated, let's say you don't really have control on the randomness of the data and the the suggested randn(1000,1) is you model, then there's no other way than comparing them by pairs.

Let be N the amount of T sequences

N=5

then all possible pairs of T sequences are

 L=combinator(N,2,'c')
 =
   2
   3
   4
   5
   3
   4
   5
   4
   5
   5

As Jan Simon mentions, sometimes it's more practical to put all data in a structure that can be indexed, instead of working with N different sequence names.

Let be T all your input Ti sequences compiled into a single matrix

T=randi([1 10],N)
T =
   2     3     9     3
   5     8    10     9
  10     3     6     3
   4     6     2     9
   6     7     2     3

Checking there are no 2 equal sequences

D=[0 0];
  for k=1:1:size(L,1)
    if  isequal(T(L(k,1),:),T(L(k,2),:))
        D=[D;L(k,:)];
    end
    end

Removing repeated sequences

    if size(D,1)>1
        D(1,:)=[];
        T(D(:,1),:)=[]; % removing one of the repeated identical pairs
    end
    T

Ahmed, I have overwritten some sequences on purpose, so the counter D shows spotted repeated sequences and these simple lines remove all repetition without losing data (when more than one repetition of same given sequence) and it works.

If you find this answer useful would you please be so kind to consider marking my answer as Accepted Answer?

To any other reader, if you find this answer useful please consider clicking on the thumbs-up vote link

thanks in advance for time and attention

John BG

<mailto:jgb2012@sky.com jgb2012@sky.com>

12 个评论
显示 10更早的评论隐藏 10更早的评论

John BG 2018-1-6

在 MATLAB Online 中打开

Checking delays for 100 strings shows that unique is the fastest option:

    N=100
      M=1000
      % T=randi([1000 9999],N,M);
      T=repmat(randi([1000 9999],1,M),N,1);
      tic
      D=[0 0];
      L=combinator(N,2,'c');
        for k=1:1:size(L,1)
          if  isequal(T(L(k,1),:),T(L(k,2),:))
            D=[D;L(k,:)];
          end
        end
        if size(D,1)>1
            D(1,:)=[];
            T(D(:,1),:)=[]; % removing one of the repeated identical pairs
        end
         toc

100: Elapsed time is 0.068783 seconds.

1000: Elapsed time is 7.953736 seconds.

single string repeated 100 times: Elapsed time is 0.112921 seconds.

      tic
      L=combinator(N,2,'c');
      D  = zeros(size(L, 1), 2);   % Pre-allocation!!!
    iD = 0;
    for k = 1:size(L,1)
      if isequal(T(L(k,1),:),T(L(k,2),:))
         iD       = iD + 1;
         D(iD, :) = L(k, :);
      end
    end
    D = D(1:iD, :);
      toc

100: Elapsed time is 0.075002 seconds.

1000: Elapsed time is 7.884542 seconds.

single string repeated 100 times: Elapsed time is 0.085103 seconds.

   tic
      L=combinator(N,2,'c');
      dup = false(size(L, 1), 2);   % Pre-allocation!!!
    for k = 1:size(L,1)
      if isequal(T(L(k,1),:),T(L(k,2),:))
         dup(k) = true;
         break;
      end
    end
    L = L(dup, :);
      toc

100: Elapsed time is 0.062778 seconds.

1000: Elapsed time is 7.863167 seconds.

single string repeated 100 times: Elapsed time is 0.030683 seconds.

      tic
      nT   = size(T, 1);
    keep = true(nT, 1);  % Pre-allocation!!!
    for i1 = 1:nT
      for i2 = i1 + 1:nT
        if isequal(T(i1, :), T(i2, :))
          keep(i1) = false;
          break;         % No need to proceed the search
        end
      end
    end
    T = T(keep, :);
      toc

100: Elapsed time is 0.068909 seconds.

1000: Elapsed time is 7.784486 seconds.

single string repeated 100 times: Elapsed time is 0.034376 seconds.

    tic
      [T2, Idx] = unique(T, 'rows');
        toc

100: Elapsed time is 0.023476 seconds.

1000: Elapsed time is 0.061907 seconds.

single string repeated 100 times: Elapsed time is 0.024031 seconds.

When increasing the amount of strings, unique outperforms any other solution, regarding time delay.

Regards

John BG

<mailto:jgb2012@sky.com jgb2012@sky.com>

Stephen23 2018-1-8

编辑：Stephen23 2018-1-8

The help clearly states that "Answers can only be accepted by someone other than the author of the question after 7 days of inactivity from the author".

Student for ever 2018-1-9

编辑：Student for ever 2018-1-9

Thanks all for helping, your comments its really useful for me. @John BG, I already accept Jan's answer.

请先登录，再进行评论。

请先登录，再回答此问题。

类别

MATLAB Language Fundamentals Data Types Numeric Types Logical

在 Help Center 和 File Exchange 中查找有关 Logical 的更多信息

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by

Checking repetition of random data

11 个评论
显示 9更早的评论隐藏 9更早的评论

采纳的回答

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

更多回答（2 个）

5 个评论
显示 3更早的评论隐藏 3更早的评论

12 个评论
显示 10更早的评论隐藏 10更早的评论

另请参阅

类别

标签

Community Treasure Hunt

Checking repetition of random data

11 个评论 显示 9更早的评论隐藏 9更早的评论

采纳的回答

1 个评论 显示 -1更早的评论隐藏 -1更早的评论

更多回答（2 个）

5 个评论 显示 3更早的评论隐藏 3更早的评论

12 个评论 显示 10更早的评论隐藏 10更早的评论

另请参阅

类别

标签

Community Treasure Hunt

11 个评论
显示 9更早的评论隐藏 9更早的评论

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

5 个评论
显示 3更早的评论隐藏 3更早的评论

12 个评论
显示 10更早的评论隐藏 10更早的评论