Sorting big data in matlab

Question

zafar khan 2017-2-2

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/323104-sorting-big-data-in-matlab

评论： Walter Roberson 2017-2-2

I have energy data with more than 25 million rows and 3 columns, the data looks like

[meter Id         Date+time            Load
  1000            19506                 .5
  1000            19501                 .5
   .                 .                   .
  1001            19500                 .6
   .                 .                    .
  .                  .                    .
 1999              19509                 .4
  .                  .                      .
  1999              73048                   .4]

I require to sort this data such that for each meter ID i.e. 1000-1999 (all 1000 meters) i get load arranged in a time series. The first three digit of time code show the day whereas the last two digits range from 0-48 showing 24 hours every half hours. I can sort the data according to meter ID but this does not arrange it according to time series. I am struggling to find a solution to find a technique to auto arrange the data such that it presents something like this;

meter ID 1000 [19501 19502 .......................... 73048 .5 1.5............................ 2.5] and so on for all meter IDs. It is easy to solve in excel using filters, however given the amount of data, excel is not able to open this file. Any recommendation.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Walter Roberson 2017-2-2

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/323104-sorting-big-data-in-matlab#answer_253114

在 MATLAB Online 中打开

sortrow(YourData, [1 2])

2 个评论
显示无隐藏无

zafar khan 2017-2-2

Sorting is one part and thank you for that. Any idea about rearranging because it is very difficult to go through some 25000 rows to find the next meter ID?

Walter Roberson 2017-2-2

在 MATLAB Online 中打开

Ah, I was able to find a way to do it without any explicit loops:

[unique_ids, ~, uidx] = unique(sorted_rows(:,1));
grouped_rowidx = accumarray( uidx, (1:size(sorted_rows,1)), [], @(R) {R}, {});
grouped_rows = cellfun(@(R) sorted_rows(R,:), grouped_rowidx, 'Uniform', 0);

请先登录，再进行评论。