Is slicing one big matrix more efficient than storing smaller matrices in a struct?

Question

Mitsu 2020-10-25

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/625543-is-slicing-one-big-matrix-more-efficient-than-storing-smaller-matrices-in-a-struct

编辑： Matt J 2020-11-6

An example of the former would be:

% allcases = 
% 
% +---------+------------+------------+------------+-----+-------------+
% | Case ID | Data col 1 | Data col 2 | Data col 3 | ... | Data col 10 |
% +---------+------------+------------+------------+-----+-------------+
% |    1    |   scalar   |   scalar   |   scalar   | ... |    scalar   |
% +---------+------------+------------+------------+-----+-------------+
% |    1    |   scalar   |   scalar   |   scalar   | ... |    scalar   |
% +---------+------------+------------+------------+-----+-------------+
% |    1    |   scalar   |   scalar   |   scalar   | ... |    scalar   |
% +---------+------------+------------+------------+-----+-------------+
% |   ...   |     ...    |     ...    |     ...    | ... |     ...     |
% +---------+------------+------------+------------+-----+-------------+
% |    n    |   scalar   |   scalar   |   scalar   | ... |    scalar   |
% +---------+------------+------------+------------+-----+-------------+

An example of the latter would be:

% allcases(case_id).matrix =
%
% +------------+------------+------------+-----+-------------+
% | Data col 1 | Data col 2 | Data col 3 | ... | Data col 10 |
% +------------+------------+------------+-----+-------------+
% |   scalar   |   scalar   |   scalar   | ... |    scalar   |
% +------------+------------+------------+-----+-------------+
% |   scalar   |   scalar   |   scalar   | ... |    scalar   |
% +------------+------------+------------+-----+-------------+
% |   scalar   |   scalar   |   scalar   | ... |    scalar   |
% +------------+------------+------------+-----+-------------+
% |     ...    |     ...    |     ...    | ... |     ...     |
% +------------+------------+------------+-----+-------------+
% |   scalar   |   scalar   |   scalar   | ... |    scalar   |
% +------------+------------+------------+-----+-------------+

Now, in a loop that goes through the cases, it needs to work with a (smaller) matrix that concerns that case only. Therefore, with the former approach we would slice the matrix (via logical or a find) while in the latter we would just call allcases(case_id).matrix.

Is there a recommended approach between these two, in terms of computational speed?

In my problem, I work with up to 100 cases, and up to 50,000 rows in the matrix containing data for each case.

3 个评论
显示 1更早的评论隐藏 1更早的评论

dpb 2020-10-25

The struct will create a fair amount of storage overhead plus have to create it.

Is the number of observations per case fixed and the same? If that were so, then straight indexing will return the values by case without the overhead of the logical matching altho since don't need the numeric indices, dispensing with the unneeded find() will be somewhat faster.

Overall, 50,000/100 ==> ~500 rows/case is pretty trivial case size; unless there are some iterative calculations or other very compute-intensive analyses, I doubt it'll make much difference at all. But, I'd put my money on not using struct as the winner simply owing to the higher-level abstraction thereof.

Sindar 2020-10-25

在 MATLAB Online 中打开

If the data/case ID is constant, and depending on what you're doing with the data, it may be useful to use paged arrays, so

allcases(:,:,case_id)

returns the same matrix as your

allcases(case_id).matrix

For example, 2020b introduced pagewise matrix multiplication which is almost certainly faster than looping over the index subsets (or a structure array)

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Matt J 2020-11-6

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/625543-is-slicing-one-big-matrix-more-efficient-than-storing-smaller-matrices-in-a-struct#answer_536255

编辑：Matt J 2020-11-6

在 MATLAB Online 中打开

Therefore, with the former approach we would slice the matrix (via logical or a find) while in the latter we would just call allcases(case_id).matrix..

Either one can be more efficient, depending on the situation. Slicing the matrix requires a memory allocation operation. On the other hand, if you have a very large number of matrix IDs, I think you will find that the non-contiguous storage in RAM that you will have with a struct will start to outweigh what you save by avoiding matrix slicing.

Incidentally, you might also consider using a cell array,

allcases{case_id}=matrix

which also avoids matrix slicing, but which should give you slightly faster indexing than a struct.