Loren's blog on datastore. How should I use counterSize and done?

Question

gujax 2021-11-28

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1597794-loren-s-blog-on-datastore-how-should-i-use-countersize-and-done

回答： gujax 2021-11-29

I am trying to use datastore in Matlab v2018b but it does not have 'partialread'.

I am trying to replicate Loren's code (https://blogs.mathworks.com/loren/2019/05/29/big-data-in-mat-files/ and the section 'Use a portion of all variables') for a large variable whose size exceeds memory.

Unfortunately its written kind of poorly with many variables thrown in and not explained. For example, I cannot figure out how to use 'done' and 'counterSize'.

I tried

[fds_partial, counter, done] = fileDatastore("smallVars*.mat", "ReadFcn", @partialReadFcn, "UniformRead", true);

But I get an error of 'too many output arguments'

I tried

[M, counter, done] = read(fds_partial) %with
fds_partial = fileDatastore("smallVars*.mat", "ReadFcn", @partialReadFcn, "UniformRead", true);

Again I get an error of too many output arguments.

So I end up reading only one block of file and the readfcn function does not proceed beyond one block.

So what gives?

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Image Analyst 2021-11-28

Did you try emailing her?

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Adam Danz 2021-11-28

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1597794-loren-s-blog-on-datastore-how-should-i-use-countersize-and-done#answer_842239

在 MATLAB Online 中打开

> I am trying to use datastore in Matlab v2018b but it does not have 'partialread'.

partialreadFcn is a function defined in Loren's blog post. No currently existing release of Matlab has a "partialread" function.

> I cannot figure out how to use 'done' and 'counterSize'.

done appears to be a flag (true|false) indicating whether there is more to read or not.

There is no variable named counteSize.

> But I get an error of 'too many output arguments'

fileDatastore only has 1 output.

doc fileDatastore

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Answer 2

gujax 2021-11-29

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1597794-loren-s-blog-on-datastore-how-should-i-use-countersize-and-done#answer_842509

I think I figured out some of the answers...

I need later versions of Matlab to get 'partial read' enabled using 'read mode'.

Also though 'done' and 'readcounter' can used in the partialread function, I am still not sure where these two variables get outputed and how I should be using them.

So its kind of useless to go through filedatastore when one can do this simply by re-reading partial files. Not sure how all this helps.