Using datastore for a matrix

Question

Chuck Noise 2023-9-9

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2018846-using-datastore-for-a-matrix

评论： Catalytic 2023-9-9

Hi

I am trying to set up a datastore in order to train a feedforward neural network with a trainingset that is too large for my workspace.

I have a training set, X, that is 80002x10000 which I stored in a .mat file called ¨TrainingData'. I am now trying to set up a datastore so I can load mini batches and train on those, but I can't get it working.

I have tried datastore and arrayDatastore.

datastore cannot determine the datastore type for the specified location and arrayDatastore just output the name of the .mat-file whenever i read, i.e.

>> read(ds)

ans =

1×1 cell array

{'X.mat'}

What am I doing wrong here?

EDIT

I just read that the arrayDatastore is only for in-memory data which makes it useless in my case

Thanks

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Catalytic 2023-9-9

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2018846-using-datastore-for-a-matrix#answer_1306096

If you're not going to store your data in memory, then the training samples have to be split up into separate .mat files. You could then use a fileDatastore to read from and manage them.