minibatchqueue without shuffling by default

2022 7 11
1 个回答
回答已采纳
更新时间：2022 8 5
2 次查看（30 天）

0 个投票

mbq = minibatchqueue(ads,MiniBatchSize=miniBatchSize,MiniBatchFormat="BC");
If without mbq.shuffle, mbq retains the original sequence in ads? That is, minibatchqueue just cuts ads into length(ads)/miniBatchSize*portions?
e.g. ads=arrayDatastore(linspace(0,2,1000),IterationDimension=1) and if miniBatchSize=100, then the 1st batch of mbq is just [0, ..., 0.2]?

请先登录，再回答此问题。

 采纳的回答

Ben
Ben 2022-7-12

0 个投票

I think you need to use IterationDimension=2 to read out [0, ..., 0.2] in one mini-batch:
ads=arrayDatastore(linspace(0,2,1000),IterationDimension=2);
miniBatchSize=100;
mbq = minibatchqueue(ads,MiniBatchSize=miniBatchSize,MiniBatchFormat="BC");
batch = next(mbq);
This will read out a batch of 100 observations, first [0, ..., 0.1982], then [0.2002,...,0.3984] etc. These will be output as 1x100 dlarray with "CB" labels - i.e. a batch of 100 observations.

6 个评论
显示 4更早的评论 隐藏 4更早的评论

robinho robinho
robinho robinho 2022-7-12
thank you so much Ben! Sorry for a typo of my OP, but I meant to post:
ads=arrayDatastore(linspace(0,2,1000)',IterationDimension=1) and if miniBatchSize=100, then the 1st batch of mbq is just [0, ..., 0.1982]?
So can we confirm that without mbq.shuffle, mbq retains the original sequence in ads and next() doesn't contain any randomness in it?
Ben
Ben 2022-7-12
Yes, there should be no randomness, and if the miniBatchSize exactly divides the number of observations then minibatchqueue will just read through the data in chunks of size miniBatchSize (the exception is that if miniBatchSize doesn't exactly divide the number of observations then your last minibatch will be smaller - the remainder of numObservations / miniBatchSize).
Once the minibatchqueue runs out of data use reset to loop back to the start:
ads=arrayDatastore(linspace(0,2,1000)',IterationDimension=1);
mbq = minibatchqueue(ads,MiniBatchSize=100);
while hasdata(mbq)
batch = next(mbq);
end
reset(mbq);
robinho robinho
robinho robinho 2022-7-20
thank you. What difference will it make to the training if mbq.shuffle isn't used?
Ben
Ben 2022-7-20
The data won't be shuffled at the end of each epoch. This could impact the model performance.
robinho robinho
robinho robinho 2022-7-21
right, but what impact would no-shuffling make?

请先登录，再进行评论。

更多回答（0 个）

类别

在 帮助中心 和 File Exchange 中查找有关 Deep Learning Toolbox 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by