Info
此问题已关闭。 请重新打开它进行编辑或回答。
Neural network: train() behavior with earlier results
1 次查看(过去 30 天)
显示 更早的评论
I have a very large dataset of around 150GB that I need to process using neural networks. As this data is quite big, I've to break it into chunks, say 5000 elements are sent as 20 batches, each batch containing 250 elements. The following dummy code can be written for this:
for count = 1:num_batches
inputs = entire_input(1 + (count-1)*num_batches, count * num_batches);
targets = entire_targets(1 + (count-1)*num_batches, count * num_batches);
net = train(net, inputs, targets);
end
Will the net again start training with the fresh batch, or will it be able to retain weights calculated for previous batch? As per some of my discussions and findings, with each new batch, the weights start taking shape of current data and may overwrite previous weights.
Please advise if this method works well, or we can use some other method instead of train().
0 个评论
回答(1 个)
Greg Heath
2018-1-29
"Need to process" doesn't provide useful information.
What are you trying to design? Curvefitter/Regressor? PatternRecognizer/Unsupervised-Classifier/Supervised-Classifier? Timeseries??
In all cases, training, validation and test data should have similar summary statistics in all run batches. Otherwise training batch n will erase some of what is learned in batches 1 to n-1.
Your response should be far less vague than your original explanation.
Hope this helps.
Greg
Thank you for formally accepting my answer
1 个评论
此问题已关闭。
另请参阅
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!