Neural Network MEMORY problem
显示 更早的评论
Hello there,
I have around 500,000 (samples) x 50 (features) matrix that I would like to train with Matlab's NN. However, with use of more then 7-8 hidden nodes (in a hidden layer), I always get memory problem. I've tried with increasing the net.trainParam.mem_reduc parameter to 1,000 and above, but that didn't help.
In order to reach some acceptable recognition rate, I would need to use more than only 7-8 hidden nodes, but memory problem doesn't allow me to :-(
So, what should I do? Should I cut my data into several smaller blocks? Would that yield similar results? Is there some alternative way to increase a processing time, but lower down the needed memory (apart from the mem_reduce parameter)? Can Matlab's NN read my data from a file (row by row)... is that possible and would that help?
I assume that I can gain the best results by training the whole data as one big matrix, so I'm trying to find a way to do that, with use of at least 30 hidden nodes in a hidden layer.
Please help :-) Thank you all! Best regards, Igor
1 个评论
Thomas Hansen
2016-11-28
try using 'trainscg' as the training function type. This use much less memory than 'trainlm'
采纳的回答
Greg Heath
2012-3-31
0 个投票
If
[ I N ] = size(input)
[O N ] = size(output)
then
Neq = N*O % No. of training equations
and for an I-H-O net
Nw = (I+1)*H+(H+1)*O. % No. of unknown weights
Choose H ~ Hub/10 where Hub is the upper bound limit obtained from imposing
Neq >= Nw:
Hub = (Neq-O)/(I+O+1)
Then Neq ~ 10*Nw.
For each net use ~ 10 random weight initializations and choose the best w.r.t. performance of nontraining data from othe groups.
Finally, the least correlated of the best can be combined into an ensemle whose output is just the average of the member nets.
Hope this helps.
Greg
更多回答（1 个）
Greg Heath
2012-3-31
0 个投票
Is this regression or classification?
What algorithm are you using?
What do the 50 input variables represent?
How many output variables? What do they represent?
How were the input variables chosen?
Are they linearly independent?
Which inputs have significant linear correlations with other input variables?
Which inputs have insignificant linear correlations with all of the output variables?
Are the output variables linearly independent?
Which outputs have significant linear correlations with other output variables?
It is probably unnecessary to use all of the data to train one net.
For large data sets, TRAINRP is recommended over the LM and CG algorithms.
Randomly partition the data into groups of managable size.
For each group, design a net so that
Neq >> Nw
where Neq is the number of training equations and Nw is the number of unknown weights
1 个评论
Greg Heath
2012-3-31
The input/output questions may be useful for variable subset reduction.
类别
在 帮助中心 和 File Exchange 中查找有关 Deep Learning Toolbox 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
