What is the maximum number of variables for practical use of fminsearch?

Question

Craig 2014-5-15

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/129747-what-is-the-maximum-number-of-variables-for-practical-use-of-fminsearch

评论： Craig 2014-5-15

As part of a project i'm comparing the effectiveness of different optimisation methods to minimise the cost function (and hence train) a Neural Network. I've just started an optimisation to minimise a cost function which takes in 1275 input arguments (1275 synaptic weights). I don't mind waiting a few hours, but if this is going to take days/weeks I'd rather give up now, and just say that this method is impractical.

How long would you guess this will take?

I have a 3 year old laptop running matlab R2011(a), with a 2GHz 4GB 2 core processor, on a 64 bit windows operating system. However unfortunately Matlab can not use parpool with fminsearch, so my CPU usage is at about 60%.

Thanks!

Note: I know the Neural Network Toolbox is much better and faster at training Neural Nets, but I programmed my own Neural Net from scratch, and because of the convoluted way I've done it I can't implement back propagation. I realised this too late so i'll just have to stick with this method for now.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

José-Luis 2014-5-15

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/129747-what-is-the-maximum-number-of-variables-for-practical-use-of-fminsearch#answer_136913

编辑：José-Luis 2014-5-15

The Nealder-Mead algorithm is meant to be used for low-dimensionality problems. fminsearch uses it.

Please see:

[1] Lagarias, J.C., J. A. Reeds, M. H. Wright, and P. E. Wright, "Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions," SIAM Journal of Optimization, Vol. 9 Number 1, pp. 112-147, 1998.

for more details.

So using it for 1275 is probably a very bad idea. How long it will take is very difficult to say beforehand. It depends on the computational complexity of evaluating your cost function. It also depends on your response surface. If is is smooth, then it might converge reasonably fast, if it is complex, it might get stuck calculating forever. Anyway, it will more likely get stuck in a local optimum than actually find the global optimum. It is likely that you would end up waiting longer than the age of the universe.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Answer 2

John D'Errico 2014-5-15

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/129747-what-is-the-maximum-number-of-variables-for-practical-use-of-fminsearch#answer_136912

Expect it to take YEARS.

About 6 variables is a reasonable maximum number of variables for fminsearch. Some people might go to 8 or even 10, if they have more patience than do I. But hundreds or thousands of variables? No.