what is a reasonable step tolerance for optimization algorithms if parameters are in the range [0,1]?

Question

SA-W 2023-3-2

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1921725-what-is-a-reasonable-step-tolerance-for-optimization-algorithms-if-parameters-are-in-the-range-0-1

编辑： Bruno Luong 2023-3-2

My parameters are bounded from below by zero and from above by roughly one.

I recently worked with a quite small step tolerence of 1e-12, but I think the default value of 1e-6 would be fine as well.

Or should the step tolerance not exclusively be determined from the order of magnitude of the parameters?

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Bruno Luong 2023-3-2

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1921725-what-is-a-reasonable-step-tolerance-for-optimization-algorithms-if-parameters-are-in-the-range-0-1#answer_1183640

Your question is not clear enough. the official doc explains how it eaxctly works

https://www.mathworks.com/help/optim/ug/tolerances-and-stopping-criteria.html

"StepTolerance is a lower bound on the size of a step, meaning the norm of (xi – xi+1). If the solver attempts to take a step that is smaller than StepTolerance, the iterations end. StepTolerance is generally used as a relative bound, meaning iterations end when |(xi – xi+1)| < StepTolerance*(1 + |xi|), or a similar relative measure. See Tolerance Details."

The written equation can be viwed as some sort of relative tolerance, unless the optimum soluion is close to 0 then it can be viewed as absolute tolerance.

2 个评论
显示无隐藏无

SA-W 2023-3-2

I have read about that. But how would you choose the step tolerance based on the range of parameters? Or are there also other aspects to be considered when choosing the tolerance value?

Bruno Luong 2023-3-2

编辑：Bruno Luong 2023-3-2

在 MATLAB Online 中打开

The range of parameters are only part of the story

Imagine you want to minimize

f(x) where |x| is about 1 (unity)

Now if you minimize a function that has parameter shifted:

g(y) := f(y-1e6)

Obviouly

yopt = argmin(g) = argmin(f) + 1e6 = xopt + 1e6

The range of x is 1, the range of y is 1e6 but However the absolute StepTolerance should be identical for both. Meaning the relative StepTolerance for g should be 1e-6 time than that of f.

So not one can't chose correctly StepTolerance base ONLY on the range.

The range of the variation of the parameters is more important than the range of the parameters.

请先登录，再进行评论。