sgdmupdate() is a single step or multiple steps until convergence?

Question

robinho robinho 2022-7-12

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1757840-sgdmupdate-is-a-single-step-or-multiple-steps-until-convergence

评论： Ben 2023-9-18

Is calling sgdmupdate() once training a net over a single step or multiple steps until convergence? If the latter, how is convergence judged?

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

robinho robinho 2022-7-15

?

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Akash 2023-9-15

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1757840-sgdmupdate-is-a-single-step-or-multiple-steps-until-convergence#answer_1311176

编辑：Akash 2023-9-15

Hi Robinho,

I understand that you are seeking clarification on whether calling "sgdmupdate" function during training involves a single step or multiple steps until convergence. Additionally, you would like to know how convergence is determined, specifically the criteria used for judging convergence.

The "sgdmupdate" function utilizes multiple steps or epochs to achieve convergence during training. Each step represents a single update of the parameters using "Stochastic gradient descent with momentum (SGDM)".

To determine convergence, you can specify the total number of training epochs using the "numEpochs" parameter. This parameter indicates the number of times the model goes through the entire training dataset. Convergence is typically judged based on the desired level of accuracy or a predefined stopping criterion.

Additionally, the convergence can also be determined by monitoring the "Stop" property of the "TrainingProgressMonitor" object. If the "Stop" property is set to true, it indicates that the training process should be stopped. This property can be triggered, for example, by clicking the "Stop" button during training.

To gain more information about "sgdmupdate" function, including its usage and convergence criteria, I recommend referring to the documentation provided at the below provided link.

Update parameters using stochastic gradient descent with momentum (SGDM) - MATLAB sgdmupdate - MathWorks India

Thanks,

Akash.

2 个评论
显示无隐藏无

feynman feynman 2023-9-17

thanks so much. So sgdmupdate runs multiple steps instead of 1?

Ben 2023-9-18

Each call to sgdmupdate performs one step of the update algorithm specified at the bottom of the documentation page: https://uk.mathworks.com/help/deeplearning/ref/sgdmupdate.html

Typically you use sgdmupdate in custom training loops, which will typically end up calling sgdmupdate multiple times as the loop progresses, and control of convergence is up to the user to manage in the custom loop.

请先登录，再进行评论。