Gradient Descent Implementation on a function, as opposed to an equation

7 次查看（过去 30 天）
PASUNURU SAI VINEETH
评论： PASUNURU SAI VINEETH 2022-9-10
I have built a function
function [RMS] = ErrorFunction(ui,vi,wi,cx,cy,cz)
which outputs a certain error based on the six intial conditions I input to the model. Now, my model is iterative and the error depends on some intermediate parameters hence it is not possible to define a relationship between the error and six inputs. My aim is to minimize the error to zero using Gradient/Steepest Descent Method and I'm hoping somone would guide me in implementing it on a function, as opposed to a straightforward explicit relationship like, f(x,y) = 4x^2-4xy+2y^2

请先登录，再回答此问题。

采纳的回答

Bjorn Gustavsson
Bjorn Gustavsson 2022-9-7
You could do something like this:
1, rewrite the function to take an array as input-parameter instead of a number of scalar parameters
function [RMS] = ErrorFunction([ui_vi_wi_cx_cy_cz])
2, calculate a numerical gradient (central difference):
curr_pars = [ui_vi_wi_cx_cy_cz]; % current point
h = 0.01; % you have to decide what a suitably small step is, and if you need different sized steps for the different input variables...
for i_v = numel(curr_pars):-1:1
cp = curr_pars;
cp(i_v) = cp(i_v) + h;
cm = curr_pars;
cm(i_v) = cm(i_v) - h;
gradErrorFunction(i_v) = (ErrorFunction(cp) - ErrorFunction(cm))/(2*h);
end
HTH
  1 个评论
PASUNURU SAI VINEETH
PASUNURU SAI VINEETH 2022-9-10
Thank you very much, that was brilliant! This approach suits my purpose.
Apologize for the late response, I've been trying out different combinations of the stepsize h and learning rate gamma before getting back to you.
clc
clear all
gamma = 2;
curr_pars = [1 1 1 1 1 1]; % current point
tic
for k = 1:100
h = [1 1 1 1 1 1];
for i_v = numel(curr_pars):-1:1
cp = curr_pars;
cp(i_v) = cp(i_v) + h(i_v);
cm = curr_pars;
cm(i_v) = cm(i_v) - h(i_v);
gradErrorFunction(i_v) = (ErrorFunction(cp) - ErrorFunction(cm))/(2*h(i_v));
end
curr_pars = curr_pars - gamma*gradErrorFunction
[err] = ErrorFunction(curr_pars)
end
toc
I was wondering if setting gamma = 2*h makes things simpler as they cancel out each other in the line
curr_pars = curr_pars - gamma*gradErrorFunction
which makes the updated parameter array solely depend on h (from the difference of error functions).
Do you think the current stepsize and learning rate are reasonable or are they too high? I observed that towards the end of 100 iterations, the error stopped reducing and started increasing. But if I reduce those two variables, the error didn't even tend to decrease in the first place. I just played around with various combinations but didn't find a way to optimize them.
Again, thanks a lot for taking the time to answer my query.

请先登录，再进行评论。

更多回答（1 个）

Torsten
Torsten 2022-9-7
Use
fun = @(x)ErrorFunction(x(1),x(2),x(3),x(4),x(5),x(6))
as the function handle you work on in the steepest decent.

类别

在 Help Center 和 File Exchange 中查找有关 Parallel Computing 的更多信息

标签

产品


版本

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by