How the Optimization Algorithm Formulates Minimization Problems

When you optimize parameters of a Simulink^® model to meet design requirements, Simulink Design Optimization™ software automatically converts the requirements into a constrained optimization problem and then solves the problem using optimization techniques. The constrained optimization problem iteratively simulates the Simulink model, compares the results of the simulations with the constraint objectives, and uses optimization methods to adjust tuned parameters to better meet the objectives.

This topic describes how the software formulates the constrained optimization problem used by the optimization algorithms. For each optimization algorithm, the software formulates one of the following types of minimization problems:

Feasibility
Tracking
Mixed feasibility and tracking

For more information on how each optimization algorithm formulates these problems, see:

Feasibility Problem and Constraint Formulation

Feasibility means that the optimization algorithm finds parameter values that satisfy all constraints to within specified tolerances but does not minimize any objective or cost function in doing so.

In the following figure, x₁, x₃, and x_n represent a combination of parameter values P₁ and P₂ and are feasible solutions because they do not violate the lower bound constraint.

P2 vs P1 graph with x1, x3, and xn as feasible solutions and x2 violating the lower bound constraint

In a Simulink model, you constrain a signal by specifying lower and upper bounds in a Check block (Check Step Response Characteristics, ...) or a requirement object (sdo.requirements.StepResponseEnvelope, ...), as shown in the following figure.

Plot showing upper and lower bounds

These constraints are piecewise linear bounds. A piecewise linear bound y_bnd with n edges can be represented as:

$y_{b n d} (t) = {\begin{matrix} y_{1} (t) & t_{1} \leq t \leq t_{2} \\ y_{2} (t) & t_{2} \leq t \leq t_{3} \\ ⋮ & ⋮ \\ y_{n} (t) & t_{n} \leq t \leq t_{n + 1} \end{matrix},$

The software computes the signed distance between the simulated response and the edge. The signed distance for lower bounds is:

$\begin{array}{l} c = [\begin{array}{l} \max_{t_{1} \leq t \leq t_{2}} y_{b n d} - y_{s i m} \\ \max_{t_{2} \leq t \leq t_{3}} y_{b n d} - y_{s i m} \\ \max_{t_{n} \leq t \leq t_{n + 1}} y_{b n d} - y_{s i m} \end{array}], \end{array}$

where y_sim is the simulated response and is a function of the parameters being optimized.

The signed distance for upper bounds is:

$c = [\begin{matrix} \max_{t_{1} \leq t \leq t_{2}} y_{s i m} - y_{b n d} \\ \max_{t_{2} \leq t \leq t_{3}} y_{s i m} - y_{b n d} \\ \max_{t_{n} \leq t \leq t_{n + 1}} y_{s i m} - y_{b n d} \end{matrix}] .$

At the command line, optimFcn supplies c directly from the Cleq field of vals.

If all the constraints are met (c ≤ 0) for some combination of parameter values, then that solution is said to be feasible. In the following figure, x₁ and x₃ are feasible solutions.

P2 vs P1 graph with x1 and x3 as feasible solutions and x2 and xn violating the constraints

When your model has multiple requirements or vector signals feeding a requirement, the constraint vector is extended with the constraint violations for each signal and bound:

$C = [c_{1}; c_{2}; \dots; c_{n}] .$

Tracking Problem

In addition to lower and upper bounds, you can specify a reference signal in a Check Against Reference block or sdo.requirements.SignalTracking object, which the Simulink model output can track. The tracking objective is a sum-squared-error tracking objective.

You specify the reference signal as a sequence of time-amplitude pairs:

$y_{r e f} (t_{r e f}), t_{r e f} \in {T_{r e f 0}, T_{r e f 1}, \dots, T_{r e f N}} .$

The software computes the simulated response as a sequence of time-amplitude pairs:

$y_{s i m} (t_{s i m}), t_{s i m} \in {T_{s i m 0}, T_{s i m 1}, \dots, T_{s i m N}},$

where some values of t_sim may match the values of t_ref.

A new time base, t_new, is formed from the union of the elements of t_ref and t_sim. Elements that are not within the minimum-maximum range of both t_ref and t_sim are omitted:

$t_{n e w} = {t : t_{s i m} \cup t_{r e f}}$

Using linear interpolation, the software computes the values of y_ref and y_sim at the time points in t_new and then computes the scaled error:

$e (t_{n e w}) = \frac{(y_{s i m} (t_{n e w}) - y_{r e f} (t_{n e w}))}{\max_{t_{n e w}} | y_{r e f} |} .$

Finally, the software computes the weighted, integral square error:

$f = \int w (t) e {(t)}^{2} d t .$

Note

The weight w(t) is 1 by default. You can specify a different value of weight only at the command line.

When your model has requirements or vector signals feeding a requirement, the tracking objective equals the sum of the individual tracking integral errors for each signal:

$F = \sum f_{i} .$

Gradient Descent Method Problem Formulations

The Gradient Descent method uses the function fmincon to optimize model parameters to meet design requirements.

Problem Type	Problem Formulation
Feasibility Problem	The software formulates the constraint C(x) as described in Feasibility Problem and Constraint Formulation. If you select the maximally feasible solution option (i.e., the optimization continues after an initial feasible solution is found), the software uses the following problem formulation: $\begin{array}{l} \min_{[x, γ]} γ \\ s . t . C (x) \leq γ \\ \underline{x} \leq x \leq \bar{x} \\ γ \leq 0 \end{array}$ γ is a slack variable that permits a feasible solution with C(x) ≤ γ rather than C(x) ≤ 0. If you do not select the maximally feasible solution option (i.e., the optimization terminates as soon as a feasible solution is found), the software uses the following problem formulation: $\begin{array}{l} \min_{x} 0 \\ s . t . C (x) \leq 0 \\ \underline{x} \leq x \leq \bar{x} \end{array}$
Tracking Problem	The software formulates the tracking objective F(x) as described in Tracking Problem and minimizes the tracking objective: $\begin{array}{l} \min_{x} F (x) \\ s . t . \underline{x} \leq x \leq \bar{x} \end{array}$
Mixed Feasibility and Tracking Problem	The software minimizes following problem formulation: $\begin{array}{l} \min_{x} F (x) \\ s . t . C (x) \leq 0 \\ \underline{x} \leq x \leq \bar{x} \end{array}$ Note When tracking a reference signal, the software ignores the maximally feasible solution option.

Problem Type

Problem Formulation

Feasibility Problem

The software formulates the constraint C(x) as described in Feasibility Problem and Constraint Formulation.

If you select the maximally feasible solution option (i.e., the optimization continues after an initial feasible solution is found), the software uses the following problem formulation:
$\begin{array}{l} \min_{[x, γ]} γ \\ s . t . C (x) \leq γ \\ \underline{x} \leq x \leq \bar{x} \\ γ \leq 0 \end{array}$
γ is a slack variable that permits a feasible solution with C(x) ≤ γ rather than C(x) ≤ 0.
If you do not select the maximally feasible solution option (i.e., the optimization terminates as soon as a feasible solution is found), the software uses the following problem formulation:
$\begin{array}{l} \min_{x} 0 \\ s . t . C (x) \leq 0 \\ \underline{x} \leq x \leq \bar{x} \end{array}$

Tracking Problem

The software formulates the tracking objective F(x) as described in Tracking Problem and minimizes the tracking objective:

$\begin{array}{l} \min_{x} F (x) \\ s . t . \underline{x} \leq x \leq \bar{x} \end{array}$

Mixed Feasibility and Tracking Problem

The software minimizes following problem formulation:

$\begin{array}{l} \min_{x} F (x) \\ s . t . C (x) \leq 0 \\ \underline{x} \leq x \leq \bar{x} \end{array}$

Note

When tracking a reference signal, the software ignores the maximally feasible solution option.

Simplex Search Method Problem Formulations

The Simplex Search method uses the function fminsearch and fminbnd to optimize model parameters to meet design requirements. fminbnd is used if one scalar parameter is being optimized, otherwise fminsearch is used. You cannot use parameter bounds $\underline{x} \leq x \leq \bar{x}$ with fminsearch.

Problem Type	Problem Formulation
Feasibility Problem	The software formulates the constraint C(x) as described in Feasibility Problem and Constraint Formulation and then minimizes the maximum constraint violation: $\min_{x} max (C (x))$
Tracking Problem	The software formulates the tracking objective F(x) as described in Tracking Problem and then minimizes the tracking objective: $\min_{x} F (x)$
Mixed Feasibility and Tracking Problem	The software formulates the problem in two steps: Finds a feasible solution. $\min_{x} max (C (x))$ Minimizes the tracking objective. The software uses the results from step 1 as initial guesses and maintains feasibility by introducing a discontinuous barrier in the optimization objective. $\begin{array}{l} \min_{x} Γ (x) \\ where \\ Γ (x) = {\begin{array}{l} \infty & if \max (C (x)) > 0 \\ F (x) & otherwise . \end{array} \end{array}$

Problem Type

Problem Formulation

Feasibility Problem

The software formulates the constraint C(x) as described in Feasibility Problem and Constraint Formulation and then minimizes the maximum constraint violation:

$\min_{x} max (C (x))$

Tracking Problem

The software formulates the tracking objective F(x) as described in Tracking Problem and then minimizes the tracking objective:

$\min_{x} F (x)$

Mixed Feasibility and Tracking Problem

The software formulates the problem in two steps:

Finds a feasible solution.
$\min_{x} max (C (x))$
Minimizes the tracking objective. The software uses the results from step 1 as initial guesses and maintains feasibility by introducing a discontinuous barrier in the optimization objective.
$\begin{array}{l} \min_{x} Γ (x) \\ where \\ Γ (x) = {\begin{array}{l} \infty & if \max (C (x)) > 0 \\ F (x) & otherwise . \end{array} \end{array}$

Pattern Search Method Problem Formulations

The Pattern Search method uses the function patternsearch (Global Optimization Toolbox) to optimize model parameters to meet design requirements.

Problem Type	Problem Formulation
Feasibility Problem	The software formulates the constraint C(x) as described in Feasibility Problem and Constraint Formulation and then minimizes the maximum constraint violation: $\begin{array}{l} \min_{x} max (C (x)) \\ s . t . \underline{x} \leq x \leq \bar{x} \end{array}$
Tracking Problem	The software formulates the tracking objective F(x) as described in Tracking Problem and then minimizes the tracking objective: $\begin{array}{l} \min_{x} F (x) \\ s . t . \underline{x} \leq x \leq \bar{x} \end{array}$
Mixed Feasibility and Tracking Problem	The software formulates the problem in two steps: Finds a feasible solution. $\begin{array}{l} \min_{x} max (C (x)) \\ s . t . \underline{x} \leq x \leq \bar{x} \end{array}$ Minimizes the tracking objective. The software uses the results from step 1 as initial guesses and maintains feasibility by introducing a discontinuous barrier in the optimization objective. $\begin{array}{l} \min_{x} Γ (x) \\ s . t . \underline{x} \leq x \leq \bar{x} \\ where \\ Γ (x) = {\begin{array}{l} \infty & if \max (C (x)) > 0 \\ F (x) & otherwise . \end{array} \end{array}$

Problem Type

Problem Formulation

Feasibility Problem

The software formulates the constraint C(x) as described in Feasibility Problem and Constraint Formulation and then minimizes the maximum constraint violation:

$\begin{array}{l} \min_{x} max (C (x)) \\ s . t . \underline{x} \leq x \leq \bar{x} \end{array}$

Tracking Problem

The software formulates the tracking objective F(x) as described in Tracking Problem and then minimizes the tracking objective:

$\begin{array}{l} \min_{x} F (x) \\ s . t . \underline{x} \leq x \leq \bar{x} \end{array}$

Mixed Feasibility and Tracking Problem

The software formulates the problem in two steps:

Finds a feasible solution.
$\begin{array}{l} \min_{x} max (C (x)) \\ s . t . \underline{x} \leq x \leq \bar{x} \end{array}$
Minimizes the tracking objective. The software uses the results from step 1 as initial guesses and maintains feasibility by introducing a discontinuous barrier in the optimization objective.
$\begin{array}{l} \min_{x} Γ (x) \\ s . t . \underline{x} \leq x \leq \bar{x} \\ where \\ Γ (x) = {\begin{array}{l} \infty & if \max (C (x)) > 0 \\ F (x) & otherwise . \end{array} \end{array}$

Gradient Computations

For the Gradient descent (fmincon) optimization solver, the gradients are computed using numerical perturbation:

$\begin{array}{l} d x = \sqrt[3]{e p s} \times \max (| x |, \frac{1}{10} x_{t y p i c a l}) \\ d L = \max (x - d x, x_{\min}) \\ d R = \min (x + d x, x_{\max}) \\ F_{L} = o p t_f c n (d L) \\ F_{R} = o p t_f c n (d R) \\ \frac{d F}{d x} = \frac{(F_{L} - F_{R})}{(d L - d R)} \end{array}$

x is a scalar design variable.
x_min is the lower bound of x.
x_max is the upper bound of x.
x_typical is the scaled value of x.
opt_fcn is the objective function.

dx is relatively large to accommodate simulation solver tolerances.

If you want to compute the gradients in any other way, you can do so in the cost function you write for performing design optimization programmatically. See sdo.optimize and GradFcn of sdo.OptimizeOptions for more information.