Next: Quasi-Newton Methods Up: Review of The Basics: Previous: Condition Number of the

Gradient Calculation: Constrained Optimization

Black Box Methods are the simplest approach to solve constrained optimization problems and consist of calculating the gradient in the following way. Let be the change in the cost functional as a result of a change in the design variables. The following relation holds

 (27)

where are the partial derivatives of with respect to the design variables, also termed as sensitivity derivatives. All quantities in this expression are straightforward to calculate except . The dimensionality of this quantity is the dimension of times the dimension of .

The calculation of is done in this approach using finite differences. That is, for each of the design parameters in the representation of as , where are a set of vectors spanning the design space, one perform the following process

ALGORITHM: Black-Box Gradient Calculation

• Solve given
• Set

Once the above process is completed for , one combines the result into

 (28)

which is used in calculation of the gradient
 (29)

Since in practical problems the dimension of may be thousands to millions, the feasibility of calculating gradients using this approach is limited to cases where the number of design variables is very small.

The Adjoint Method is an efficient way for calculating gradients for constrained optimization problems even for very large dimensional design space. The idea is to use the expression for the gradient as appears in (18). Thus, one introduces into the solution process an extra unknown, , which satisfies the adjoint equation (13).

A minimization algorithm is then a repeated application of the following three steps.