24.1. Nelder-Mead

The Nelder-Mead algorithm [(ref nelder.mead:simplex)] is a direct search method for finding a local minimum of a function f(x). This algorithm does not require any gradient or Hessian information of f, and therefore has some expected advantages and disadvantages compared to the other TAO solvers. The obvious advantage is that it is easier to write an application when no derivatives need to be calculated. The downside is that this algorithm can be very slow to converge or can even stagnate, and performs poorly for large numbers of variables.

This solver keeps a set of N+1 sorted vectors x1,x2,...,xN+1 and their corresponding objective function values . At each iteration, xN+1 is removed from the set and replaced with

where can be one of depending upon the values of each possible .

The algorithm terminates when the residual fN+1 - f1 becomes sufficiently small. Because of the way new vectors can be added to the sorted set, the minimum function value and/or the residual may not be impacted at each iteration.

There are two options that can be set specifically for the Nelder-Mead algorithm, -tao_nm_lamda <value> sets the initial set of vectors (x0 plus value in each cartesion direction), the default value is 1. tao_nm_mu <value> sets the value of , the default is .