NMM: Week 10 - Optimization + Search

Here we explore minimization approaches to multidimensional search and optimization. Nelder-Mead (a.k.a. Downhill Simplex) is explored on the Rosenbrock function using Python's SciPy.

The Nelder-Mead algorithm can take following moves from simplex:

• 1. reflect about mean
• 2. reflect and grow (one point move)
• 3. reflect and shrink (one point move)
• 4. shrink (one point move)
• 5. shrink towards minimum (two point simplex move)

To determine convergence, we also need to define a threshold change in f(x,y) at which we call a win. In the code example here, we simply define this as a static threshold, though adaptive methods can be used to get around issues in more complicated functions.

With more time, I'd like to overlay a plot of the algorithm over the graph itself, but for now, the code outputs local minimum f(x,y) at each step...