Unconstrained optimization: Problems 14.1, 14.2, 14.3
Implement Nelder-Mead, Line Minimization, Direction Set, and Conjugate Gradient and use them to search for a minimum of the Rosenbrock function, starting from x=y=-1.
I worked from an IPython notebook this week, you can access the generated html here.
Nelder-Mead, sweet and simple.
Line minization with golden section search. After struggling with the initial bracketing for a bit, I translated a subroutine from numerical recipes.
Powell's direction set method