Robert B. Schnabel, CU, Parallel Nonlinear Optimization: Limitations, Opportunities, and Challenges, here. 1995 he did Numerical Methods for Unconstrained Optimization and Nonlinear equations with Jack Dennis back in the day, here. The game in parallel nonlinear programming must have completely changed since 2006, if only because of the way microprocessors packaged the arithmetic. Need relatively recent references for Finding Nimo.
Erricos John Kontoghiorghes, CRC, Handbook of Parallel Computing and Statistics, here.
Hager and Wellein, CRC, Introduction to High Performance Computing for Scientists and Engineers, here. Hager seems to know this.
Bertsekas, Nonlinear Programming lecture slides, here. Maybe we want Bertsekas? not sure this is 2005.
Carl Laird, Purdue, Parallel Computing and Opportunities in Nonlinear Optimization, here. Mar 2014
SIAM Journal on Control and Optimization, here.
Shewchuk, 1994, CMU, An Introduction to Conjugate Gradient Method without Agonizing Pain, here.
Boyd, Stanford, Conjugate Gradient Method, here.
Bolz et.al., Sparse Matrix solvers on the GPU: COnjugate Gradients and Multigrid, here.
Strang, Mathematical Methods for Engineers, MITOPENCOURSEWARE, here. 2006