Minimization of the Himmelblau Function --------------------------------------- In this example we want to use AlgoPy to help compute the minimum of the non-convex multi-modal bivariate `Himmelblau function `_ .. math:: f(x, y) = (x^2 + y - 11)^2 + (x + y^2 - 7)^2 The idea is that by using AlgoPy to provide the gradient and hessian of the objective function, the nonlinear optimization procedures in `scipy.optimize `_ will more easily find the :math:`x` and :math:`y` values that minimize :math:`f(x, y)`. Here is the python code: .. literalinclude:: himmelblau_minimization.py Here is its output:: properties of the function at a local min: point: [ 3. 2.] function value: 0.0 autodiff gradient: [ 0. 0.] finite differences gradient: [ 0. 0.] autodiff hessian: [[ 74. 20.] [ 20. 34.]] finite differences hessian: [[ 74. 20.] [ 20. 34.]] --------------------------------------------------------- searches beginning from the easier init point [ 3.1 2.1] --------------------------------------------------------- properties of the function at the initial guess: point: [ 3.1 2.1] function value: 0.7642 autodiff gradient: [ 9.824 5.704] finite differences gradient: [ 9.824 5.704] autodiff hessian: [[ 81.72 20.8 ] [ 20.8 39.32]] finite differences hessian: [[ 81.72 20.8 ] [ 20.8 39.32]] strategy: default (Nelder-Mead) options: default Optimization terminated successfully. Current function value: 0.000000 Iterations: 31 Function evaluations: 59 [ 2.99997347 2.0000045 ] strategy: ncg options: default gradient: autodiff hessian: autodiff Optimization terminated successfully. Current function value: 0.000000 Iterations: 4 Function evaluations: 5 Gradient evaluations: 4 Hessian evaluations: 4 [ 3. 2.] strategy: ncg options: default gradient: autodiff hessian: finite differences Optimization terminated successfully. Current function value: 0.000000 Iterations: 4 Function evaluations: 5 Gradient evaluations: 18 Hessian evaluations: 0 [ 3. 2.] strategy: cg options: default gradient: autodiff Optimization terminated successfully. Current function value: 0.000000 Iterations: 7 Function evaluations: 18 Gradient evaluations: 18 [ 3.00000005 1.99999991] strategy: cg options: default gradient: finite differences Optimization terminated successfully. Current function value: 0.000000 Iterations: 7 Function evaluations: 72 Gradient evaluations: 18 [ 3.00000004 1.99999991] strategy: bfgs options: default gradient: autodiff Optimization terminated successfully. Current function value: 0.000000 Iterations: 5 Function evaluations: 9 Gradient evaluations: 9 [ 2.99999993 1.99999986] strategy: bfgs options: default gradient: finite differences Optimization terminated successfully. Current function value: 0.000000 Iterations: 5 Function evaluations: 36 Gradient evaluations: 9 [ 2.99999993 1.99999986] strategy: slsqp options: default gradient: autodiff Optimization terminated successfully. (Exit mode 0) Current function value: 2.68119275432e-08 Iterations: 5 Function evaluations: 9 Gradient evaluations: 5 [ 2.99997095 2.00001136] strategy: slsqp options: default gradient: finite differences Optimization terminated successfully. (Exit mode 0) Current function value: 2.68250888103e-08 Iterations: 5 Function evaluations: 24 Gradient evaluations: 5 [ 2.99997095 2.00001136] strategy: powell options: default Optimization terminated successfully. Current function value: 0.000000 Iterations: 5 Function evaluations: 126 [ 3. 2.] strategy: tnc options: default gradient: autodiff (array([ 2.999999, 2.000002]), 11, 1) strategy: tnc options: default gradient: finite differences (array([ 2.99999646, 2.00000762]), 14, 1) --------------------------------------------------------- searches beginning from the more difficult init point [-0.27 -0.9 ] --------------------------------------------------------- properties of the function at the initial guess: point: [-0.27 -0.9 ] function value: 181.61189441 autodiff gradient: [-0.146732 -0.3982 ] finite differences gradient: [-0.146732 -0.3982 ] autodiff hessian: [[-44.7252 -4.68 ] [ -4.68 -17.36 ]] finite differences hessian: [[-44.7252 -4.68 ] [ -4.68 -17.36 ]] strategy: default (Nelder-Mead) options: default Optimization terminated successfully. Current function value: 0.000000 Iterations: 54 Function evaluations: 105 [-2.80514623 3.13132056] strategy: ncg options: default gradient: autodiff hessian: autodiff Optimization terminated successfully. Current function value: 181.611894 Iterations: 1 Function evaluations: 5 Gradient evaluations: 1 Hessian evaluations: 1 [-0.26999996 -0.89999989] strategy: ncg options: default gradient: autodiff hessian: finite differences Optimization terminated successfully. Current function value: 181.611894 Iterations: 1 Function evaluations: 5 Gradient evaluations: 3 Hessian evaluations: 0 [-0.26999996 -0.89999989] strategy: cg options: default gradient: autodiff Optimization terminated successfully. Current function value: 0.000000 Iterations: 10 Function evaluations: 35 Gradient evaluations: 35 [ 3.00000011 1.99999978] strategy: cg options: default gradient: finite differences Optimization terminated successfully. Current function value: 0.000000 Iterations: 10 Function evaluations: 140 Gradient evaluations: 35 [ 3.0000001 1.99999978] strategy: bfgs options: default gradient: autodiff Optimization terminated successfully. Current function value: 0.000000 Iterations: 11 Function evaluations: 29 Gradient evaluations: 29 [ 3. 1.99999999] strategy: bfgs options: default gradient: finite differences Optimization terminated successfully. Current function value: 0.000000 Iterations: 11 Function evaluations: 116 Gradient evaluations: 29 [ 3. 1.99999999] strategy: slsqp options: default gradient: autodiff Optimization terminated successfully. (Exit mode 0) Current function value: 6.19262349912e-09 Iterations: 10 Function evaluations: 22 Gradient evaluations: 10 [ 2.99999684 2.00002046] strategy: slsqp options: default gradient: finite differences Optimization terminated successfully. (Exit mode 0) Current function value: 6.18718154108e-09 Iterations: 10 Function evaluations: 52 Gradient evaluations: 10 [ 2.99999683 2.00002045] strategy: powell options: default Optimization terminated successfully. Current function value: 0.000000 Iterations: 6 Function evaluations: 155 [ 3.58442834 -1.84812653] strategy: tnc options: default gradient: autodiff (array([ 3., 2.]), 42, 1) strategy: tnc options: default gradient: finite differences (array([ 2.99999981, 1.99999997]), 39, 1)