How to minimize a function with SciPy

Function minimization turns a Python objective function into the input values that make its value as small as possible. In SciPy, scipy.optimize.minimize() is the standard entry point for smooth numerical optimization tasks such as tuning model parameters, calibrating constants, or checking a known optimum.

The minimize() call accepts an initial guess, an objective callable, and optional derivative information. For a smooth unconstrained problem, BFGS is a clear first method because it uses first-derivative information and returns an OptimizeResult object with the solution vector, objective value, status fields, and evaluation counts.

A small two-variable quadratic keeps the optimum easy to verify without adding plotting or model setup. The point [2, -1] should return an objective value of 0.5, and the gradient norm should be near zero after convergence.

1. Create a Python script named minimize_function.py.

   minimize_function.py

   import numpy as np
   from scipy.optimize import minimize
    
    
   def objective(values):
       x, y = values
       return (x - 2.0) ** 2 + (y + 1.0) ** 2 + 0.5
    
    
   def gradient(values):
       x, y = values
       return np.array([2.0 * (x - 2.0), 2.0 * (y + 1.0)])
    
    
   start = np.array([0.0, 0.0])
   expected = np.array([2.0, -1.0])
    
   result = minimize(
       objective,
       start,
       method="BFGS",
       jac=gradient,
       options={"gtol": 1e-10},
   )
    
   print(f"success: {result.success}")
   print(f"message: {result.message}")
   print(f"x: {result.x}")
   print(f"fun: {result.fun:.6f}")
   print(f"gradient_norm: {np.linalg.norm(result.jac):.2e}")
   print(f"distance_to_expected: {np.linalg.norm(result.x - expected):.2e}")
   print(f"iterations: {result.nit}")
   print(f"function_calls: {result.nfev}")

   jac supplies the analytic gradient. If jac is omitted, SciPy estimates derivatives for methods that support finite differences.

2. Run the minimization script.

   $ python minimize_function.py
   success: True
   message: Optimization terminated successfully.
   x: [ 2. -1.]
   fun: 0.500000
   gradient_norm: 4.44e-16
   distance_to_expected: 2.22e-16
   iterations: 2
   function_calls: 3

3. Confirm that the optimizer reported success.

   success should be True, and message should describe the termination reason before the solution vector is used in later calculations.

4. Read the returned minimum point and objective value.

   result.x contains the input vector at the minimum, and result.fun contains the objective value at that point. For this quadratic, [2, -1] and 0.5 match the known optimum.

5. Check the gradient norm and known-answer distance.

   The printed gradient_norm and distance_to_expected values are near machine precision, so the returned point is stationary and matches the expected minimum for the test function.

6. Replace the objective, gradient, and starting point with the function from your project.

   minimize() is a local optimizer. Try several starting points or add appropriate bounds when the objective has multiple basins or invalid variable ranges.

   Use scipy.optimize.linprog() instead for a linear objective with linear constraints.
   Related: How to solve a linear programming problem with SciPy

7. Remove the demo script when it was only created for the check.

   $ rm minimize_function.py