In this chapter, we treat methods for finding a zero x* ∈ D (i.e., a point x* ∈ D with F(x*) = 0) of a continuously differentiable function F : D ⊆ ℝn → ℝn. Such problems arise in many applications, such as the analysis of nonlinear electronic circuits, chemical equilibrium problems, or chemical process design. Nonlinear systems of equations must also frequently be solved as subtasks in solving problems involving differential equations. For example, the solution of initial value problems for stiff ordinary differential equations requires implicit methods (see Section 4.5), which in each time step solve a nonlinear system, and nonlinear boundary value problems for ordinary differential equations are often solved by multiple shooting methods where large banded nonlinear systems must be solved to ensure the correct matching of pieces of the solution. Nonlinear partial differential equations are reduced by finite element methods to solving sequences of huge structured nonlinear systems. Because most physical processes, whether in satellite orbit calculations, weather forecasting, oil recovery, or electronic chip design, can be described by nonlinear differential equations, the efficient solution of systems of nonlinear equations is of considerable practical importance.

Stationary points of scalar multivariate function f : D ⊆ ℝn → ℝ lead to a nonlinear system of equations for the gradient, ∇f(x) = 0. The most important case, finding the extrema of such functions, occurs frequently in industrial applications where some practical objective such as profit, quality, weight, cost, or loss must be maximized or minimized, but also in thermodynamical or mechanical applications where some energy functional is to be minimized.