Introduction

In mathematics, the term optimization refers to the study of problems that have the following forms:

• given: a function f : A → R from a certain set A to the real numbers;

• sought: an element x0 in Asuch that f (x0) ≤ f (x), ∀x ∈ A(“minimization”) or such that f (x0) ≥ f (x)∀x ∈ A(“maximization”).

Typically, A is a certain subset of the Euclidean space Rn, often specified by a set of constraints, equalities, or inequalities that the members of A have to satisfy. The elements of A are called feasible solutions. The function f is called an objective function, or cost function. A feasible solution that minimizes (or maximizes, if that is the goal) the objective function is called an optimal solution. The domain A of f is called the search space, and the elements of A are called candidate solutions or feasible solutions.

Such a formulation is sometimes called a mathematical program. Many real-world and theoretical problems may be modeled in this general framework. In this chapter, we discuss the following major subfields of the mathematical programming:

1. Linear programming (LP) studies the case in which the objective function f is linear and the set A is specified using only linear equalities and inequalities.

2. Convex programming studies the case in which the constraints and the optimization goals are all convex or linear.

3. Nonlinear programming (NLP) studies the general case in which the objective function or the constraints or both contain nonlinear parts.

4. Dynamic programming studies the case in which the optimization strategy is based on splitting the problem into smaller subproblems or considers the optimization problems over time.