In a stochastic control problem one observes and then seeks to favorably influence the behavior of some stochastic system. Such problems involve dynamic optimization, meaning that observations and actions are spread out in time. In this chapter several simple but fundamental stochastic control problems will be solved directly from first principles, with heavy reliance on the ubiquitous Ito formula. For each of the closely related problems to be considered, the optimal policy involves the imposition of control barriers; these may be either jump barriers or reflecting barriers, depending on the cost structure assumed.

Our first problem can be informally described as follows. Consider a controller who continuously monitors the contents of a storage system, such as an inventory or a bank account. In the absence of control, the contents process Z = {Zt, t ≥ 0} fluctuates as a (μ,σ) Brownian motion. The controller can at any time increase or decrease the contents of the system by any amount desired, but is obliged to keep Zt ≥ 0, and there are three types of costs to be considered. First, to increase the contents from x to x + δ, the controller must pay a fixed charge K plus a proportional charge kδ. Similarly, it costs L + lδ to decrease the contents from x to x – δ. Finally, inventory holding costs are continuously incurred at rate kZt.