Book contents
- Frontmatter
- Contents
- Preface
- Guide to Notation and Terminology
- 1 Brownian Motion
- 2 Stochastic Storage Models
- 3 Further Analysis of Brownian Motion
- 4 Stochastic Calculus
- 5 Optimal Stopping of Brownian Motion
- 6 Reflected Brownian Motion
- 7 Optimal Control of Brownian Motion
- 8 Brownian Models of Dynamic Inference
- 9 Further Examples
- Appendix A Stochastic Processes
- Appendix B Real Analysis
- References
- Index
7 - Optimal Control of Brownian Motion
Published online by Cambridge University Press: 05 December 2013
- Frontmatter
- Contents
- Preface
- Guide to Notation and Terminology
- 1 Brownian Motion
- 2 Stochastic Storage Models
- 3 Further Analysis of Brownian Motion
- 4 Stochastic Calculus
- 5 Optimal Stopping of Brownian Motion
- 6 Reflected Brownian Motion
- 7 Optimal Control of Brownian Motion
- 8 Brownian Models of Dynamic Inference
- 9 Further Examples
- Appendix A Stochastic Processes
- Appendix B Real Analysis
- References
- Index
Summary
In a stochastic control problem one observes and then seeks to favorably influence the behavior of some stochastic system. Such problems involve dynamic optimization, meaning that observations and actions are spread out in time. In this chapter several simple but fundamental stochastic control problems will be solved directly from first principles, with heavy reliance on the ubiquitous Ito formula. For each of the closely related problems to be considered, the optimal policy involves the imposition of control barriers; these may be either jump barriers or reflecting barriers, depending on the cost structure assumed.
Our first problem can be informally described as follows. Consider a controller who continuously monitors the contents of a storage system, such as an inventory or a bank account. In the absence of control, the contents process Z = {Zt, t ≥ 0} fluctuates as a (μ,σ) Brownian motion. The controller can at any time increase or decrease the contents of the system by any amount desired, but is obliged to keep Zt ≥ 0, and there are three types of costs to be considered. First, to increase the contents from x to x + δ, the controller must pay a fixed charge K plus a proportional charge kδ. Similarly, it costs L + lδ to decrease the contents from x to x – δ. Finally, inventory holding costs are continuously incurred at rate kZt.
- Type
- Chapter
- Information
- Brownian Models of Performance and Control , pp. 109 - 136Publisher: Cambridge University PressPrint publication year: 2013