In this chapter, we study how the viscosity solutions of HJB equations in Chapter 4 turn smooth. We first observe that the DPP holds for the value function v(x) with initial state x and v is a viscosity solution of the HJB equation associated with the problem. Next, the C2-regularity of v is shown by the following procedure:

1. (a) Consider the boundary value problem on any fixed ball, also called the Dirichlet problem, with the boundary condition given by v.

2. (b) In the Dirichlet problem, we have a classical solution w, which is also a viscosity solution.

3. (c) By the uniqueness of viscosity solutions, we obtain v = w and thus v is smooth.

This viscosity solutions technique is developed by D. Duffie et al. [51] in an optimal consumption/investment problem.

The plan of this chapter is as follows. We present a brief review of the Dirichlet problem for linear elliptic equations, and show the existence of classical solutions of the Dirichlet problem for HJB equations. Finally, we examine the important role of (a)–(c) in the stochastic linear-quadratic (LQ) control problems. As is well known in [40, 65] the control region for the stochastic linear regulator problem is unbounded and the associated HJB equation admits a classical solution. Based on the viscosity solutions technique, the optimal control policies of the LQ problems with bounded control regions are given. The technique (a)–(c) can be applied to economic problems in Part II.