Hostname: page-component-5c6d5d7d68-lvtdw Total loading time: 0 Render date: 2024-08-18T15:21:35.931Z Has data issue: false hasContentIssue false
  • English
  • Français

Linear programming and continuous markovian decision problems

Published online by Cambridge University Press:  14 July 2016

Hisashi Mine  and
Yoshio Tabata
Hisashi Mine
Affiliation:
Kyoto University
Yoshio Tabata
Affiliation:
Kyoto University
Get access Rights & Permissions [Opens in a new window]

Summary

This paper is concerned with a continuous time parameter Markovian sequential decision process, and presents a method which transforms a given continuous parameter problem into a discrete one. It is proved that the optimal stationary policy for the resulting discrete time parameter Markovian decision process is also the optimal stationary policy for the original continuous one, and vice versa. The resulting discrete parameter problem may be more easily solved than the continuous one by applying the linear programming method. A simple numerical example is presented.

Type
Research Papers
Information
Journal of Applied Probability , Volume 7 , Issue 3 , December 1970 , pp. 657 - 666
Copyright
Copyright © Applied Probability Trust 1970 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Howard, R. A. (1960) Dynamic Programming and Markov Processes. Technology Press of M.I.T., Cambridge.Google Scholar
[2] Blackwell, D. (1962) Discrete dynamic programming. Ann. Math. Statist. 33, 719–726.Google Scholar
[3] Wolfe, P. and Dantzig, G. B. (1962) Linear programming in a Markov chain. Operat. Res. 10, 702–710.Google Scholar
[4] De Ghellinck, G. T. and Eppen, G. D. (1967) Linear programming solutions for separable Markovian decision problems. Management Sci. 13, 371–394.Google Scholar
[5] Fox, B. (1966) Markov renewal programming by linear fractional programming. SIAM. J. Appl. Math. 14, 1418–1432.Google Scholar
[6] Miller, B. L. (1968) Finite state continuous time Markov decision processes with an infinite planning horizon. J. Math. Anal. Appl. 22, 552–569.Google Scholar
[7] Chung, K. L. (1960) Markov Chains with Stationary Transition Probabilities. Springer, Berlin.Google Scholar
[8] Widder, D. V. (1946) The Laplace Transform. Princeton Univ. Press, Princeton, New Jersey.Google Scholar