Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-23T13:27:52.831Z Has data issue: false hasContentIssue false
  • English
  • Français

Estimating Derivatives Via Poisson's Equation

Published online by Cambridge University Press:  27 July 2009

Bennett L. Fox  and
Paul Glasserman
Bennett L. Fox
Affiliation:
Department of MathematicsUniversity of Colorado Denver, Colorado 80217-3364
Paul Glasserman
Affiliation:
Graduate School of Business Columbia University New York, New York 10027
Get access Rights & Permissions [Opens in a new window]

Abstract

Let x(j) be the expected reward accumulated up to hitting an absorbing set in a Markov chain, starting from state j. Suppose the transition probabilities and the one-step reward function depend on a parameter, and denote by y(j) the derivative of x(j) with respect to that parameter. We estimate y(0) starting from the respective Poisson equations that x = [x(0),x(l),…] and y = [y(0),y(l),…] satisfy. Relative to a likelihood-ratio-method (LRM) estimator, our estimator generally has (much) smaller variance; in a certain sense, it is a conditional expectation of that estimator given x. Unlike LRM, however, we have to estimate certain components of x. Our method has broader scope than LRM: we can estimate sensitivity to opening arcs.

Type
Articles
Information
Probability in the Engineering and Informational Sciences , Volume 5 , Issue 4 , October 1991 , pp. 415 - 428
Copyright
Copyright © Cambridge University Press 1991

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Billingsley, P. (1968). Convergence of probability measures. New York: Wiley.Google Scholar
Bratley, P., Fox, B.L. & Schrage, L. (1987). A guide to simulation, 2nd ed.New York: Springer-Verlag.CrossRefGoogle Scholar
Cochran, W.G. (1977). Sampling techniques. New York: Wiley.Google Scholar
Fox, B.L. (1990). Computing cumulative reward to absorption and its gradient: Deterministic versus simulation methods. Technical Report, Mathematics Department, University of Colorado at Denver.Google Scholar
Fox, B.L. & Glynn, P.W. (1990). Discrete-time conversion for simulating finite-horizon Markov processes. SIAM Journal on Applied Mathematics 50: 1457–1473.CrossRefGoogle Scholar
Fox, B.L. & Glynn, P.W. Splitting as conditional Monte Carlo. Manuscript in preparation.Google Scholar
Fox, B.L. & Glynn, P.W. Manuscript in preparation.Google Scholar
Glasserman, P. (1990). Discrete-time ‘inversion’ and derivative estimation for Markov chains. Operations Research Letters 9: 305–313.CrossRefGoogle Scholar
Glynn, P.W. (1987). Likelihood ratio gradient estimation: An overview. In Thesen, A., Grant, H. & Kelton, W. David (eds.), Proceedings of the Winter Simulation Conference. San Diego, CA: Society for Computer Simulation, pp. 366–374.CrossRefGoogle Scholar
Goyal, A., Shahabuddin, P., Heidelberger, P., Nicola, V.F. & Glynn, P.W. (1989). Unified framework for simulating Markovian models of highly dependable systems. Technical Report RC 14772, IBM Research Division, Yorktown Heights, NY.Google Scholar
L'Ecuyer, P. (1990). A unified view of the IPA, SF and LR gradient estimation techniques. Management Science 36: 1364–1383.CrossRefGoogle Scholar
Rao, C.R. (1973). Linear statistical inference and its applications, 2nd ed.New York: Wiley.CrossRefGoogle Scholar
Ross, S.M. (1983). Stochastic processes. New York: Wiley.Google Scholar