Hostname: page-component-77c89778f8-vsgnj Total loading time: 0 Render date: 2024-07-19T06:23:17.546Z Has data issue: false hasContentIssue false
  • English
  • Français

Optimal sequential selection and resource allocation under uncertainty

Published online by Cambridge University Press:  01 July 2016

Shirish D. Chikte
Shirish D. Chikte*
Affiliation:
The University of Rochester
*
∗Postal address: Department of Electrical Engineering, The University of Rochester, Rochester, NY 14627, U.S.A.
Get access Rights & Permissions [Opens in a new window]

Abstract

Consider a decision-maker who is in charge of a number of activities. At each of a sequence of decision points in time, he selects—on the basis of their performance—the set of activities to be continued further and allocates his limited resources among them. Activities receiving larger allocations tend to improve their performance, while others receiving smaller allocations tend to deteriorate. We present a controlled random walk model for the progress of these activities. The problem of maximizing the net infinite horizon discounted return is formulated in the framework of Markov decision theory, and existence of optimal strategies established. It is shown that both the optimal selection and allocation strategies exhibit a ‘favoring the leaders' behavior. Finally, explicit solutions to certain special cases are obtained illustrating these results.

Keywords

OPTIMAL STOCHASTIC RESOURCE ALLOCATIONOPTIMAL SELECTIONRANDOM WALKMARKOV DECISION THEORY
Type
Research Article
Information
Advances in Applied Probability , Volume 12 , Issue 4 , December 1980 , pp. 942 - 957
Copyright
Copyright © Applied Probability Trust 1980 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Bertsekas, D. P. (1977) Monotone mappings with application in dynamic programming. SIAM J. Control and Optimization 15, 438–464.Google Scholar
[2] Blackwell, D. (1965) Discounted dynamic programming. Ann. Math. Statist. 36, 226–235.Google Scholar
[3] Chikte, S. D. (1977) Markov Decision Models for Optimal Stochastic Resource Allocation Problems. Ph.D. Thesis, Polytechnic Institute of New York.Google Scholar
[4] Chikte, S. D. and Deshmukh, S. D. (1977) Preventive maintenance and replacement under additive damage. Discussion Paper No. 37, Centre for Statistics and Probability, Northwestern University.Google Scholar
[5] Degroot, M. H. (1970) Optimal Statistical Decisions. McGraw-Hill, New York.Google Scholar
[6] Denardo, E. V. (1967) Contraction mappings in the theory underlying dynamic programming. SIAM Rev. 9, 165–177.Google Scholar
[7] Derman, C. (1963) On optimal replacement rules when changes of state are Markovian. Chapter 9 of Mathematical Optimization Techniques , ed. Bellman, R., University of California Press.Google Scholar
[8] Deshmukh, S. D. and Chikte, S. D. (1977) Stochastic evolution and control of an economic activity. J. Econom. Theory 15, 112–122.Google Scholar
[9] Deshmukh, S. D. and Chikte, S. D. (1977) Dynamic investment strategies for a risky R and D project. J. Appl. Prob. 14, 144–152.Google Scholar
[10] Dubins, L. E. and Savage, L. J. (1976) Inequalities for Stochastic Processes: How to Gamble if You Must. Dover, New York.Google Scholar
[11] Gittins, J. C. (1969) Optimal resource allocation in chemical research. Adv. Appl. Prob. 1, 238–270.Google Scholar
[12] Gittins, J. C. (1972) Some problems of stochastic resource allocation. J. Appl. Prob. 9, 360–369.Google Scholar
[13] Hinderer, K. (1970) Foundations of Non-Stationary Dynamic Programming with Discrete Time Parameter. Springer-Verlag, New York.Google Scholar
[14] Howard, R. A. (1960) Dynamic Programming and Markov Processes. Wiley, New York.Google Scholar
[15] Laska, E., Meisner, M. and Siegel, C. (1972) Contributions to the theory of optimal resource allocation. J. Appl. Prob. 9, 337–359.Google Scholar
[16] Lehmann, E. L. (1955) Ordered families of distributions. Ann. Math. Statist. 26, 399–419.Google Scholar
[17] Lippman, S. A. (1975) On dynamic programming with unbounded rewards. Management Sci. 21, 1225–1233.Google Scholar
[18] McGill, J. (1969) Optimal control of a queueing system with variable number of exponential servers. Technical Report No. 123, Department of Operations Research and Department of Statistics, Stanford University.Google Scholar
[19] Nash, P. and Gittins, J. C. (1977) A Hamiltonian approach to optimal stochastic resource allocation. Adv. Appl. Prob. 9, 55–68.CrossRefGoogle Scholar
[20] Radner, R. and Rothschild, M. (1975) On the allocation of effort. J. Econom. Theory 10, 358–376.Google Scholar
[21] Winston, W. (1978) A stochastic game model of a weapons development competition. SIAM J. Control and Optimization 16, 411–419.Google Scholar