We establish strong consistency (i.e., almost sure convergence) of infinitesimal perturbation analysis (IPA) estimators of derivatives of steady-state means for a broad class of systems. Our results substantially extend previously available results on steady-state derivative estimation via IPA.

Our basic assumption is that the process under study is regenerative, but our analysis uses regenerative structure in an indirect way: IPA estimators are typically biased over regenerative cycles, so straightforward differentiation of the regenerative ratio formula does not necessarily yield a valid estimator of the derivative of a steady-state mean. Instead, we use regeneration to pass from unbiasedness over fixed, finite time horizons to convergence as the time horizon grows. This provides a systematic way of extending results on unbiasedness to strong consistency.

Given that the underlying process regenerates, we provide conditions under which a certain augmented process is also regenerative. The augmented process includes additional information needed to evaluate derivatives; derivatives of time averages of the original process are time averages of the augmented process. Thus, through this augmentation we are able to apply standard renewal theory results to the convergence of derivatives.

Let x(j) be the expected reward accumulated up to hitting an absorbing set in a Markov chain, starting from state j. Suppose the transition probabilities and the one-step reward function depend on a parameter, and denote by y(j) the derivative of x(j) with respect to that parameter. We estimate y(0) starting from the respective Poisson equations that x = [x(0),x(l),…] and y = [y(0),y(l),…] satisfy. Relative to a likelihood-ratio-method (LRM) estimator, our estimator generally has (much) smaller variance; in a certain sense, it is a conditional expectation of that estimator given x. Unlike LRM, however, we have to estimate certain components of x. Our method has broader scope than LRM: we can estimate sensitivity to opening arcs.

We consider a communication network that can support both wideband video calls and narrowband data traffic. First we consider a single link and prove a weak convergence result to justify a piecewise-deterministic Markov process approximation to the system. We then generalize this approximation to allow priorities and more than one link. This second approximation is a generalization of the Erlang fixed-point approximation for loss networks and is justified via a diverse routing limit theorem.

In this paper we consider the problem of finding the optimal order for two servers in series when there is no queue capacity. We show that it is better for the faster server to be first. The strength of this conclusion will depend on the strength of the assumption made about the service distribution. We also find the optimal order for some systems where both servers have the same average service time and different service distributions.

A Markov decision chain with countable state space incurs two types of costs: an operating cost and a holding cost. The objective is to minimize the expected discounted operating cost, subject to a constraint on the expected discounted holding cost. The existence of an optimal randomized simple policy is proved. This is a policy that randomizes between two stationary policies, that differ in at most one state. Several examples from the control of discrete time queueing systems are discussed.

Generalized semi-Markov processes and stochastic Petri nets provide building blocks for specification of discrete event system simulations on a finite or countable state space. The two formal systems differ, however, in the event scheduling (clock-setting) mechanism, the state transition mechanism, and the form of the state space. We have shown previously that stochastic Petri nets have at least the modeling power of generalized semi-Markov processes. In this paper we show that stochastic Petri nets and generalized semi-Markov processes, in fact, have the same modeling power. Combining this result with known results for generalized semi-Markov processes, we also obtain conditions for time-average convergence and convergence in distribution along with a central limit theorem for the marking process of a stochastic Petri net.

This paper models primary computer storage in the context of a general (GI/GI/l) queueing system. Queued items are described by sizes, or storage requirements, as well as by arrival and service times; the sum of the sizes of the items in the system is the occupied storage. Capacity constraints are represented by two different protocols for determining whether an arriving item is admitted to the system: (1) an item is accepted if and only if at its arrival time the currently occupied storage does not exceed a given constant C > 0, and (2) an item is accepted if and only if at its arrival time the occupied storage is at most C, and the occupied storage plus the item's size is at most C(l + ε) for some given ε > 0. We prove for both systems that in heavy traffic the occupied storage, suitably normalized, converges weakly to reflected Brownian motion with boundaries at 0 and at C. A distinctive feature of the proof is the characterization of reflected Brownian motion as a limit of unrestricted penalized processes.

These results make more plausible an earlier conjecture of the authors, i.e., that one obtains the same heavy traffic limit when the admission rule is: accept an item if and only if at its arrival time the occupied storage plus the item's size is no greater than C.

Two multivariate aging classes described by Johnson and Kotz [7] and Zahedi [11] and known as vector multivariate increasing/decreasing hazard rate (VMIHR/VMDHR) class and decreasing/increasing multivariate mean remaining life of type 2 (DMMRL/IMMRL-2) class, respectively, as well as componentwise NBUE/NWUE class, are considered. The chain of implications between these classes and some characterizing properties of them with regard to the equilibrium distribution corresponding to a multivariate distribution function and concave (convex) transformation of the residual random vector have been established. The “closure under mixture’ property of the first two classes are also studied.