We establish functional central limit theorems (FCLTs) for a cumulative input process to a fluid queue from the superposition of independent on–off sources, where the on periods and off periods may have heavy-tailed probability distributions. Variants of these FCLTs hold for cumulative busy-time and idle-time processes associated with standard queueing models. The heavy-tailed on-period and off-period distributions can cause the limit process to have discontinuous sample paths (e.g., to be a non-Brownian stable process or more general Lévy process) even though the converging processes have continuous sample paths. Consequently, we exploit the Skorohod M1 topology on the function space D of right-continuous functions with left limits. The limits here combined with the previously established continuity of the reflection map in the M1 topology imply both heavy-traffic and non-heavy-traffic FCLTs for buffer-content processes in stochastic fluid networks.

We study an example of R. Bellman's gold-mining problem related to a programming job on the computer. The problem is formulated by dynamic programming and the optimal strategy is explicitly derived. The Bayesian version when the parameter involved is unknown is also solved by the same method. It is shown that the optimal strategy in each of two versions has the “no-island” (or, in other words, “control-limit”) property.

We examine several approaches to derive lower and upper bounds on tail probabilities of discrete distributions. Numerical comparisons exemplify the quality of the bounds.

In [7], several conjectures are listed about uniformly most reliable graphs, and, to date, no counterexamples have been found. These include the conjectures that an optimal reliable graph has degrees that are almost regular, has maximum girth, and has minimum diameter. In this article, we consider simple graphs and present one counterexample and another possible counterexample of these conjectures: maximum girth (i.e., maximize the length of the shortest circuit of the graph G) and minimum diameter (i.e., minimize the maximum possible distance between any pair of vertices).

The classical regenerative method of simulation output analysis exploits the regenerative structure of a stochastic process to break up a path into independent and identically distributed cycles based on a single sequence of regeneration times. If a process is regenerative with respect to more than one sequence of regeneration times, the classical regenerative method does not exploit the additional structure, and the variance of the resulting estimator for certain performance measures (e.g., the time-average variance constant) can vary greatly, depending on the particular regeneration sequence chosen. In a previous article, we introduced an efficiency-improvement technique for regenerative simulation of processes having two sequences of regeneration times based on permuting regenerative cycles associated with the second sequence of regeneration points. In this article, we show how to exploit more than two regeneration sequences. In particular, for birth–death Markov chains, the regenerations associated with hitting times to each state can all be exploited. We present empirical results that show significant variance reductions in some cases, and the results seem to indicate that the permuted estimator for the time-average variance constant can have a variance that is independent of the primary regeneration sequence used to run the simulation.

In this article, we formalize the framework for the supplementary variable technique in stochastic models. Specifically, we show that the use of remaining or elapsed times (or any metric) as supplementary variables leads to the notion of forward or backward Chapman–Kolmogorov equations, respectively. We further show that for a class of queueing systems, using remaining time as the supplementary variable makes analysis simpler.

We investigate the sensitivity of the leaky-bucket filter, which is known as a traffic-shaping mechanism of source traffic entering an ATM network. Adopting the gradual input source model and using the infinitesimal perturbation analysis, we derive the strongly consistent estimators for the gradients of mean cell delay and a smoothness index of output process with respect to the design parameters of the system. Simulation results show the validity of our estimators.

A simulation-based algorithm for learning good policies for a discrete-time stochastic control process with unknown transition law is analyzed when the state and action spaces are compact subsets of Euclidean spaces. This extends the Q-learning scheme of discrete state/action problems along the lines of Baker [4]. Almost sure convergence is proved under suitable conditions.