Books in the Stochastic Modeling Series series

This book presents an algebraic development of the theory of countable state space Markov chains with discrete- and continuous-time parameters. A Markov chain is a stochastic process characterized by the Markov prop erty that the distribution of future depends only on the current state, not on the whole history.

This book presents similarity between Gaussian and non-Gaussian stable multivariate distributions and introduces the one-dimensional stable random variables. It discusses the most basic sample path properties of stable processes, namely sample boundedness and continuity.

Offers an approach for the study of constrained Markov decision processes. This book considers a controller that minimizes one cost objective, subject to inequality constraints on others. It is divided into three sections that build upon each other, providing frameworks and algorithms for a variety of applications.