Information theory was created by Claude E. Shannon for the study of certain quantitative aspects of information, primarily as an analysis of the impact of coding on information transmission. Research in this field has resulted in several mathematical theories. Our subject is the stochastic theory, often referred to as the Shannon theory, which directly descends from Shannon's pioneering work.

This book is intended for graduate students and research workers in mathematics (probability and statistics), electrical engineering and computer science. It aims to present a well-integrated mathematical discipline, including substantial new developments of the 1970s. Although applications in engineering and science are not covered, we hope to have presented the subject so that a sound basis for applications has also been provided. A heuristic discussion of mathematical models of communication systems is given in the Introduction, which also offers a general outline of the intuitive background for the mathematical problems treated in the book.

As the title indicates, this book deals with discrete memoryless systems. In other words, our mathematical models involve independent random variables with finite range. Idealized as these models are from the point of view of most applications, their study reveals the characteristic phenomena of information theory without burdening the reader with the technicalities needed in the more complex cases. In fact, the reader needs no other prerequisites than elementary probability and a reasonable mathematical maturity. By limiting our scope to the discrete memoryless case, it was possible to use a unified, basically combinatorial approach.