Rate distortion theory is the part of information theory which studies data compression with a fidelity criterion. In this chapter we consider the notion of rate-distortion function which is a theoretical limit for quantizer performances. The Blahut algorithm for finding the rate-distortion function numerically is given. In order to compare the performances of different quantizers, some results of the high-resolution quantization theory are discussed. Comparison of quantization procedures for the source with the generalized Gaussian distribution is performed.

Rate-distortion function

Each quantization procedure is characterized by the average distortion D and by the quantization rate R. The goal of compression system design is to optimize the rate-distortion tradeoff. In order to compare different quantizers, the rate-distortion function R(D) (Cover and Thomas 1971) is introduced. Our goal is to find the best quantization procedure for a given source. We say that for a given source at a given distortion D = D0, a quantization procedure with rate-distortion function R1(D) is better than another quantization procedure with rate-distortion function R2(D) if R1 (D0) ≤ R2 (D0). Unfortunately, very often it is difficult to point out the best quantization procedure. The reason is that the best quantizer can have very high computational complexity or sometimes, even, it can be unknown. On the other hand, it is possible to find the best rate-distortion function without finding the best quantization procedure. This theoretical lower limit for the rate at a given distortion is provided by the information rate distortion function (Cover and Thomas 1971).