The elements of dithering and estimation provide tools to control the distribution of the quantization error, and to compute the average distortion. Another important parameter of source coding is the coding rate.

In this chapter we focus on the quantizer entropy as a measure for the coding rate. Although the lattice is unbounded, entropy coding keeps its coding rate finite. We examine the entropy-distortion trade-off for a general source and lattice quantizer in Sections 5.2–5.4, and compare it to Shannon's rate-distortion function R(D) – the ultimate compression rate of any system achieving a distortion level D – in Sections 5.5–5.6. As we shall see, the redundancy above R(D) is small for all sources; even for a simple scalar lattice quantizer it is at most ≈3/4 bit, and only ≈1/4 bit at high-resolution quantization. Furthermore, if we combine a Wiener filter at the quantizer output (as we did in Section 4.5), then the redundancy of a scalar ECDQ for a Gaussian source is at most ≈1/4 bit at any resolution.

The Shannon entropy

In fixed-rate lossless coding, all elements of the data are mapped into binary codewords of identical length. Let A denote the data alphabet. Since there are 2l binary words of length l, the codeword length must be at least the base-2 logarithm of the size of A, rounded up to the nearest integer.