Book contents
- Frontmatter
- Contents
- List of contributors
- Preface
- 1 Introduction
- 2 Error-detecting codes
- 3 Repetition and Hamming codes
- 4 Data compression: efficient coding of a random message
- 5 Entropy and Shannon's Source Coding Theorem
- 6 Mutual information and channel capacity
- 7 Approaching the Shannon limit by turbo coding
- 8 Other aspects of coding theory
- References
- Index
4 - Data compression: efficient coding of a random message
Published online by Cambridge University Press: 05 June 2012
- Frontmatter
- Contents
- List of contributors
- Preface
- 1 Introduction
- 2 Error-detecting codes
- 3 Repetition and Hamming codes
- 4 Data compression: efficient coding of a random message
- 5 Entropy and Shannon's Source Coding Theorem
- 6 Mutual information and channel capacity
- 7 Approaching the Shannon limit by turbo coding
- 8 Other aspects of coding theory
- References
- Index
Summary
In this chapter we will consider a new type of coding. So far we have concentrated on codes that can help detect or even correct errors; we now would like to use codes to represent some information more efficiently, i.e. we try to represent the same information using fewer digits on average. Hence, instead of protecting data from errors, we try to compress it such as to use less storage space.
To achieve such a compression, we will assume that we know the probability distribution of the messages being sent. If some symbols are more probable than others, we can then take advantage of this by assigning shorter code-words to the more frequent symbols and longer codewords to the rare symbols. Hence, we see that such a code has codewords that are not of fixed length.
Unfortunately, variable-length codes bring with them a fundamental problem: at the receiving end, how do you recognize the end of one codeword and the beginning of the next? To attain a better understanding of this question and to learn more about how to design a good code with a short average codeword length, we start with a motivating example.
A motivating example
You would like to set up your own telephone system that connects you to your three best friends. The question is how to design efficient binary phone numbers. In Table 4.1 you find six different ways of how you could choose them.
- Type
- Chapter
- Information
- A Student's Guide to Coding and Information Theory , pp. 55 - 80Publisher: Cambridge University PressPrint publication year: 2012