We begin with the description of the most basic and primitive codes. Let A = a1, …, ak be a finite set: the set is called an alphabet, and its elements are called symbols. The main interest is in the set of finite sequences sn = aiaj … of the symbols of some length n, called messages or for us just data. The problem is to send or store them in a manner that costs the sending device little time and storage space. Again for practical reasons these devices use binary symbols 0 and 1, while the original symbols are represented as a sequence of binary symbols, such as the eight bits long “bytes.” Let for each symbol a in A, C: a ↦ C(a) be a one-to-one map, called a code, from the alphabet into the set of binary strings. It is extended to the messages by concatenation C : aiaj … an ↦ C(ai)C(aj) … C(an). Both binary strings C(a) and C(ai)C(aj) … C(an) are called codewords.

This will give us an upside down binary tree, with the root on top, whose nodes are the codewords, first of the symbols for n = 1 and then of the length 2 messages, and so on. The left hand tree in Figure 2.1 illustrates the code for the symbols of the alphabet A = {a, b, c}.