Introduction

In this chapter we consider the optimization of scalar filters for single-input single-output (SISO) channels. A number of optimization problems which arise in different contexts will be considered. In Sec. 10.2 we begin with the digital communication system of Fig. 10.1 for a fixed channel H(jω). We consider the optimization of the continuous-time prefilter (transmitted pulse shape) and postfilter (equalizer) to minimize the mean square reconstruction error under the zero-forcing condition on the product F(jω)H(jω)G(jω). The zero-forcing condition does not uniquely determine the above product. It will be shown that the optimal product (under the zero-forcing condition) is the so-called optimal compaction filter of the channel (Sec. 10.2.3). Usually the filters that result from the above optimization problem are ideal, unrealizable, filters and can only be approximated. The equivalent digital channel therefore requires further equalization. In Sec. 10.3 we consider the problem of jointly optimizing a digital prefilter–postfilter pair to minimize the mean square error. Both the zero-forcing and the non-ZF situation are considered.

Section 10.4 revisits Fig. 10.1 for an arbitrary channel H(jω) from a more general viewpoint and formulates some general conditions on the filters F(jω) and G(jω) for optimality. The most general forms of the postfilter and prefilter for optimality are established. These forms were first derived by Ericson [1971, 1973]. Using these results we can argue that the optimization of the continuoustime filters in Fig. 10.1 can always be reformulated as the optimization of a digital prefilter–postfilter pair.