An important theme in this book is the study of systems y = Φx. We have looked carefully at the case when Φ is linear and can be taken as an m × n matrix or more generally a linear operator on a Hilbert space. In the various applications we have considered thus far, x is the signal or input, Φ is the transform and y is the sample or output. We have studied in detail the reconstruction of x from y and have developed many tools to deal with this case.

This chapter is concerned with the parsimonious representation of data x, a fundamental problem in many sciences. Here, typically x is a vector in some high-dimensional vector space (object space), and Φ is an operation (often nonlinear) we perform on x, for example dimension reduction, reconstruction, classification, etc. The parsimonious representation of data typically means obtaining accurate models y of naturally occurring sources of data, obtaining optimal representations of such models, and rapidly computing such optimal representations. Indeed, modern society is fraught with many high-dimensional highly complex data problems, critical in diverse areas such as medicine, geology, critical infrastructure, health and economics, to name just a few.

This is an introduction to some topics in this subject showing how to apply mathematical tools we have already developed: linear algebra and SVD, Fourier transforms/series, wavelets/infinite products, compressive sampling and linear filters, PCA and dimension reduction/clustering, compression methods (linear and nonlinear).