This article provides an insight into the history of interactive graphical control interfaces for computer-based and computer-assisted music applications and identifies a number of the challenges which currently face contemporary designers of new software and hardware tools in this context. By studying some of the key features of this rich and varied legacy, it is possible to establish some useful criteria to underpin the development of new and improved tools for manipulating visual representations of music data which can be as diverse as common music notation, acoustic spectra, and all manner of simulations modelling the physical control surfaces associated with devices such as commercial synthesizers and audio mixing consoles.