Using Hubble Space Telescope WFPC2 images of 3CR compact steep spectrum (CSS) radio sources and literature radio maps makes detailed comparison between optical and radio structures possible. All CSS sources for which the optical and radio orientation can reliably be measured, display good alignment between the optical and radio emission. This alignment is observed down to the lowest redshift in the sample, at z ~ 0.1. Optical emission from CSS galaxies is found to be dominated by this aligned component, especially at higher redshifts. Both the alignment and the relative brightness of this component indicate that CSS sources have large amounts of matter co-spatial with the radio structure. In order to assess the nature of the aligned light, we calculated line emission contamination in the passband using ground based spectra. From this we can conclude that at least in some sources (notably 3C 213.1, 3C346, and 3C 380) line emission is not important. The detailed optical–radio correspondence makes optical synchrotron the most likely mechanism for these three sources.