In many visual effects applications, we need to relate images taken from different perspectives or at different times. For example, we often want to track a point on a set as a camera moves around during a shot so that a digital creature can be later inserted at that location. In fact, finding and tracking many such points is critical for algorithms that automatically estimate the 3D path of a camera as it moves around a scene, a problem called matchmoving that is the subject of Chapter 6. However, not every point in the scene is a good choice for tracking, since many points look alike. In this chapter, we describe the process of automatically detecting regions of an image that can be reliably located in other images of the same scene; we call these special regions features. Once the features in a given image have been found, we also discuss the problems of describing, matching, and tracking them in different images of the same scene.

In addition to their core use for matchmoving, feature detection is also important for certain algorithms that estimate dense correspondence between images and video sequences (Chapter 5), as well as for both marker-based and markerless motion capture (Chapter 7). Outside the domain of visual effects, feature matching and tracking is commonly used for stitching images together to create panoramas [72], localizing mobile robots [432], and quickly finding objects [456] or places [424] in video databases.