Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-848d4c4894-wg55d Total loading time: 0 Render date: 2024-06-12T12:11:02.275Z Has data issue: false hasContentIssue false

12 - Relating to Three Dimensions

from Part IV - The 2D Image in a 3D World

Published online by Cambridge University Press:  25 October 2017

Wesley E. Snyder  and
Hairong Qi
Wesley E. Snyder
Affiliation:
North Carolina State University
Hairong Qi
Affiliation:
University of Tennessee
Get access

Summary

I've always been passionate about geometry and the study of three-dimensional forms.

– Erno Rubik

Introduction

Most of the images we see are projections of surfaces in the three-dimensional world around us. They result from light reflected off surfaces in that 3D world, passing through the lens of a camera, and intersecting the focal plane of the camera. These images that result from light reflected in this way are called “luminance” or “brightness” images in this book.

What was not described earlier in this book is the relationship between the 3D world and 2D images, including matching of one to another. We begin by reviewing the geometry of a simple projective camera and relating it to position of points in three dimensions. What we would really like is a range image from every scene, but that isn't always feasible, so we look at several aspects of the 2D–3D relationship:

  • • (Section 12.2) It is necessary to determine the 3D position of a point in space that is seen by two cameras, assuming the two cameras are known. This is the problem commonly known as stereopsis. When we say a camera is “known” we mean we know where it is, which way it is pointing, and all its internal parameters like focal length, resolution, and others.

  • • (Section 12.3) Actually it is not really necessary to know all about the cameras. Almost all the relevant information about both cameras can be determined if there are several points in each camera view that can be put into correspondence. An adequate solution to this problem leads to a wonderful little matrix called the fundamental matrix that contains all we need to know for the two-camera problem. Underlying this work is the correspondence problem, the problem of identifying which point in one image corresponds to which point in the other image. Finding a robust solution to the correspondence problem may be difficult.

  • • (Section 12.4) Once we have an approach to the correspondence problem, we can do partial matching of images and image stitching.

  • • (Section 12.5) If, instead of two cameras, we have one camera and a controllable light source, we can still find the 3D location of points in space. We address controllable lighting and how to achieve range imaging in this context.

    • Type
    Chapter
    Information
    Fundamentals of Computer Vision , pp. 303 - 349
    Publisher: Cambridge University Press
    Print publication year: 2017

    Access options

    Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

    References

    [12.1] M., Armstrong, A., Zisserman, and R., Hartley. Self-calibration from image triplets. Computer Vision –ECCV'96, pages 1–16, 1996.
    [12.2] J., Berkmann and T., Caelli. Computation of surface geometry and segmentation using covariance techniques. IEEE Trans. Pattern Anal. and Machine Intel., 16 (11), 1994.Google Scholar
    [12.3] G., Bilbro and W., Snyder. Range image restoration using mean field annealing. In Advances in Neural Network Information Processing Systems. Morgan-Kaufmann, 1989.
    [12.4] T., Binford. Visual perception by computer. In IEEE Conf. on Systems and Control, December 1971.
    [12.5] G., Blais and M., Levine. Registering multiview range data to create 3d computer objects. IEEE Trans. Pattern Anal. and Machine Intel., 17 (8), 1995.Google Scholar
    [12.6] K., Boyer, M., Mirza, and G., Ganguly. The robust sequential estimator: A general approach and its application to surface organization in range data. IEEE Trans. Pattern Anal. and Machine Intel., 16 (10), 1994.Google Scholar
    [12.7] D., Caspi, N., Kiryati, and J., Shamir. Range imaging with adaptive color structured light. IEEE Trans. Pattern Anal. and Machine Intel., 20 (5), May 1998.Google Scholar
    [12.8] M., Fischler and R., Bolles. Random sample consensus: A paradigm for model fitting with application to image analysis and automated cartography. Communications of the ACM, 24 (6), 1981.Google Scholar
    [12.9] A., Fitzgibbon, M., Pilu, and R., Fisher. Direct least square fitting of ellipses. IEEE Trans. Pattern Anal. and Machine Intel., 21 (5), May 1999.Google Scholar
    [12.10] P., Flynn. 3-d object recognition with symmetric models: Symmetry extraction and encoding. IEEE Trans. Pattern Anal. and Machine Intel., 16 (8), August 1994.Google Scholar
    [12.11] R., Gonzalez and P., Wintz. Digital Image Processing. Pearson, 1977.
    [12.12] A., Gross. Toward object-based heuristics. IEEE Trans. Pattern Anal. and Machine Intel., 16 (8), August 1994.Google Scholar
    [12.13] A., Gross and T., Boult. Recovery of SHGCs from a single intensity view. IEEE Trans. Pattern Anal. and Machine Intel., 18 (2), February 1996.Google Scholar
    [12.14] G., Gurevich. Foundations of the Theory of Algebraic Invariants. P. Nordcliff Ltd., The Netherlands, 1964.
    [12.15] R., Hartley and A., Zisserman. Multiple view geometry in computer vision, second edition, volume 2. Cambridge University Press, 2004.
    [12.16] G., Healey and R., Kondepudy. Radiometric CCD camera calibration and noise estimation. IEEE Trans. Pattern Anal. and Machine Intel., 16 (3), March 1994.Google Scholar
    [12.17] A., Hoover, D., Goldgof, and K., Bowyer. Extracting a valid boundary representation from a segmented range image. IEEE Trans. Pattern Anal. and Machine Intel., 17 (9), September 1995.Google Scholar
    [12.18] B. K. P., Horn. Robot Vision. MIT Press, 1986.
    [12.19] Y., Iwahori, R., Woodham, and A., Bagheri. Principal components analysis and neural network implementation of photometric stereo. In IEEE Conf. on Physics-Based Modeling in Computer Vision, 1995.
    [12.20] B., Julesz. Foundations of Cyclopean Perception. University of Chicago Press, 1971.
    [12.21] B., Karaçali and W., Snyder. Noise reduction in surface reconstruction from a given gradient field. International Journal of Computer Vision, 60 (1), October 2004.Google Scholar
    [12.22] K., Koster and M., Spann. MIR: An approach to robust clustering –application to range image segmentation. IEEE Trans. Pattern Anal. and Machine Intel., 22 (5), 2000.Google Scholar
    [12.23] A., Laurentini. The visual hull concept for silhouette-based image understanding. IEEE Trans. Pattern Anal. and Machine Intel., 16 (2), February 1994.Google Scholar
    [12.24] A., Laurentini. How far 3d shapes can be understood from 2d silhouettes. IEEE Trans. Pattern Anal. and Machine Intel., 17 (2), February 1995.Google Scholar
    [12.25] S., LaValle and S., Hutchinson. A Bayesian segmentation methodology for parametric image models. IEEE Trans. Pattern Anal. and Machine Intel., 17 (2), 1995.Google Scholar
    [12.26] S., Lavallee and R., Szeliski. Recovering the position and orientation for free-form objects from image contours using 3d distance maps. IEEE Trans. Pattern Anal. and Machine Intel., 17 (4), April 1995.Google Scholar
    [12.27] K., Lee, Y., Choy, and S., Cho. Geometric structure analysis of document images: A knowledge-based approach. IEEE Trans. Pattern Anal. and Machine Intel., 22 (11), 2000.Google Scholar
    [12.28] H. C., Longuet-Higgins. A computer algorithm for reconstructing a scene from two projections. Nature, September 1981.
    [12.29] J., Michel, N., Nandhakumar, and V., Velten. Thermophysical algebraic invariants from infrared imagery for object recognition. IEEE Trans. Pattern Anal. and Machine Intel., 19 (1), January 1997.Google Scholar
    [12.30] F., Mokhtarian. Silhouette-based isolated object recognition through curvature scale space. IEEE Trans. Pattern Anal. and Machine Intel., 17 (5), May 1995.Google Scholar
    [12.31] R., Morano, C., Ozturk, R., Conn, S., Dubin, S., Zietz, and J., Nissano. Structured light using pseudorandom codes. IEEE Trans. Pattern Anal. and Machine Intel., 20 (3), March 1998.Google Scholar
    [12.32] S., Nayar and R., Bolle. Reflectance based object recognition. Int. J. of Computer Vision, 1996.
    [12.33] S., Nayar and Y., Nakagawa. Shape from focus. IEEE Trans. Pattern Anal. and Machine Intel., 16 (8), August 1994.Google Scholar
    [12.34] M., Oren and S., Nayar. A theory of specular surface geometry. Int. J. of Computer Vision, 1996.
    [12.35] N., Page, W., Snyder, and S., Rajala. Turbine blade image processing system. In Proc. SPIE 0397, Applications of Digital Image Processing, volume 261, Oct 1983.Google Scholar
    [12.36] P., Rosen and G., West. Nonparametric segmentation of curves into various representations. IEEE Trans. Pattern Anal. and Machine Intel., 17 (12), 1995.Google Scholar
    [12.37] H., Schultz. Retrieving shape information from multiple image of a specular surface. IEEE Trans. Pattern Anal. and Machine Intel., 16 (2), February 1994.Google Scholar
    [12.38] H., Shum, K., Ikeuchi, and R., Reddy. Principal component analysis with missing data and its application to polyhedral object modeling. IEEE Trans. Pattern Anal. and Machine Intel., 17 (9), 1995.Google Scholar
    [12.39] W., Snyder and G., Bilbro. Segmentation of range images. In Int. Conference on Robotics and Automation, March 1985.
    [12.40] B., Soroka. Generalized cylinders from parallel slices. In Proceedings of the Conference on Pattern Recognition and Image Processing, 1979.
    [12.41] B., Soroka and R., Bajcsy. Generalized cylinders from serial sections. In 3rd Int. Joint Conf. on Pattern Recognition, Nov 1976.
    [12.42] M., Soucy and D., Laurendeau. A general surface approach to the integration of a set of range views. IEEE Trans. Pattern Anal. and Machine Intel., 17 (4), April 1995.Google Scholar
    [12.43] C., Stewart. Minipran: A new robust estimator for computer vision. IEEE Trans. Pattern Anal. and Machine Intel., 17 (10), 1995.Google Scholar
    [12.44] J., Stone and S., Isard. Adaptive scale filtering: A general method for obtaining shape from texture. IEEE Trans. Pattern Anal. and Machine Intel., 17 (7), July 1995.Google Scholar
    [12.45] M., Subbarao and T., Choi. Accurate recovery of three-dimensional shape from image focus. IEEE Trans. Pattern Anal. and Machine Intel., 17 (3), March 1995.Google Scholar
    [12.46] S., Sull and N., Ahuja. Integrated 3-d analysis and analysis-guided synthesis of flight image sequences. IEEE Trans. Pattern Anal. and Machine Intel., 16 (4), April 1994.Google Scholar
    [12.47] S., Sullivan, L., Sandford, and J., Ponce. Using geometric distance fits for 3-d object modeling and recognition. IEEE Trans. Pattern Anal. and Machine Intel., 16 (12), 1994.Google Scholar
    [12.48] Y., Sun, I., Liu, and J., Grady. Reconstruction of 3-d tree-like structures from three mutually orthogonal projections. IEEE Trans. Pattern Anal. and Machine Intel., 16 (3), March 1994.Google Scholar
    [12.49] B., Super and A., Bovik. Shape from texture using local spectral moments. IEEE Trans. Pattern Anal. and Machine Intel., 17 (4), April 1995.Google Scholar
    [12.50] K., Tarabanis, R., Tsai, and A., Kaul. Computing occlusion-free viewpoints. IEEE Trans. Pattern Anal. and Machine Intel., 18 (3), March 1996.Google Scholar
    [12.51] G., Taubin. Nonplanar curve and surface estimation in 3-space. In IEEE Robotics and Automation Conference, May 1988.
    [12.52] P., Torr and D., Murray. The development and comparison of robust methods for estimating the fundamental matrix. International Journal of Computer Vision, 24 (3), 1997.Google Scholar
    [12.53] E., Trucco and R., Fisher. Experiments in curvature-based segmentation of range data. IEEE Trans. Pattern Anal. and Machine Intel., 17 (2), 1995.Google Scholar
    [12.54] R., Wang, A., Hanson, and E., Riseman. Fast extraction of ellipses. In Ninth International Conference on Pattern Recognition, 1988.
    [12.55] M., Wani and B., Batchelor. Edge-region-based segmentation of range images. IEEE Trans. Pattern Anal. and Machine Intel., 16 (3), 1994.Google Scholar
    [12.56] I., Weiss and M., Ray. Model-based recognition of 3d objects from single images. IEEE Trans. Pattern Anal. and Machine Intel., 23 (2), 2001.Google Scholar
    [12.57] R., Woodham. Photometric method for determining surface orientation from multiple images. Optical Engineering, 19, 1980.Google Scholar
    [12.58] Y., Lei and K., Wong. Ellipse detection based on symmetry. Pattern Recognition Letters, 20, 1999.Google Scholar
    [12.59] X., Yu, T., Bui, and A., Kryzak. Robust estimation for range image segmentation and reconstruction. IEEE Trans. Pattern Anal. and Machine Intel., 16 (5), 1994.Google Scholar
    [12.60] M., Zerroug and R., Nevatia. Three dimensional descriptions based on the analysis of the invariant and quasi-invariant properties of some curved-axis generalized cylinders. IEEE Trans. Pattern Anal. and Machine Intel., 18 (3), March 1996.Google Scholar
    [12.61] R., Zhang, P., Tsai, J., Cryer, and M., Shah. Shape-from-shading: A survey. IEEE Trans. Pattern Anal. and Machine Intel., 21 (8), Aug 1999.Google Scholar
    [12.62] J., Zheng. Acquiring 3-d models from sequences of contours. IEEE Trans. Pattern Anal. and Machine Intel., 16 (2), February 1994.Google Scholar

    Save book to Kindle

    To save this book to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

    Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

    Find out more about the Kindle Personal Document Service.

    Available formats
    ×

    Save book to Dropbox

    To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

    Available formats
    ×

    Save book to Google Drive

    To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

    Available formats
    ×