Removing camera shake in smartphones without hardware stabilization

Filip Sroubek; Jan Flusser

doi:10.1017/CBO9781107360181.006

5 - Removing camera shake in smartphones without hardware stabilization

Published online by Cambridge University Press: 05 June 2014

Filip Sroubek and

Jan Flusser

Edited by

A. N. Rajagopalan and

Rama Chellappa

Show author details

Filip Sroubek: Affiliation:
Czech Academy of Sciences, Czech Republic
Jan Flusser: Affiliation:
Czech Academy of Sciences, Czech Republic
A. N. Rajagopalan: Affiliation:
Indian Institute of Technology, Madras
Rama Chellappa: Affiliation:
University of Maryland, College Park

Book contents

Get access

Summary

Introduction

Processing images has become an everyday practice in a wide range of applications in science and technology and we rely on our images with ever growing emphasis. Our understanding of the world is however limited by measuring devices that we use to acquire images. Inadequate measuring conditions together with technological limitations of the measuring devices result in acquired images that represent a degraded version of the “true” image.

Blur induced by camera motion is a frequent problem in photography – mainly when the light conditions are poor. As the exposure time increases, involuntary camera motion has a growing effect on the acquired image. Image stabilization (IS) devices that help to reduce the motion blur by moving the camera sensor in the opposite direction are becoming more common. However, such a hardware remedy has its limitations as it can only compensate for motion of a very small extent and speed. Deblurring the image offline using mathematical algorithms is usually the only choice we have to obtain a sharp image. Motion blur can be modeled by convolution and the deblurring process is called deconvolution, which is a well-known ill-posed problem. In general, the situation is even more complicated, since we usually have no or limited information about the blur shape.

We can divide the deconvolution methods into two categories: methods that estimate the blur and the sharp image directly from the acquired image (blind deconvolution), and methods that use information from other sensors to estimate the blur (semi-blind deconvolution).

Type: Chapter
Information: Motion Deblurring
Algorithms and Systems
, pp. 100 - 122

DOI: https://doi.org/10.1017/CBO9781107360181.006 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2014

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Ben-Ezra, M. & Nayar, S. K. (2004). Motion-based motion deblurring. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(6), 689–98.Google Scholar

Campisi, P. & Egiazarian, K., eds. (2007). Blind Image Deconvolution, Theory and Application. CRC Press.

Cho, S. & Lee, S. (2009). Fast motion deblurring. ACM Transactions on Graphics (Proceedings of SIGGRAPHA sia 2009), 28(5), 145:1–8.Google Scholar

Favaro, P., Burger, M. & Soatto, S. (2004). Scene and motion reconstruction from defocus and motion-blurred images via anisotropic diffusion. In T., Pajdla & J., Matas, eds., ECCV 2004, LNCS 3021. Berlin: Springer-Verlag, pp. 257–69.

Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T. & Freeman, W. T. (2006). Removing camera shake from a single photograph. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2006). New York: ACM, pp. 787–94.

Galatsanos, N. P., Mesarovic, V. Z., Molina, R. & Katsaggelos, A. K. (2000). Hierarchical Bayesian image restoration from partially known blurs. IEEE Transactions on Image Processing, 9(10), 1784–97.Google Scholar

Giannakis, G. & Heath, R. (2000). Blind identification of multichannel FIR blurs and perfect image restoration. IEEE Transactions on Image Processing, 9(11), 1877–96.Google Scholar

Gupta, A., Joshi, N., Zitnick, C. L., Cohen, M. & Curless, B. (2010). Single image deblurring using motion density functions. In Proceedings of the 11th European Conference on Computer Vision: Part I. European Conference on Computer Vision. Berlin: Springer-Verlag, pp. 171–84.

Gurelli, M. I. & Nikias, C. L. (1995). EVAM: An eigenvector based algorithm for multichannel blind deconvolution of input colored signals. IEEE International Conference on Acoustics, Speech and Signal Processing, 43, 134–49.Google Scholar

Haindl, M. & Šimberová, S. (2002). Model-based restoration of short-exposure solar images. In L., Jain & R., Howlett, eds., Frontiers in Artificial Intelligence and Applications. Vol. 87 of Knowledge-Based Intelligent Engeneering Systems. Amsterdam: IOS Press, pp. 697–706.

Harikumar, G. & Bresler, Y. (1999). Perfect blind restoration of images blurred by multiple filters: theory and efficient algorithms. IEEE Transactions on Image Processing 8(2), 202–19.Google Scholar

Harmeling, S., Michael, H. & Scholkopf, B. (2010). Space-variant single-image blind deconvolution for removing camera shake. In J., Lafferty, C. K. I., Williams, J., Shawe-Taylor, R., Zemel & A., Culotta, eds., Advances in Neural Information Processing Systems 23, pp. 829–37.

Hirsch, M., Schuler, C. J., Harmeling, S. & Scholkopf, B. (2011). Fast removal of nonuniform camera shake. In Proceedings of the IEEE International Computer Vision Conference, pp. 463–70.

Jia, J. (2007). Single image motion deblurring using transparency. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8.

Joshi, N., Kang, S. B., Zitnick, C. L. & Szeliski, R. (2010). Image deblurring using inertial measurement sensors. ACM Transactions on Graphics, 29, 30:1–9.Google Scholar

Levin, A., Weiss, Y., Durand, F. & Freeman, W. (2009). Understanding and evaluating blind deconvolution algorithms. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1964–71.

Levin, A., Weiss, Y., Durand, F. & Freeman, W. T. (2011). Understanding blind deconvolution algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(12), 2354–67.Google Scholar

Liu, R. & Jia, J. (2008). Reducing boundary artifacts in image deconvolution. In IEEE International Conference on Image Processing, pp. 505–8.

Miskin, J. & MacKay, D. J. (2000). Ensemble learning for blind image separation and deconvolution. In M., Girolani, ed., Advances in Independent Component Analysis, Springer-Verlag, pp. 123–42.

Molina, R., Mateos, J. & Katsaggelos, A. K. (2006). Blind deconvolution using a variational approach to parameter, image, and blur estimation. IEEE Transactions on Image Processing 15(12), 3715–27.Google Scholar

Nagy, J. G. & O'Leary, D. P. (1998). Restoring images degraded by spatially variant blur. SIAM Journal on Scientific Computing, 19(4), 1063–82.Google Scholar

Pai, H.-T. & Bovik, A. (2001). On eigenstructure-based direct multichannel blind image restoration. IEEE Transactions on Image Processing, 10(10), 1434–46.Google Scholar

Panci, G., Campisi, P., Colonnese, S. & Scarano, G. (2003). Multichannel blind image deconvolution using the bussgang algorithm: spatial and multiresolution approaches. IEEE Transactions on Image Processing, 12(11), 1324–37.Google Scholar

Pillai, S. & Liang, B. (1999). Blind image deconvolution using a robust GCD approach. IEEE Transactions on Image Processing, 8(2), 295–301.Google Scholar

Rosenfeld, A. & Kak, A. C. (1982). Digital Picture Processing, 2nd edn. Orlando, FL: Academic Press, Inc.

Rudin, L., Osher, S. & Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica D, 60, 259–68.Google Scholar

Schulz, T. (1993). Multiframe blind deconvolution of astronomical images. Journal of the Optical Society of America A, 10(5), 1064·73.Google Scholar

Shan, Q., Jia, J. & Agarwala, A. (2008). High-quality motion deblurring from a single image. In ACM Special Interest Group on Graphics and Interactive Techniques. NewYork: ACM, pp. 1-10.

Šorel, M. & Šroubek, F. (2009). Space-variant deblurring using one blurred and one underexposed image. In Proceedings of the IEEE International Conference on Image Processing, pp. 157–60.

Šorel, M., Šroubek, F. & Flusser, J. (2010). Towards super-resolution in the presence of spatially varying blur. In M., Peyman, ed., Super-resolution Imaging. CRC Press, pp. 187–217.

Šroubek, F. & Flusser, J. (2005). Multichannel blind deconvolution of spatially misaligned images. IEEE Transactions on Image Processing, 14(7), 874–83.Google Scholar

Stockham, T. G. Jr. (1966). High-speed convolution and correlation. In Proceedings of the April 26–28, 1966, Spring Joint Computer Conference. New York: ACM, pp. 229–33.

Tai, Y.-W., Du, H., Brown, M. S. & Lin, S. (2010). Correction of spatially varying image and video motion blur using a hybrid camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 1012–28.Google Scholar

Tico, M., Trimeche, M. & Vehvilainen, M. (2006). Motion blur identification based on differently exposed images. In Proceedings of the IEEE International Conference on Image Processing, pp. 2021–4.

Šorel, M. (2012). Removing boundary artifacts for real-time iterated shrinkage deconvolution. IEEE Transactions on Image Processing, 21(4), 2329–34.Google Scholar

Šorel, M. & Flusser, J. (2008). Space-variant restoration of images degraded by camera motion blur. IEEE Transactions on Image Processing, 17(2), 105–16.Google Scholar

Šroubek, F., Cristóbal, G. & Flusser, J. (2007). A unified approach to superresolution and multichannel blind deconvolution. IEEE Transactions on Image Processing, 16(9), 2322–32.Google Scholar

Šroubek, F. & Flusser, J. (2003). Multichannel blind iterative image restoration. IEEE Transactions on Image Processing, 12(9), 1094–106.Google Scholar

Šroubek, F., Šorel, M., Horackova, I. & Flusser, J. (2013). Patch-based blind deconvolution with parametric interpolation of convolution kernels. In Proceedings of the IEEE International Conference on Image Processing, pp. 1-4.

Whyte, O., Sivic, J., Zisserman, A. & Ponce, J. (2010). Non-uniform deblurring for shaken images. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 491–8.

Xu, L. & Jia, J. (2010). Two-phase kernel estimation for robust motion deblurring. In Proceedings of the 11th European Conference on Computer Vision: Part I. Berlin: Springer-Verlag, pp. 157–70.

Yuan, L., Sun, J., Quan, L. & Shum, H.-Y. (2007). Image deblurring with blurred/noisy image pairs. In ACM Special Interest Group on Graphics and Interactive Techniques, 1:1-10.Google Scholar

Book contents

5 - Removing camera shake in smartphones without hardware stabilization

Summary

Access options

References

Save book to Kindle

Save book to Dropbox

Save book to Google Drive