Book contents
- Frontmatter
- Contents
- List of contributors
- Preface
- 1 Automatic code generation for real-time convex optimization
- 2 Gradient-based algorithms with applications to signal-recovery problems
- 3 Graphical models of autoregressive processes
- 4 SDP relaxation of homogeneous quadratic optimization: approximation bounds and applications
- 5 Probabilistic analysis of semidefinite relaxation detectors for multiple-input, multiple-output systems
- 6 Semidefinite programming, matrix decomposition, and radar code design
- 7 Convex analysis for non-negative blind source separation with application in imaging
- 8 Optimization techniques in modern sampling theory
- 9 Robust broadband adaptive beamforming using convex optimization
- 10 Cooperative distributed multi-agent optimization
- 11 Competitive optimization of cognitive radio MIMO systems via game theory
- 12 Nash equilibria: the variational approach
- Afterword
- Index
2 - Gradient-based algorithms with applications to signal-recovery problems
Published online by Cambridge University Press: 23 February 2011
- Frontmatter
- Contents
- List of contributors
- Preface
- 1 Automatic code generation for real-time convex optimization
- 2 Gradient-based algorithms with applications to signal-recovery problems
- 3 Graphical models of autoregressive processes
- 4 SDP relaxation of homogeneous quadratic optimization: approximation bounds and applications
- 5 Probabilistic analysis of semidefinite relaxation detectors for multiple-input, multiple-output systems
- 6 Semidefinite programming, matrix decomposition, and radar code design
- 7 Convex analysis for non-negative blind source separation with application in imaging
- 8 Optimization techniques in modern sampling theory
- 9 Robust broadband adaptive beamforming using convex optimization
- 10 Cooperative distributed multi-agent optimization
- 11 Competitive optimization of cognitive radio MIMO systems via game theory
- 12 Nash equilibria: the variational approach
- Afterword
- Index
Summary
This chapter presents, in a self-contained manner, recent advances in the design and analysis of gradient-based schemes for specially structured, smooth and nonsmooth minimization problems. We focus on the mathematical elements and ideas for building fast gradient-based methods and derive their complexity bounds. Throughout the chapter, the resulting schemes and results are illustrated and applied on a variety of problems arising in several specific key applications such as sparse approximation of signals, total variation-based image-processing problems, and sensor-location problems.
Introduction
The gradient method is probably one of the oldest optimization algorithms going back as early as 1847 with the initial work of Cauchy. Nowadays, gradient-based methods have attracted a revived and intensive interest among researches both in theoretical optimization, and in scientific applications. Indeed, the very large-scale nature of problems arising in many scientific applications, combined with an increase in the power of computer technology have motivated a “return” to the “old and simple” methods that can overcome the curse of dimensionality; a task which is usually out of reach for the current more sophisticated algorithms.
One of the main drawbacks of gradient-based methods is their speed of convergence, which is known to be slow. However, with proper modeling of the problem at hand, combined with some key ideas, it turns out that it is possible to build fast gradient schemes for various classes of problems arising in applications and, in particular, signal-recovery problems.
- Type
- Chapter
- Information
- Publisher: Cambridge University PressPrint publication year: 2009
- 125
- Cited by