Hostname: page-component-848d4c4894-r5zm4 Total loading time: 0 Render date: 2024-06-30T22:23:34.118Z Has data issue: false hasContentIssue false
  • English
  • Français

Convergence analysis of adaptive trust region methods

Published online by Cambridge University Press:  15 June 2007

Zhen-Jun Shi ,
Xiang-Sun Zhang  and
Jie Shen
Zhen-Jun Shi
Affiliation:
College of Operations Research and Management, Qufu Normal University, Rizhao, Shandong 276826, P.R. China, and Department of Computer & Information Science, University of Michigan-Dearborn, Michigan MI48128, USA; zjshi@umd.umich.edu or zjshi@qrnu.edu.cn
Xiang-Sun Zhang
Affiliation:
Institute of Applied Mathematics, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, P.O. Box 2734, Beijing 100080, P.R. China; zxs@amt.ac.cn
Jie Shen
Affiliation:
Department of Computer & Information Science, University of Michigan-Dearborn, Michigan MI48128, USA; shen@umich.edu
Get access

Abstract

In this paper, we propose a new class of adaptive trust region methods for unconstrained optimization problems and develop some convergence properties. In the new algorithms, we use the current iterative information to define a suitable initial trust region radius at each iteration. The initial trust region radius is more reasonable in the sense that the trust region model and the objective function are more consistent at the current iterate. The global convergence, super-linear and quadratic convergence rate are analyzed under some mild conditions. Numerical results show that some special adaptive trust region methods are available and efficient in practical computation.

Keywords

Adaptive trust region methodunconstrained optimizationglobal convergencesuper-linear convergence.
Type
Research Article
Information
RAIRO - Operations Research , Volume 41 , Issue 1 , January 2007 , pp. 105 - 121
Copyright
© EDP Sciences, ROADEF, SMAI, 2007

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Byrd, R.H., Nocedal, J. and Yuan, Y.X., Global convergence of a class of quasi-Newton methods on convex problems. SIAM J. Numer. Anal. 24 (1987) 1171–1190. CrossRef
A.R. Conn, N.I.M. Gould and P.L. Toint, Trust-Region Methods, Society for Industrial and Applied Mathematics (SIAM), Philadelphia PA (2000).
Corradi, G., A trust region algorithm for unconstrained optimization. Int. J. Comput. Math. 65 (1997) 109–119. CrossRef
Dai, Y.H., Xu, D.C., A new family of trust region algorithms for unconstrained optimization. J. Comput. Math. 21 (2003) 221–228.
R. Fletcher, Practical Method of Optimization, Unconstrained Optimization, Vol. 1, John Wiley, New York (1980).
Fan, J.Y., Ai, W.B. and Zhang, Q.Y., A line search and trust region a lgorithm with trust region radius converging to zero. J. Comput. Math. 22 (2004) 865–872.
Fu, J.H. and Sun, W.Y., Nonmonotone adaptive trust-region method for unconstrained optimization problems. Appl. Math. Comput. 163 (2005) 489–504.
Gertz, E.M., A quasi-Newton trust-region method. Math. Program. Ser. A 100 (2004) 447–470.
N.I.M. Gould, D. Orban, A. Sartenaer and P.L. Toint, Sensitivity of trust-region algorithms to their parameters. 4OR 3 (2005) 227–241.
Gould, N.I.M., Orban, D. and Toint, P.L., Numerical methods for large-scale nonlinear optimization. Acta Numer. 14 (2005) 299–361. CrossRef
Hei, L., A self-adaptive trust region algorithm. J. Comput. Math. 21 (2003) 229–236.
J.J. Moré, Recent developments in algorithms and software for trust region methods, in Mathematical Programming: The State of the Art, edited by A. Bachem, M. Grotchel and B. Korte, Springer- Verlag, Berlin (1983) 258–287.
Moré, J.J., B.S. Grabow and K.E. Hillstrom, Testing Unconstrained Optimization software. ACM Trans. Math. Software 7 (1981) 17–41. CrossRef
Mauricio, D. and Maculan, N., A trust region method for zero-one nonlinear programming. RAIRO Rech. Opér. 31 (1997) 331–341. CrossRef
J. Nocedal and S.J. Wright, Numerical Optimization. Springer-Verlag, New York (1999).
Ni, Q., A globally convergent method of moving asymptotes with trust region technique. Optim. Methods Softw. 18 (2003) 283–297. CrossRef
M.J.D. Powell, Convergence properties of a class minimization algorithms, in Nonlinear Programming 2, edited by O.L. Mangasarian, R.R. Meyer and S.M. Robinson, Academic Press, New York (1975) 1–27.
Powell, M.J.D., On the global convergence of trust region algorithms for unconstrained optimization. Math. Prog. 29 (1984) 297–303. CrossRef
Sartenaer, A., Automatic determination of an initial trust region in nonlinear programming. SIAM J. Sci. Comput. 18 (1997) 1788–1803. CrossRef
Shultz, G.A., Schnabel, R.B. and Byrd, R.H., A family of trust-region-based algorithms for unconstrained minimization with strong global convergence. SIAM J. Numer. Anal. 22 (1985) 47–67. CrossRef
Sun, W.Y., Nonmonotone trust region method for solving optimization problems. Appl. Math. Comput. 156 (2004) 159–174.
J. Sun, On piecewise quadratic Newton and trust region problems. Math. Programming, Ser. B 76 (1997) 451–467.
Z.J. Shi and H.Q. Wang, A new self-adaptive trust region method for unconstrained optimization. Technical Report, College of Operations Research and Management, Qufu Normal University (2004).
Zhang, J.Z. and Trust, C.X. Xu region dogleg path algorithms for unconstrained minimization, Management science in China (Kowloon, 1996). Ann. Oper. Res. 87 (1999) 407–418. CrossRef
Zhang, X.S., Trust region method in neural network. Acta Math. Appl. Sinica 12 (1996) 1–10. CrossRef
X.S. Zhang, Trust region method in neural network. Acta Math. Appl. Sinica (English Ser.) 13 (1997) 342–352.
Zhang, X.S., Zhang, J.L. and Liao, L.Z., An adaptive trust region method and its convergence. Science in China 45 (2002) 620-631.
Zhang, X.S., Chen, Z.W. and Zhang, J.L., A self-adaptive trust region method for unconstrained optimization. OR Transactions 5 (2001) 53–62.
Y.X. Yuan, A review of trust region algorithms for optimization, in ICIAM 99 (Edinburgh), Oxford Univ. Press, Oxford (2000) 271–282.
Yuan, Y.X., Trust region algorithms for nonlinear equations. Information 1 (1998) 7–20.