Hostname: page-component-77c89778f8-vpsfw Total loading time: 0 Render date: 2024-07-17T14:28:21.607Z Has data issue: false hasContentIssue false
  • English
  • Français

On the discrepancy principle and generalised maximum likelihood for regularisation

Published online by Cambridge University Press:  17 April 2009

Mark A. Lukas
Mark A. Lukas
Affiliation:
School of Mathematical and Physical SciencesMurdoch UniversityMurdoch WA 6150Australia
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Let fnλ be the regularised solution of a general, linear operator equation, K f0 = g, from discrete, noisy data yi = g(xi ) + εi, i = 1, …, n, where εi are uncorrelated random errors with variance σ2. In this paper, we consider the two well–known methods – the discrepancy principle and generalised maximum likelihood (GML), for choosing the crucial regularisation parameter λ. We investigate the asymptotic properties as n → ∞ of the “expected” estimates λD and λM corresponding to these two methods respectively. It is shown that if f0 is sufficiently smooth, then λD is weakly asymptotically optimal (ao) with respect to the risk and an L2 norm on the output error. However, λD oversmooths for all sufficiently large n and also for all sufficiently small σ2. If f0 is not too smooth relative to the regularisation space W, then λD can also be weakly ao with respect to a whole class of loss functions involving stronger norms on the input error. For the GML method, we show that if f0 is smooth relative to W (for example f0 ∈ Wθ, 2, θ > m, if W = Wm, 2), then λM is asymptotically sub-optimal and undersmoothing with respect to all of the loss functions above.

Type
Research Article
Information
Bulletin of the Australian Mathematical Society , Volume 52 , Issue 3 , December 1995 , pp. 399 - 424
Copyright
Copyright © Australian Mathematical Society 1995

References

[1]Anderssen, R.S. and Bloomfield, P., ‘Numerical differentiation procedures for non-exact data’, Numer. Math. 22 (1974), 157–182.CrossRefGoogle Scholar
[2]Cox, D.D., ‘Approximation of regularization estimators’, Ann. Statist. 16 (1988), 694–712.CrossRefGoogle Scholar
[3]Craven, P. and Wahba, G., ‘Smoothing noisy data with spline functions’, Numer. Math. 31 (1979), 377–403.CrossRefGoogle Scholar
[4]Davies, A.R., ‘On the maximum likelihood regularization of Fredholm convolution equations of the first kind’, in Treatment of integral equations by numerical methods, (Baker, C.T.H. and Miller, G.F., Editors) (Academic Press, London, 1982), pp. 95–105.Google Scholar
[5]Davies, A.R., Iqbal, M., Maleknejad, K. and Redshaw, T.C., ‘A comparison of statistical regularization and Fourier extrapolation methods for numerical deconvolution’, in Numerical treatment of inverse problems in differential and integral equations, (Deuflhard, P. and Hairer, E., Editors)(Birkhäuser, Berlin, 1983), pp. 320–334.CrossRefGoogle Scholar
[6]Davies, A.R. and Anderssen, R.S., ‘Improved estimates of statistical regularization parameters in Fourier differentiation and smoothing’, Numer. Math. 48 (1986), 671–697.CrossRefGoogle Scholar
[7]Davies, A.R. and Anderssen, R.S., ‘Optimization in the regularization of ill-posed problems’, J. Austral. Math. Soc. Ser. B 28 (1986), 114–133.CrossRefGoogle Scholar
[8]Engl, H.W. and Gfrerer, H., ‘A posteriori parameter choice for general regularization methods for solving linear ill-posed problems’, Appl. Numer. Math. 4 (1988), 395–417.CrossRefGoogle Scholar
[9]Kimeldorf, G.S. and Wahba, G., ‘Some results on Tchebycheffian spline functions’, J. Math. Anal. Appl. 33 (1971), 82–95.CrossRefGoogle Scholar
[10]Lukas, M.A., ‘Regularization’, in The application and numerical solution of integral equations, (Anderssen, R.S., de Hoog, F.R. and Lukas, M.A., Editors)(Sijthoff and Noordhoff, Alphen aan den Rijn, The Netherlands, 1980), pp. 151–182.CrossRefGoogle Scholar
[11]Lukas, M.A., ‘Convergence rates for regularized solutions’, Math. Comp. 51 (1988), 107–131.CrossRefGoogle Scholar
[12]Lukas, M.A., ‘Assessing regularised solutions’, J. Austral. Math. Soc. Ser. B 30 (1988), 24–42..CrossRefGoogle Scholar
[13]Lukas, M.A., ‘Methods for choosing the regularization parameter’, in Inverse problems in partial differential equations, (Pani, A.K. and Anderssen, R.S., Editors), Proceedings of the Centre for Mathematics and its Applications (Australian National University, Canberra, ACT, 1992), pp. 89–110.Google Scholar
[14]Lukas, M.A., ‘Asymptotic optimality of generalized cross-validation for choosing the reg-ularization parameter’, Numer. Math. 66 (1993), 41–66.CrossRefGoogle Scholar
[15]Lukas, M.A., ‘Convergence rates for moment collocation solutions of linear operator equations’, Numer. Funct. Anal. Optim. (to appear).Google Scholar
[16]Morozov, V.A., ‘On the solution of functional equations by the method of regularization’. Soviet Math. Dokl. 7 (1966), 414–417.Google Scholar
[17]Morozov, V.A., Methods for solving incorrectly posed problems (Springer-Verlag, Berlin, Heidelberg, New York, 1984).CrossRefGoogle Scholar
[18]Nychka, D.W. and Cox, D.D., ‘Convergence rates for regularized solutions of integral equations from discrete noisy data’, Ann. Statist. 17 (1989), 556–572.CrossRefGoogle Scholar
[19]Wahba, G., ‘Convergence rates of certain approximate solutions to Fredholm integral equations of the first kind’, J. Approx. Theory 7 (1973), 167–185.CrossRefGoogle Scholar
[20]Wahba, G., ‘Smoothing noisy data by spline functions’, Numer. Math. 24 (1975), 383–393.CrossRefGoogle Scholar
[21]Wahba, G., ‘Practical approximate solutions to linear operator equations when the data are noisy’, SIAM J. Numer. Anal. 14 (1977), 651–667.CrossRefGoogle Scholar
[22]Wahba, G., ‘A comparison of GCV and GML for choosing the smoothing parameter in the generalized spline smoothing problem’, Ann. Statist. 13 (1985), 1378–1402.CrossRefGoogle Scholar
[23]Wahba, G., Spline models for observational data, CBMS-NSF Regional Conference Series in Applied Mathematics 59 (SIAM, Philadelphia, 1990).CrossRefGoogle Scholar