Hostname: page-component-5c6d5d7d68-txr5j Total loading time: 0 Render date: 2024-08-14T02:15:25.327Z Has data issue: false hasContentIssue false
  • English
  • Français

GENERALIZED ADDITIVE PARTIAL LINEAR MODELS WITH HIGH-DIMENSIONAL COVARIATES

Published online by Cambridge University Press:  07 August 2013

Heng Lian  and
Hua Liang
Heng Lian
Affiliation:
Nanyang Technological University
Hua Liang
Affiliation:
University of Rochester
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper studies generalized additive partial linear models with high-dimensional covariates. We are interested in which components (including parametric and nonparametric components) are nonzero. The additive nonparametric functions are approximated by polynomial splines. We propose a doubly penalized procedure to obtain an initial estimate and then use the adaptive least absolute shrinkage and selection operator to identify nonzero components and to obtain the final selection and estimation results. We establish selection and estimation consistency of the estimator in addition to asymptotic normality for the estimator of the parametric components by employing a penalized quasi-likelihood. Thus our estimator is shown to have an asymptotic oracle property. Monte Carlo simulations show that the proposed procedure works well with moderate sample sizes.

Type
ARTICLES
Information
Econometric Theory , Volume 29 , Issue 6 , December 2013 , pp. 1136 - 1161
Copyright
Copyright © Cambridge University Press 2013 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

1

We sincerely thank Professor Oliver Linton and two anonymous reviewers for their insightful and constructive comments, which have improved the manuscript significantly. Lian’s research was supported by Singapore MOE Tier 1 RG 62/11. Liang’s research was partially supported by NSF grants DMS-1007167 and DMS-1207444 and by Award 11228103 from the National Natural Science Foundation of China. Address correspondence to Hua Liang, Department of Biostatistics and Computational Biology, University of Rochester, Rochester, NY 14642, USA; e-mail: Hua_Liang@URMC.Rochester.edu.

References

REFERENCES

Andrews, D. (1991) Asymptotic normality of series estimators for nonparametric and semiparametric regression models. Econometrica 59, 307–345.CrossRefGoogle Scholar
Andrews, D. & Whang, Y. (1990) Additive interactive regression models: Circumvention of the curse of dimensionality. Econometric Theory 6, 466–479.CrossRefGoogle Scholar
Belloni, A. & Chernozhukov, V. (2012) Post-l1-penalized estimators in high-dimensional linearregression models. Bernoulli, forthcoming.Google Scholar
Bickel, P., Ritov, Y., & Tsybakov, A. (2009) Simultaneous analysis of LASSO and Dantzig selector. Annals of Statistics 37, 1705–1732.CrossRefGoogle Scholar
Buja, A., Hastie, T., & Tibshirani, R. (1989) Linear smoothers and additive models. Annals of Statistics 17, 453–510.CrossRefGoogle Scholar
Burda, M.C., Härdle, W., Müller, M., & Werwatz, A. (1998) Semiparametric analysis of German East-West migration intentions: Facts and theory. Journal of Applied Econometrics 13, 525–541.Google Scholar
Chen, H. (1988) Convergence rates for parametric components in a partly linear model. Annals of Statistics 16, 136–146.CrossRefGoogle Scholar
Chen, X. (2007) Large sample sieve estimation of semi-nonparametric models. Handbook of Econometrics 6, 5549–5632.CrossRefGoogle Scholar
Donald, S. & Newey, W. (1994) Series estimation of semilinear models. Journal of Multivariate Analysis 50, 30–40.CrossRefGoogle Scholar
Du, P., Ma, S.G., & Liang, H. (2010) Penalized variable selection procedure for Cox models with semiparametric relative risk. Annals of Statistics 38, 2092–2117.Google ScholarPubMed
Engle, R., Granger, C., Rice, J., & Weiss, A. (1986) Semiparametric estimates of the relation between weather and electricity sales. Journal of the American Statistical Association 81, 310–320.CrossRefGoogle Scholar
Fan, J.Q. & Li, R. Z. (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96, 1348–1360.CrossRefGoogle Scholar
Fan, J.Q. & Peng, H. (2004) Nonconcave penalized likelihood with a diverging number of parameters. Annals of Statistics 32, 928–961.CrossRefGoogle Scholar
Friedman, J., Hastie, T., Höfling, H., & Tibshirani, R. (2007) Pathwise coordinate optimization. Annals of Applied Statistics 1, 302–332.CrossRefGoogle Scholar
Härdle, W., Liang, H., & Gao, J.T. (2000) Partially Linear Models. Springer Physica.CrossRefGoogle Scholar
Härdle, W., Mammen, E., & Müller, M. (1998) Testing parametric versus semiparametric modeling in generalized linear models. Journal of the American Statistical Association 93, 1461–1474.Google Scholar
Heckman, N.E. (1986) Spline smoothing in partly linear models. Journal of the Royal Statistical Society, Series B 48, 244–248.Google Scholar
Hiirdle, W., Müller, M., Sperlich, S., & Werwatz, A. (2004) Nonparametric and Semiparametric Models. Springer-Verlag.CrossRefGoogle Scholar
Horowitz, J. (1998) Semiparametric Methods in Econometrics. Springer-Verlag.CrossRefGoogle Scholar
Huang, J. (1998) Functional ANOVA models for generalized regression. Journal of Multivariate Analysis 67, 49–71.CrossRefGoogle Scholar
Huang, J., Horowitz, J. L., & Wei, F. (2010) Variable selection in nonparametric additive models. Annals of Statistics 38, 2282–2313.CrossRefGoogle ScholarPubMed
Huang, J.H.Z., Wu, C.O., & Zhou, L. (2004) Polynomial spline estimation and inference for varying coefficient models with longitudinal data. Statistica Sinica 14, 763–788.Google Scholar
Juhl, T. & Xiao, Z. (2005) Partially linear models with unit roots. Econometric Theory 21, 877–906.CrossRefGoogle Scholar
Kneib, T., Konrath, S., & Fahrmeir, L. (2011) High dimensional structured additive regression models: Bayesian regularization, smoothing and predictive performance. Journal of the Royal Statistical Society, Series C 60, 51–70.CrossRefGoogle Scholar
Lam, C. & Fan, J. (2008) Profile-kernel likelihood inference with diverging number of parameters. Annals of Statistics 36, 2232–2260.Google ScholarPubMed
Leeb, H. & Pötscher, B. (2005) Model selection and inference: Facts and fiction. Econometric Theory 21, 21–59.CrossRefGoogle Scholar
Leeb, H. & Pötscher, B. (2008) Sparse estimators and the oracle property, or the return of Hodges’ estimator. Journal of Econometrics 142, 201–211.CrossRefGoogle Scholar
Li, Q. (2000) Efficient estimation of additive partially linear models. International Economic Review 41, 1073–1092.CrossRefGoogle Scholar
Li, Q. & Stengos, T. (1996) Semiparametric estimation of partially linear panel data models. Journal of Econometrics 71, 389–397.CrossRefGoogle Scholar
Li, Q. & Wooldridge, J.M. (2002) Semiparametric estimation of partially linear models for dependent data with generated regressors. Econometric Theory 18, 625–645.CrossRefGoogle Scholar
Li, R. & Liang, H. (2008) Variable selection in semiparametric regression modeling. Annals of Statistics 36, 261–286.CrossRefGoogle ScholarPubMed
Linton, O. & Härdle, W. (1996) Estimation of additive regression models with known links. Biometrika 83, 529–540.CrossRefGoogle Scholar
Linton, O. & Nielsen, J. (1995) A kernel method of estimating structured nonparametric regression based on marginal integration. Biometrika 82, 93–100.CrossRefGoogle Scholar
Liu, X., Wang, L., & Liang, H. (2011) Estimation and variable selection for semiparametric additive partial linear models. Statistica Sinica 21, 1225–1248.CrossRefGoogle ScholarPubMed
Marx, B. & Eilers, P. (1998) Direct generalized additive modeling with penalized likelihood. Computational Statistics & Data Analysis 28, 193–209.CrossRefGoogle Scholar
Müller, M. & Rönz, B. (2000) Credit Scoring Using Semiparametric Methods. Springer Lecture Notes in Statistics. Springer-Verlag.CrossRefGoogle Scholar
Newey, W. (1997) Convergence rates and asymptotic normality for series estimators. Journal of Econometrics 79, 147–168.CrossRefGoogle Scholar
Robinson, P.M. (1988) Root n-consistent semiparametric regression. Econometrica 56, 931–954.Google Scholar
Ruppert, D., Wand, M., & Carroll, R. (2003) Semiparametric Regression. Cambridge University Press.CrossRefGoogle Scholar
Severini, T.A. & Staniswalis, J.G. (1994) Quasi-likelihood estimation in semiparametric models. Journal of the American Statistical Association 89, 501–511.CrossRefGoogle Scholar
Speckman, P.E. (1988) Kernel smoothing in partial linear models. Journal of the Royal Statistical Society, Series B 50, 413–436.Google Scholar
Stone, C. (1986) The dimensionality reduction principle for generalized additive models. Annals of Statistics 14, 590–606.CrossRefGoogle Scholar
Stone, C. (1994) The use of polynomial splines and their tensor products in multivariate function estimation. Annals of Statistics 22, 118–171.CrossRefGoogle Scholar
Su, L. & Jin, S. (2010) Profile quasi-maximum likelihood estimation of partially linear spatial autoregressive models. Journal of Econometrics 157, 18–33.CrossRefGoogle Scholar
Tibshirani, R. (1996) Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 58, 267–288.Google Scholar
van der Geer, S.A. (2000) Applications of Empirical Process Theory. Cambridge University Press.Google Scholar
Wang, H., Li, R., & Tsai, C.L. (2007) Tuning parameter selectors for the smoothly clipped absolute deviation method. Biometrika 94, 553–568.CrossRefGoogle ScholarPubMed
Wang, H.S. & Xia, Y.C. (2009) Shrinkage estimation of the varying coefficient model. Journal of the American Statistical Association 104, 747–757.CrossRefGoogle Scholar
Wang, L., Liu, X., Liang, H., & Carroll, R. (2011) Estimation and variable selection for generalized additive partially linear models. Annals of Statistics 39, 1827–1851.CrossRefGoogle Scholar
Wood, S. (2004) Stable and efficient multiple smoothing parameter estimation for generalized additive models. Journal of the American Statistical Association 99, 673–686.CrossRefGoogle Scholar
Wu, G. & Xiao, Z. (2002) A generalized partially linear model of asymmetric volatility. Journal of Empirical Finance 9, 287–319.CrossRefGoogle Scholar
Xue, L. (2009) Consistent variable selection in additive models. Statistica Sinica 19, 1281–1296.Google Scholar
Xue, L. & Yang, L. (2006) Additive coefficient modeling via polynomial spline. Statistica Sinica 16, 1423–1446.Google Scholar
Yatchew, A. (2003) Semiparametric Regression for the Applied Econometrician. Cambridge University Press.CrossRefGoogle Scholar
Yu, K., Park, B., & Mammen, E. (2008) Smooth backfitting in generalized additive models. Annals of Statistics 36, 228–260.CrossRefGoogle Scholar
Yuan, M. & Lin, Y. (2006) Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society, Series B 68, 49–67.CrossRefGoogle Scholar
Zhang, C.H. & Huang, J. (2008) The sparsity and bias of the lasso selection in high-dimensional linear regression. Annals of Statistics 36, 1567–1594.CrossRefGoogle Scholar
Zhang, H., Cheng, G., & Liu, Y. (2011) Linear or nonlinear? Automatic structure discovery for partially linear models. Journal of the American Statistical Association 106, 1099–1112.CrossRefGoogle ScholarPubMed
Zou, H. (2006) The adaptive LASSO and its oracle properties. Journal of the American Statistical Association 101, 1418–1429.CrossRefGoogle Scholar
Zou, H. & Li, R.Z. (2008) One-step sparse estimates in nonconcave penalized likelihood models. Annals of Statistics 36, 1509–1533.CrossRefGoogle ScholarPubMed