Penalized regression via the restricted bridge estimator

YÜZBAŞI B., Arashi M., Akdeniz F.

SOFT COMPUTING, cilt.25, sa.13, ss.8401-8416, 2021 (SCI-Expanded) identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 25 Sayı: 13
  • Basım Tarihi: 2021
  • Doi Numarası: 10.1007/s00500-021-05763-9
  • Dergi Adı: SOFT COMPUTING
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, Applied Science & Technology Source, Compendex, Computer & Applied Sciences, INSPEC, zbMATH
  • Sayfa Sayıları: ss.8401-8416
  • Anahtar Kelimeler: Bridge regression, Restricted estimation, Machine learning, Quadratic approximation, Newton-Raphson, Variable selection, Multicollinearity, LINEAR-REGRESSION, VARIABLE SELECTION, RIDGE, LASSO, LIKELIHOOD, SHRINKAGE, ROBUST
  • İnönü Üniversitesi Adresli: Evet

Özet

This article is concerned with the bridge regression, which is a special family in penalized regression with penalty function Sigma(p)(j=1) |beta(j) (q) with q > 0, in a linear model with linear restrictions. The proposed restricted bridge (RBRIDGE) estimator simultaneously estimates parameters and selects important variables when a piece of prior information about parameters are available in either low-dimensional or high-dimensional case. Using local quadratic approximation, we approximate the penalty term around a local initial values vector. The RBRIDGE estimator enjoys a closed-form expression that can be solved when q > 0. Special cases of our proposal are the restricted LASSO (q = 1), restricted RIDGE (q = 2), and restricted Elastic Net (1 < q < 2) estimators. We provide some theoretical properties of the RBRIDGE estimator for the low-dimensional case, whereas the computational aspects are given for both low- and high-dimensional cases. An extensive Monte Carlo simulation study is conducted based on different prior pieces of information. The performance of the RBRIDGE estimator is compared with some competitive penalty estimators and the ORACLE. We also consider four real-data examples analysis for comparison sake. The numerical results show that the suggested RBRIDGE estimator outperforms outstandingly when the prior is true or near exact.