Stacked Elastic Net

Predictive and interpretable models via the stacked elastic net

Armin Rauschenberger1, 2, * AR, Enrico Glaab1, † EG, and Mark A. van de Wiel2, † MvdW

1 Luxembourg Centre for Systems Biomedicine (LCSB), University of Luxembourg, Luxembourg

2 Department of Epidemiology and Data Science, Amsterdam UMC, The Netherlands

* To whom correspondence should be addressed.

† These authors share senior authorship.

Abstract

Machine learning in the biomedical sciences should ideally provide predictive and interpretable models. When predicting outcomes from clinical or molecular features, applied researchers often want to know which features have effects, whether these effects are positive or negative and how strong these effects are. Regression analysis includes this information in the coefficients but typically renders less predictive models than more advanced machine learning techniques. Here, we propose an interpretable meta-learning approach for high-dimensional regression. The elastic net provides a compromise between estimating weak effects for many features and strong effects for some features. It has a mixing parameter to weight between ridge and lasso regularization. Instead of selecting one weighting by tuning, we combine multiple weightings by stacking. We do this in a way that increases predictivity without sacrificing interpretability. The R package starnet is available on GitHub and CRAN.

Full text (open access)

Rauschenberger et al. (2021). “Predictive and interpretable models via the stacked elastic net”. Bioinformatics 37(14):2012-2016. doi: 10.1093/bioinformatics/btaa535. (Click here to access PDF.)