A Geometrical Interpretation of Collinearity: A Natural Way to Justify Ridge Regression and Its Anomalies

Justifying ridge regression from a geometrical perspective is one of the main contributions of this paper. To the best of our knowledge, this question has not been treated previously. This paper shows that ridge regression is a particular case of raising procedures that provide greater flexibility by transforming the matrix X associated with the model. Thus, raising procedures, based on a geometrical idea of the vectorial space associated with the columns of matrix X, lead naturally to ridge regression and justify the presence of the well-known constant k on the main diagonal of matrix (XX)-X-'. This paper also analyses and compares different alternatives to raising with respect to collinearity mitigation. The results are illustrated with an empirical application.