Difference between Bayesian Ridge Regression and Automatic Relevance Determination Regression?

Question

Does anyone know what is the difference between Bayessian ridge regression and ARDR? To be more precise, between these two function from sklearn:

linear_model.BayesianRidge()  
linear_model.ARDRegression()

When I looked the theory behind, both are using lambda and weights, and ARDRegression is defined: Fit regression model with Bayesian Ridge Regression.

But still I don't get what is difference since both models are using weights and lambda to determine regression models. Can someone explain this more intuitively?

Answer 1

There is some further detail on this page. Specifically: ARDR proposes a different prior - assuming an elliptical Gaussian prior distribution where each weight has its own standard deviation, rather than a spherical Gaussian prior distribution as in the case of BRR.

According to this paper, "By regularizing all the features identically, BRR is not well suited when only few features are relevant". On the other hand, "The regularization performed by ARD is very adaptive, as all the weights are regularized differently". It does however point out some downsides to ARDR, including that it is "prone to underfitting when the model contains too many regressors ... and also suffers from convergence issues"