Near-optimal parameter selection methods for l2 regularization

Tarig Ballal, Mohamed Abdalla Elhag Suliman, Tareq Y. Al-Naffouri

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper focuses on the problem of selecting the regularization parameter for linear least-squares estimation. Usually, the problem is formulated as a minimization problem with a cost function consisting of the square sum of the l norm of the residual error, plus a penalty term of the squared norm of the solution multiplied by a constant. The penalty term has the effect of shrinking the solution towards the origin with magnitude that depends on the value of the penalty constant. By considering both squared and non-squared norms of the residual error and the solution, four different cost functions can be formed to achieve the same goal. In this paper, we show that all the four cost functions lead to the same closed-form solution involving a regularization parameter, which is related to the penalty constant through a different constraint equation for each cost function. We show that for three of the cost functions, a specific procedure can be applied to combine the constraint equation with the mean squared error (MSE) criterion to develop approximately optimal regularization parameter selection algorithms. Performance of the developed algorithms is compared to existing methods to show that the proposed algorithms stay closest to the optimal MSE.
Original languageEnglish (US)
Title of host publication2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP)
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages1295-1299
Number of pages5
ISBN (Print)9781509059904
DOIs
StatePublished - Mar 12 2018

Fingerprint Dive into the research topics of 'Near-optimal parameter selection methods for l2 regularization'. Together they form a unique fingerprint.

Cite this