TY - GEN

T1 - Near-optimal parameter selection methods for l2 regularization

AU - Ballal, Tarig

AU - Suliman, Mohamed Abdalla Elhag

AU - Al-Naffouri, Tareq Y.

N1 - KAUST Repository Item: Exported on 2020-04-23

PY - 2018/3/12

Y1 - 2018/3/12

N2 - This paper focuses on the problem of selecting the regularization parameter for linear least-squares estimation. Usually, the problem is formulated as a minimization problem with a cost function consisting of the square sum of the l norm of the residual error, plus a penalty term of the squared norm of the solution multiplied by a constant. The penalty term has the effect of shrinking the solution towards the origin with magnitude that depends on the value of the penalty constant. By considering both squared and non-squared norms of the residual error and the solution, four different cost functions can be formed to achieve the same goal. In this paper, we show that all the four cost functions lead to the same closed-form solution involving a regularization parameter, which is related to the penalty constant through a different constraint equation for each cost function. We show that for three of the cost functions, a specific procedure can be applied to combine the constraint equation with the mean squared error (MSE) criterion to develop approximately optimal regularization parameter selection algorithms. Performance of the developed algorithms is compared to existing methods to show that the proposed algorithms stay closest to the optimal MSE.

AB - This paper focuses on the problem of selecting the regularization parameter for linear least-squares estimation. Usually, the problem is formulated as a minimization problem with a cost function consisting of the square sum of the l norm of the residual error, plus a penalty term of the squared norm of the solution multiplied by a constant. The penalty term has the effect of shrinking the solution towards the origin with magnitude that depends on the value of the penalty constant. By considering both squared and non-squared norms of the residual error and the solution, four different cost functions can be formed to achieve the same goal. In this paper, we show that all the four cost functions lead to the same closed-form solution involving a regularization parameter, which is related to the penalty constant through a different constraint equation for each cost function. We show that for three of the cost functions, a specific procedure can be applied to combine the constraint equation with the mean squared error (MSE) criterion to develop approximately optimal regularization parameter selection algorithms. Performance of the developed algorithms is compared to existing methods to show that the proposed algorithms stay closest to the optimal MSE.

UR - http://hdl.handle.net/10754/630488

UR - https://ieeexplore.ieee.org/document/8309170/

UR - http://www.scopus.com/inward/record.url?scp=85048132497&partnerID=8YFLogxK

U2 - 10.1109/GlobalSIP.2017.8309170

DO - 10.1109/GlobalSIP.2017.8309170

M3 - Conference contribution

SN - 9781509059904

SP - 1295

EP - 1299

BT - 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP)

PB - Institute of Electrical and Electronics Engineers (IEEE)

ER -