Let us slightly tweak the Least squares solution WReg = (XTX + γI)−1XTY . This is called Regularized least squares. On the face, it looks like we are simply adding noise to the diagonal. Will such a diagonal loading make it invertible?
a) yes
b) no