# Proving orthogonality properties of residuals for \$L^p\$ optimization

by aghostinthefigures   Last Updated June 23, 2018 02:20 AM

Consider an \$L^2\$ (also called least-squares) optimization problem over a vector space \$V\$, in which one is trying to minimize the \$L^2\$-norm of the difference between some "target" vector \$y\$ and vectors in some subspace \$W\$ of \$V\$. (This "difference vector" is often called the residual vector.)

The Hilbert projection theorem guarantees that there is a unique vector \$x\$ in \$W\$ that minimizes the norm of this residual vector, and consequently also guarantees that the residual vector \$x - y\$ is orthogonal to the subspace \$W\$.

If I consider the same problem, but with the key difference that I now want to minimize a \$L^{p\neq2}\$ norm of the residual, I can provably lose uniqueness of the optimizing vector \$x\$ (certainly the case in \$L^{1}\$).

However, I don't know if you also lose orthogonality of the residual vector w.r.t. the subspace \$W\$, and for which \$L^p\$ spaces this occurs. Is there a way to show that the residual for such an optimization problem is/isn't orthogonal to the subspace \$W\$ for the cases where \$L^{p\neq2}\$?

Tags :