Norm and Li-norm

| Tags Machine Learning 

In mathematics, a norm is a function that assigns a strictly positive length or size to each vector in a vector space, other than the zero vector. A seminorm, on the other hand, is allowed to assign zero length to some non-zero vectors(in addition to the zero vector). Norm may come in many forms and many names, including these popular names: Euclidean distance, Mean-squared Error, etc. Usually, norm appears in this form:

$$| x |$$

where $$$x$$$ is a vector or matrix, for example, a Euclidean norm of a vector $$$a = [1, 2, 3]^T$$$ is:

$$ | a |_2 = \sqrt{1^2 + 2^2 + 3^2} $$


$$| x |_0 = \sum x_i^0$$

That is the number of non-zero elements.


$$| x |_1 = \sum |x_i| $$

That is the sum of each element's absolute value.


$$| x |_2 = \sqrt{\sum x_i^2}$$

It is also called Euclidean norm.


For $$$p \geq 1$$$,

$$| x |_p = (\sum |x_i|^p)^{\frac{1}{p}}$$

For infinity, the definition looks very tricky,

$$|x|_{\infty} = \sqrt[\infty]{\sum |x_i|^{\infty}} $$

Fortunately, this equation can be transformed into a very simple format, let's suppose $$$m$$$ be the largest number of $$$x$$$, by the property of infinity we have $$$m^{\infty} \gg x_i^{\infty}$$$, when $$$x_i \neq m$$$.So, $$$| x |_{\infty} = \sqrt[\infty]{k|m|^{\infty}}$$$, where k is the number of largest numbers in $$$x$$$, still thanks to the property of infinity, we have $$$\sqrt[\infty]{k} = 1 $$$, at last, the Linfinity-norm is:

$$|x|_{\infty} = max(|x_i|)$$

To Be Continued

Previous     Next