<aside> 📌 By Dr. Nir Regev
</aside>
<aside> 📌 Sign up to Circuit of Knowledge blog for unlimited tutorials and content
</aside>
<aside> 📌 If it’s knowledge you’re after, join our growing Slack community!
</aside>
July 5th 2024
Jensen's inequality is a fundamental result in probability theory and statistics that relates the value of a convex function of an expectation to the expectation of the convex function. It states that for a convex function $f$ and a random variable $X$, the following inequality holds:
$$ ⁍ $$
In other words, the value of the convex function evaluated at the expected value of $X$ is less than or equal to the expected value of the convex function evaluated at $X$.
Let $f$ be a convex function and $X$ be a random variable with probability density function $p(x)$. The expectation of $X$ is given by:
$$ ⁍ $$
Jensen's inequality states that:
$$ ⁍ $$
which can be written as:
$$ ⁍ $$
The inequality holds for any convex function $f$. A function $f$ is said to be convex if for any two points $x_1$ and $x_2$ in its domain and any $\lambda \in [0, 1]$, the following inequality holds:
$$ ⁍ $$
Intuitively, this means that the line segment connecting any two points on the graph of a convex function lies above or on the graph.
Let's demonstrate Jensen's inequality using Python code. We'll use the exponential function, which is a convex function, as an example.