“The exponential distribution teaches us that sometimes, forgetting the past is a mathematical necessity”, Nir Regev
In the realm of probability and statistics, the exponential distribution emerges as a fascinating subject due to its unique memoryless property. This intriguing characteristic distinguishes it from nearly all other distributions, making it a pivotal model in various scientific and engineering applications. In this blog post, we delve into the essence of the exponential distribution, focusing on its memoryless property, and demystify it through mathematical exposition and visual aids.
The exponential distribution is widely used to model the time between events in a Poisson process, where events occur continuously and independently at a constant average rate. It is defined by the probability density function (PDF):
$$ ⁍ $$
Here, $\lambda$ is the rate parameter of the distribution, and $e$ is the base of the natural logarithm.
The standout feature of the exponential distribution is its memoryless property, formally stated as:
$$ ⁍ $$
This means that the probability the process takes more than $s + t$ units of time, given that it has already taken more than $s$ units, is the same as the probability of it taking more than $t$ units from the start. The process does not "remember" how long it has already lasted.
Let's break down this property with a visual demonstration.
To truly grasp the memoryless nature of the exponential distribution, we'll use Python to generate a graph illustrating this property. (Please run this script)
import numpy as np
import matplotlib.pyplot as plt
# Define the rate parameter
lambda_param = 1
s = 2 # Time already waited
# Define the range for t
t_range = np.linspace(0, 10, 400) # Extended range for clarity
pdf = lambda_param*np.exp(-lambda_param * t_range)
shifted_pdf = lambda_param*np.exp(-lambda_param * (t_range - s))
# Ensuring we only plot valid values for the shifted pdf
valid_indices = t_range >= s
t_range_shifted = t_range[valid_indices]
shifted_pdf_valid = shifted_pdf[valid_indices]
# Plotting
plt.figure(figsize=(12, 8))
# Plot the original pdf
plt.plot(t_range, pdf, label='PDF $f_T(t) = \\lambda e^{-\\lambda t}$', color='blue')
# Highlight the shifted pdf to demonstrate the memoryless property
plt.plot(t_range_shifted, shifted_pdf_valid, label=f'Shifted PDF from $t = {s}$', linestyle='--', color='red')
# Adding a vertical line at s to indicate the starting point of the shifted "pdf"
plt.axvline(t=s, color='green', linestyle=':', label=f'Time Shift $s = {s}$')
# Annotations and labels
plt.title('Visualizing the Memoryless Property of the Exponential Distribution')
plt.xlabel('Time ($t$)')
plt.ylabel('PDF (Probability Density Function)')
plt.legend()
plt.grid(True)
plt.show()
This plot showcases the exponential distribution's probability density function. The area under the curve represents the probability of an event occurring within a specific time frame. We see graphically what it means to condition on $T>2$.
Now, let's dive deeper into the memoryless property with a Monte-Carlo simulation: