Probability path:
Bernoulli random variable → Central limit theorem → Chi-squared distribution → Maximum likelihood estimation
In one sentence: Add up many independent random variables, standardize the sum, and the result looks like a Gaussian curve, whatever the original distribution was.
Keywords: central limit theorem, normal distribution, sum of random variables, sample mean, law of large numbers, Gaussian noise, Python simulation, Matlab code.
What is the central limit theorem?
The central limit theorem (CLT) is one of the most useful results in probability. It says that the sum of many independent random variables, after standardization, becomes approximately normal, even when the individual variables are not normal.
Let $X_1, \ldots, X_N$ be independent and identically distributed (i.i.d.) with mean $\mu$ and finite variance $\sigma^2$. The sum $S_N = X_1 + \cdots + X_N$ has mean $N\mu$ and variance $N\sigma^2$. Both grow with $N$, so the sum itself never settles to a fixed curve. Standardizing it does:
$$Z_N = \frac{S_N – N\mu}{\sigma\sqrt{N}} = \frac{\bar{X}_N – \mu}{\sigma/\sqrt{N}}$$
The CLT states that $Z_N$ converges in distribution to the standard normal, $\mathcal{N}(0,1)$, as $N \to \infty$.
The individual variables can have almost any shape: uniform, binomial, Poisson, exponential, and so on. The only conditions used here are a finite mean and a finite variance.
Why it matters
The CLT lets you make inferences about a population from a sample, without knowing the population’s distribution. It underlies confidence intervals, hypothesis tests, and the common rule that a sample mean is approximately normal once the sample is large enough.
Where it shows up in communications
- Thermal noise. Thermal noise in a resistor comes from a huge number of small, independent contributions from moving charge carriers. The CLT makes the observed voltage approximately Gaussian, which is why the additive white Gaussian noise (AWGN) model is the default in link simulations. See Johnson–Nyquist noise.
- Shot noise. Shot noise is a different mechanism. Charge arrives in discrete events, so the count is Poisson. A Poisson count with a large mean is approximately Gaussian, again by the CLT, but the physical cause is different from thermal noise. The Poisson distribution article gives the details.
- Cascaded filters. The impulse response of a cascade of low-pass filters approaches a Gaussian shape as the number of stages grows. Engelberg [1] shows this with the CLT.
- Error rates. Gaussian tails of noise give the Q-function that appears in BER formulas. See Q-function and erfc.
Law of large numbers and the CLT
The two theorems describe the same average from different angles.
- The law of large numbers (LLN) says the sample mean $\bar{X}_N$ converges to $\mu$. It needs only a finite mean, and it says nothing about the shape of the fluctuations around $\mu$.
- The CLT describes those fluctuations. They are approximately Gaussian, and their size shrinks like $\sigma/\sqrt{N}$.
With a finite variance, the CLT implies the LLN. The LLN also holds for some distributions with infinite variance, where the CLT in the form above does not apply. Figure 3 below shows both effects together.
Demonstration: dice
Rolling one fair die gives a uniform integer from 1 to 6. Its mean is $\mu = 3.5$ and its variance is $\sigma^2 = 35/12 \approx 2.92$. Rolling $N$ dice and standardizing the total gives $Z_N$.
The histograms below use 100,000 trials for each $N$. The red curve is the $\mathcal{N}(0,1)$ density.

How close is “close”? Figure 2 measures the largest gap between the empirical CDF of $Z_N$ and the normal CDF $\Phi$. Both dice and coins converge at roughly the $1/\sqrt{N}$ rate. Coins start farther away, because a single coin toss has only two outcomes and is far from Gaussian, so the gap is larger at every $N$.


Figure 3 shows the LLN and the CLT for the running average of dice rolls. Five independent runs converge to $\mu = 3.5$, and the 95% band from the CLT, $\mu \pm 1.96\,\sigma/\sqrt{N}$, narrows as $N$ grows.
Simulate it in Python
The script below draws random dice, standardizes the sums, and overlays the normal density. It runs as written with NumPy, SciPy and Matplotlib.
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import norm
rng = np.random.default_rng(0) # fixed seed so the figure is reproducible
faces = 6 # 6 = dice, 2 = coins
num_terms = [1, 2, 5, 10, 50, 100]
n_samples = 100_000
mu = (faces + 1) / 2 # mean of one draw
var = (faces**2 - 1) / 12 # variance of one draw
fig, axes = plt.subplots(2, 3, figsize=(9, 5.6), constrained_layout=True)
grid = np.linspace(-4, 4, 400)
for ax, n in zip(axes.ravel(), num_terms):
draws = rng.integers(1, faces + 1, size=(n_samples, n)) # n draws per trial
s = draws.sum(axis=1) # sum of n draws
z = (s - n * mu) / np.sqrt(n * var) # standardize
ax.hist(z, bins=60, range=(-4, 4), density=True, color="#4C72B0", alpha=0.85)
ax.plot(grid, norm.pdf(grid), color="#C44E52", lw=1.6) # N(0,1) density
ax.set_xlim(-4, 4)
ax.set_title(f"N = {n} dice")
ax.set_xlabel("standardized sum $Z_N$")
fig.savefig("clt_dice.png", dpi=150)
print("mean of Z_100:", round(float(z.mean()), 3), " std of Z_100:", round(float(z.std()), 3))Expected last line of output: mean of Z_100: -0.002 std of Z_100: 0.999 (the random draws change the last digits).
Change faces = 6 to faces = 2 to repeat the experiment with coins.
Simulate it in Matlab
The same experiment in base MATLAB (randi and histogram, no toolboxes needed):
% Central limit theorem: standardized sums of fair dice
rng(0); % fixed seed so the figure is reproducible
k = 6; % 6 = dice, 2 = coins
Ns = [1 2 5 10 50 100];
nSamples = 1e5;
mu = (k + 1) / 2; % mean of one draw
v = (k^2 - 1) / 12; % variance of one draw
edges = -4:0.1:4;
grid = linspace(-4, 4, 400);
figure
for i = 1:numel(Ns)
n = Ns(i);
draws = randi(k, nSamples, n); % n draws per trial
z = (sum(draws, 2) - n * mu) / sqrt(n * v); % standardize each trial
subplot(2, 3, i)
histogram(z, edges, 'Normalization', 'pdf', 'FaceColor', [0.3 0.45 0.69])
hold on
plot(grid, exp(-grid.^2 / 2) / sqrt(2 * pi), 'r', 'LineWidth', 1.5) % N(0,1)
hold off
xlim([-4 4])
title(sprintf('N = %d dice', n))
endKey takeaways
- Standardized sums of i.i.d. variables with finite variance approach $\mathcal{N}(0,1)$, whatever the shape of each variable.
- The sample mean fluctuates around $\mu$ by about $\sigma/\sqrt{N}$. This is the CLT view. The LLN says those fluctuations vanish.
- For a die, about 10 terms already look normal. Variables with very few outcomes, such as a single coin, need more terms (Figure 2).
FAQ
Do the variables have to be normal? No. The CLT needs i.i.d. variables with a finite mean and finite variance. The variables can be discrete, skewed or bounded.
How large must N be? There is no single number. It depends on how skewed the variables are. The “N = 30” rule of thumb is a convenient guide, not a guarantee. Figure 2 shows the error shrinking as $1/\sqrt{N}$, and it shrinks more slowly for skewed variables.
Similar topics
- Bernoulli random variable
- Chi-squared distribution PDF
- Maximum likelihood estimation
- Fading channels: Rayleigh fading
References
[1] S. Engelberg, “The central limit theorem and low-pass filters,” Proceedings of the 2004 11th IEEE International Conference on Electronics, Circuits and Systems (ICECS 2004), Tel Aviv, Israel, 2004, pp. 65–68. DOI
[2] Central limit theorem, Wikipedia.
[3] Law of large numbers, Wikipedia.
Next in the probability path:
Chi-squared distribution PDF → Maximum likelihood estimation