Chi square distribution – demystified

[ratings]

A random variable is always associated with a probability distribution. When the random variable undergoes mathematical transformation the underlying probability distribution no longer remains the same. Consider a random variable $latex Z$ whose probability distribution function (PDF) is a standard normal distribution ($latex \mu=0$ and $latex \sigma^2=1$). Now, if the random variable is squared (a mathematical transformation), then the PDF of $latex Z^2$ is no longer a standard normal distribution. The new transformed distribution is called Chi square Distribution with $latex 1$ degree of freedom. The PDF of $latex Z$ and $latex Z^2$ are plotted in Figure 1.

The mean of the random variable $latex Z$ is $latex E(Z) = 0$ and for the transformed variable Z², the mean is given by $latex E(Z^2)=1$. Similarly, the variance of the random variable $latex Z$ is $latex \sigma^2_Z=1$, whereas the variance of the transformed random variable $latex Z^2$ is $latex \sigma^2_{Z^2}=2$. In addition to the mean and variance, the shape of the distribution is also changed. The distribution of the transformed variable $latex Z^2$ is no longer symmetric. In fact, the distribution is skewed to one side. Also the random variable $latex Z^2$ can take only positive values whereas the random variable $latex Z$ can take negative values too (note the x-axis in the plots above).

Since the new transformation is based on only one parameter ($latex Z$), the degree of freedom for this transformation is $latex 1$. Therefore, the transformed random variable $latex Z^2$ follows – “Chi-square distribution with $latex 1$ degree of freedom”.
Suppose, if $latex Z_1,Z_2,\cdots,Z_k$ are independent random variables that follows standard normal distribution($latex \mu=0$ and $latex \sigma^2=1$), then the transformation,

$latex \chi_k^2 = Z_1^2 + Z_2^2+ \cdots+Z_k^2 &s=2$

is a Chi square distribution with k degrees of freedom. The following figure illustrates how the definition of the Chi square distribution as a transformation of normal distribution for $latex 1$ degree of freedom and $latex 2$ degrees of freedom. In the same manner, the transformation can be extended to $latex k$ degrees of freedom.

The above equation is derived from $latex k$ random variables that follow standard normal distribution. For a standard normal distribution, the mean $latex \mu=0$. Therefore, the transformation $latex \chi_k^2$ is called central Chi-square distribution. If, the underlying $latex k$ random variables follow normal distribution with non-zero mean, then the transformation $latex \chi_k^2$ is called non-central Chi-square distribution [2] . In channel modeling, the central Chi-squared distribution is related to Rayleigh Fading scenario and the non-central Chi-square distribution is related to Rician Fading scenario.

Mathematically, the PDF of the central Chi-squared distribution with $latex k$ degrees of freedom is given by

$latex f_{\chi_k^2 }(x)= \frac{1}{2^{\frac{k}{2}}\Gamma \left(\frac{k}{2}\right )}x^{\frac{k}{2}-1}e^{-\frac{x}{2}} &s=2$

The mean and variance of the central Chi-squared distributed random variable is given by

$latex \mu = E\left[\chi_k^2\right] = k &s=2$

$latex \sigma^2 = var\left[\chi_k^2\right] = 2k &s=2$

Relation to Rayleigh distribution

The connection between Chi square distribution and the Rayleigh distribution can be established as follows

1. If a random variable $latex R$ has standard Rayleigh distribution, then the transformation $latex R^2$ follows chi-square distribution with $latex 2$ degrees of freedom.
2. If a random variable $latex C$ has the chi-square distribution with $latex 2$ degrees of freedom, then the transformation $latex \sqrt{C}$ has standard Rayleigh distribution.

Applications:

Chi-square distribution is used in hypothesis testing (to compare the observed data with expected data that follows a specific hypothesis) and in estimating variances of a parameter.

Matlab Simulation:

Check this book for full Matlab code.
Wireless Communication Systems using Matlab – by Mathuranathan Viswanathan

Python Code

Python numpy package has a chisquare() generator, which can be used in a straightforward manner to obtain the Chi square distributed sequences.

#---------Chi square distribution gaussianwaves.com-----
import numpy as np
import matplotlib.pyplot as plt
#%matplotlib inline
plt.style.use('ggplot')

ks=np.arange(start=1,stop=6,step=1) #degrees of freedoms to simulate
nSamp=1000000 #number of samples to generate

fig, ax = plt.subplots(ncols=1, nrows=1, constrained_layout=True)

for i,k in enumerate(ks):
    #Generate central Chi-square distributed random numbers
    X = np.random.chisquare(df=k, size = nSamp)
    ax.hist(X,bins=500,density=True,label=r'$k$={}'.format(k), \
    histtype='step',alpha=0.75, linewidth=3)

ax.set_xlim(left=0,right=8);ax.set_ylim(bottom=0,top=0.5);ax.legend();
ax.set_title('PDFs of Chi square distribution');
ax.set_xlabel(r'$\chi_k^2$');ax.set_ylabel(r'$f_{\chi_k^2}(x)$');
plt.show()

Rate this article: [ratings]

For further reading

[1] Ernie Croot, “Notes on Chi-squared distribution”, Georgia institute of technology, School of mathematics, Oct 2005.↗

Similar topics

[table id = 33/]

Books by the author

[table id = 23/]