Chi square distribution – demystified

PoorBelow averageAverageGoodExcellent (8 votes, average: 4.50 out of 5)

A random variable is always associated with a probability distribution. When the random variable undergoes mathematical transformation the underlying probability distribution no longer remains the same. Consider a random variable Z whose probability distribution function (PDF) is a standard normal distribution (\mu=0 and \sigma^2=1). Now, if the random variable is squared (a mathematical transformation), then the PDF of Z^2 is no longer a standard normal distribution. The new transformed distribution is called Chi square Distribution with 1 degree of freedom. The PDF of Z and Z^2 are plotted in Figure 1.

Transformation of Normal Distribution to Chi Square Distribution
Figure 1: Transformation of Normal Distribution to Chi Square Distribution

The mean of the random variable Z is E(Z) = 0 and for the transformed variable Z2, the mean is given by E(Z^2)=1. Similarly, the variance of the random variable Z is \sigma^2_Z=1, whereas the variance of the transformed random variable Z^2 is \sigma^2_{Z^2}=2. In addition to the mean and variance, the shape of the distribution is also changed. The distribution of the transformed variable Z^2 is no longer symmetric. In fact, the distribution is skewed to one side. Also the random variable Z^2 can take only positive values whereas the random variable Z can take negative values too (note the x-axis in the plots above).

Since the new transformation is based on only one parameter (Z), the degree of freedom for this transformation is 1. Therefore, the transformed random variable Z^2 follows – “Chi-square distribution with 1 degree of freedom”.
Suppose, if Z_1,Z_2,\cdots,Z_k are independent random variables that follows standard normal distribution(\mu=0 and \sigma^2=1), then the transformation,

\chi_k^2 = Z_1^2 + Z_2^2+ \cdots+Z_k^2

is a Chi square distribution with k degrees of freedom. The following figure illustrates how the definition of the Chi square distribution as a transformation of normal distribution for 1 degree of freedom and 2 degrees of freedom. In the same manner, the transformation can be extended to k degrees of freedom.

Constructing chi square random variable with two degrees of freedom
Figure 2: Illustration of Chi-square Distribution with 2 degrees of freedom

The above equation is derived from k random variables that follow standard normal distribution. For a standard normal distribution, the mean \mu=0. Therefore, the transformation \chi_k^2 is called central Chi-square distribution. If, the underlying k random variables follow normal distribution with non-zero mean, then the transformation \chi_k^2 is called non-central Chi-square distribution [2] . In channel modeling, the central Chi-squared distribution is related to Rayleigh Fading scenario and the non-central Chi-square distribution is related to Rician Fading scenario.

Mathematically, the PDF of the central Chi-squared distribution with k degrees of freedom is given by

f_{\chi_k^2 }(x)= \frac{1}{2^{\frac{k}{2}}\Gamma \left(\frac{k}{2}\right )}x^{\frac{k}{2}-1}e^{-\frac{x}{2}}

The mean and variance of the central Chi-squared distributed random variable is given by

\mu = E\left[\chi_k^2\right] = k

\sigma^2 = var\left[\chi_k^2\right] = 2k

Relation to Rayleigh distribution

The connection between Chi square distribution and the Rayleigh distribution can be established as follows

  1. If a random variable R has standard Rayleigh distribution, then the transformation R^2 follows chi-square distribution with 2 degrees of freedom.
  2. If a random variable C has the chi-square distribution with 2 degrees of freedom, then the transformation \sqrt{C} has standard Rayleigh distribution.

Applications:

Chi-square distribution is used in hypothesis testing (to compare the observed data with expected data that follows a specific hypothesis) and in estimating variances of a parameter.

Matlab Simulation:

Check this book for full Matlab code.
Wireless Communication Systems using Matlab – by Mathuranathan Viswanathan

central Chi square Distribution with k degrees of freedom
Figure 3: Simulated output – central Chi square Distribution with k degrees of freedom

Python Code

Python numpy package has a chisquare() generator, which can be used in a straightforward manner to obtain the Chi square distributed sequences.

#---------Chi square distribution gaussianwaves.com-----
import numpy as np
import matplotlib.pyplot as plt
#%matplotlib inline
plt.style.use('ggplot')

ks=np.arange(start=1,stop=6,step=1) #degrees of freedoms to simulate
nSamp=1000000 #number of samples to generate

fig, ax = plt.subplots(ncols=1, nrows=1, constrained_layout=True)

for i,k in enumerate(ks):
    #Generate central Chi-square distributed random numbers
    X = np.random.chisquare(df=k, size = nSamp)
    ax.hist(X,bins=500,density=True,label=r'$k$={}'.format(k), \
    histtype='step',alpha=0.75, linewidth=3)

ax.set_xlim(left=0,right=8);ax.set_ylim(bottom=0,top=0.5);ax.legend();
ax.set_title('PDFs of Chi square distribution');
ax.set_xlabel(r'$\chi_k^2$');ax.set_ylabel(r'$f_{\chi_k^2}(x)$');
plt.show()

Rate this article: PoorBelow averageAverageGoodExcellent (8 votes, average: 4.50 out of 5)

For further reading

[1] Ernie Croot, “Notes on Chi-squared distribution”, Georgia institute of technology, School of mathematics, Oct 2005.↗

Similar topics

Random Variables - Simulating Probabilistic Systems
● Introduction
Plotting the estimated PDF
● Univariate random variables
 □ Uniform random variable
 □ Bernoulli random variable
 □ Binomial random variable
 □ Exponential random variable
 □ Poisson process
 □ Gaussian random variable
 □ Chi-squared random variable
 □ Non-central Chi-Squared random variable
 □ Chi distributed random variable
 □ Rayleigh random variable
 □ Ricean random variable
 □ Nakagami-m distributed random variable
Central limit theorem - a demonstration
● Generating correlated random variables
 □ Generating two sequences of correlated random variables
 □ Generating multiple sequences of correlated random variables using Cholesky decomposition
Generating correlated Gaussian sequences
 □ Spectral factorization method
 □ Auto-Regressive (AR) model

Books by the author

Wireless Communication Systems in Matlab
Wireless Communication Systems in Matlab
Second Edition(PDF)

PoorBelow averageAverageGoodExcellent (162 votes, average: 3.78 out of 5)

Digital modulations using Python
Digital Modulations using Python
(PDF ebook)

PoorBelow averageAverageGoodExcellent (123 votes, average: 3.60 out of 5)

digital_modulations_using_matlab_book_cover
Digital Modulations using Matlab
(PDF ebook)

PoorBelow averageAverageGoodExcellent (126 votes, average: 3.70 out of 5)

Hand-picked Best books on Communication Engineering
Best books on Signal Processing

3 thoughts on “Chi square distribution – demystified”

  1. I used your description here to study experimental data generated by a position detector where the middle of a hole would be the coordinate 0 for both X and Y. I then calculated the radial offset from the measured X and Y positions. From your text it seemed like I should use k=2 for the Chi Squared distribution and that did not fit the data. Then by trial and error I found that using k=3 worked perfectly well. However, I would like to know why this worked. How did I misunderstand what you wrote? The radius is sqrt(X^2 + Y^2) , but R^2 should correspond to your first example.

    I got a probability distribution as follows:

    𝑃(𝑟)=(𝑟/𝜎^2 ) 𝑒^(−𝑟^2/(2𝜎^2 ))

    where 𝜎 is the variance.

    Reply
    • The connection between Chi-squared distribution and the Rayleigh distribution can be established as follows

      If a random variable R has standard Rayleigh distribution, then the transformation R^2 follows chi-square distribution with 2 degrees of freedom.

      If a random variable C has the chi-square distribution with 2 degrees of freedom, then the transformation √C has standard Rayleigh distribution.

      Reply

Post your valuable comments !!!