Before going through the contents in this page , readers are advised to grasp fundamental concepts like random variable, PDF , CDF and types of probability distributions

## Random Variable:

A random variable is a mapping from sample space \( \Omega\) to a set of real numbers. What does this mean ? Lets take the usual evergreen example of “flipping a coin”.

In a “coin-flipping” experiment, the outcome is not known prior to the experiment, that is we cannot predict it with certainty (non-deterministic/stochastic). But we know the all possible outcomes – Head or Tail. Assign real numbers to the all possible events (this is called “sample space”), say “0” to “Head” and “1” to “Tail”, and associate a variable “**X**” that could take these two values. This variable “**X**” is called a random variable, since it can randomly take any value ‘0’ or ‘1’ before performing the actual experiment.

Obviously, we do not want to wait till the coin-flipping experiment is done. Because the outcome will lose its significance, we want to associate some probability to each of the possible event. In the coin-flipping experiment, all outcomes are equally probable (given that the coin is fair and unbiased). This means that we can say that the probability of getting Head ( our random variable** X** = 0 ) as well that of getting Tail ( **X** =1 ) is 0.5 (i.e. 50-50 chance for getting Head/Tail).

This can be written as,

$$ P(\mathbf{X}=0)=0.5\;and\;P(\mathbf{X}=1)=0.5$$

## Cumulative Distribution Function:

Mathematically, a complete description of a random variable is given be “Cumulative Distribution Function“- F_{X}(x). Here the bold faced “**X”** is a random variable and “x” is a dummy variable which is a place holder for all possible outcomes ( “0” and “1” in the above mentioned coin flipping experiment). The Cumulative Distribution Function is defined as,

$$ F_{\textbf{X}}(x)= P(\textbf{X}\leq x)$$

If we plot the CDF for our coin-flipping experiment, it would look like the one shown in the figure on your right.

The example provided above is of discrete nature, as the values taken by the random variable are discrete (either “0” or “1”) and therefore the random variable is called Discrete Random Variable.

If the values taken by the random variables are of continuous nature (Example: Measurement of temperature), then the random variable is called Continuous Random Variable and the corresponding cumulative distribution function will be smoother without discontinuities.

## Probability Distribution function :

Consider an experiment in which the probability of events are as follows. The probabilities of getting the numbers 1,2,3,4 individually are 1/10,2/10,3/10,4/10 respectively. It will be more convenient for us if we have an equation for this experiment which will give these values based on the events. For example, the equation for this experiment can be given by f(x)=x/10 where x=1,2,3,4. This equation ( equivalently a function) is called probability distribution function.

## Probability Density function (PDF) and Probability Mass Function(PMF):

Its more common deal with Probability Density Function (PDF)/Probability Mass Function (PMF) than CDF.

The PDF (defined for Continuous Random Variables) is given by taking the first derivate of CDF.

$$ f_\textbf{X}(x)=\frac{dF_\textbf{X}(x)}{dx}$$

For discrete random variable that takes on discrete values, is it common to defined Probability Mass Function.

$$ f_\textbf{X}(x)=P(\textbf{X}=x)$$

The previous example was simple. The problem becomes slightly complex if we are asked to find the probability of getting a value less than or equal to 3. Now the straight forward approach will be to add the probabilities of getting the values x=1,2,3 which comes out to be 1/10+2/10+3/10 =6/10. This can be easily modeled as a probability density function which will be the integral of probability distribution function with limits 1 to 3.

Based on the probability density function or how the PDF graph looks, PDF fall into different categories like binomial distribution, Uniform distribution, Gaussian distribution, Chi-square distribution, Rayleigh distribution, Rician distribution etc. Out of these distributions, you will encounter Gaussian distribution or Gaussian Random variable in digital communication very often.

## Mean:

The mean of a random variable is defined as the weighted average of all possible values the random variable can take. Probability of each outcome is used to weight each value when calculating the mean. Mean is also called expectation (E[X])

For continuos random variable X and probability density function f_{X}(x)

$$ E\left[X \right] = \int_{-\infty }^{\infty}xf_X(x)dx $$

For discrete random variable X, the mean is calculated as weighted average of all possible values (x_{i}) weighted with individual probability (p_{i})

$$ E\left[X \right] = \mu{_X} = \sum_{-\infty }^{\infty}x_{i}p_{i} $$

## Variance :

Variance measures the spread of a distribution. For a continuous random variable X, the variance is defined as

$$ var \left[X\right] = \int_{-\infty }^{\infty} \left(x – E\left[X \right] \right)^2 f_X(x) dx $$

For discrete case, the variance is defined as

$$ var \left[X\right] = {\sigma^2}_X = \sum_{-\infty }^{\infty} \left( x_i – \mu_X\right)^2 p_{i} $$

Standard Deviation (\( \sigma \)) is defined as the square root of variance \( {\sigma^2}_X \)

## Properties of Mean and Variance:

For a constant – “c” following properties will hold true for mean

$$ E\left[cX\right] = c E\left[X\right]$$

$$ E\left[X+c\right] = E\left[X\right]+c$$

$$ E\left[c\right] = c$$

For a constant – “c” following properties will hold true for variance

$$ var\left[cX\right] = c^2 var\left[X\right]$$

$$ var\left[X+c\right] = var\left[X\right]$$

$$ var\left[c\right] = 0$$

**PDF and CDF define a random variable completely. For example: If two random variables X and Y have the same PDF, then they will have the same CDF and therefore their mean and variance will be same.
On the otherhand, mean and variance describes a random variable only partially. If two random variables X and Y have the same mean and variance, they may or may not have the same PDF or CDF.**

## Gaussian Distribution :

Gaussian PDF looks like a bell. It is used most widely in communication engineering. For example , all channels are assumed to be Additive White Gaussian Noise channel. What is the reason behind it ? Gaussian noise gives the smallest channel capacity with fixed noise power. This means that it results in the worst channel impairment. So the coding designs done under this most adverse environment will give superior and satisfactory performance in real environments. For more information on “Gaussianity” refer [1]

The PDF of the Gaussian Distribution (also called as Normal Distribution) is completely characterized by its mean (\( \mu\)) and variance(\( \sigma\)),

$$ f(x)=\frac{1}{\sqrt{2\pi \sigma ^{2}}}e^{^{\frac{-(x-\mu )^{2}}{2\sigma ^{2}}}}$$

Since PDF is defined as the first derivative of CDF, a reverse engineering tell us that CDF can be obtained by taking an integral of PDF.

Thus to get the CDF of the above given function,

$$ F_{\textbf{X}}(x;\mu,\sigma^{2})=\frac{1}{\sqrt{2\pi}}\int_{-\infty }^{\frac{x-\mu}{\sigma}}e^{\frac{-t^{2}}{2}}dt$$

Equations for PDF and CDF for certain distributions are consolidated below

Probability Distribution | Probability Density Function(PDF) | Cumulative Distribution Function (CDF) |
---|---|---|

Gaussian/Normal Distribution – \( \mathcal{N}(\mu,\sigma^{2})\) | \( f(x)=\frac{1}{\sqrt{2\pi \sigma ^{2}}}e^{^{\frac{-(x-\mu )^{2}}{2\sigma ^{2}}}}\) | \( F_{\textbf{X}}(x;\mu,\sigma^{2})=\frac{1}{\sqrt{2\pi}}\int_{-\infty }^{\frac{x-\mu}{\sigma}}e^{\frac{-t^{2}}{2}}dt\) |

## Reference :

[1] S.Pasupathy, “Glories of Gaussianity”, IEEE Communications magazine, Aug 1989 – 1, pp 38.

## See Also:

[1] Uniform Random Variables and Uniform Distribution

[2] Chi-Squared distribution

[3] Derivation of Error Rate Performance of an optimum BPSK receiver in AWGN channel

[4] Eb/N0 Vs BER for BPSK over Rician Fading Channel

[5] BER Vs Eb/N0 for QPSK modulation over AWGN

[6] BER Vs Eb/N0 for 8-PSK modulation over AWGN

[7] Simulation of M-PSK modulation techniques in AWGN channel

[8] Performance comparison of Digital Modulation techniques

Pingback: Bias of an Estimator | GaussianWaves()

Pingback: Simulation and Analysis of White Noise in Matlab | GaussianWaves()

Pingback: Introduction to Cramer Rao Lower Bound (CRLB) | GaussianWaves()

Pingback: Eb/N0 Vs BER for BPSK over Rayleigh Channel and AWGN Channel | GaussianWaves()

Pingback: Rayleigh Fading Simulation – Young’s model | GaussianWaves()

Pingback: Score, Fisher Information and Estimator Sensitivity | GaussianWaves()

Pingback: Theoretical derivation of MLE for Exponential Distribution: | GaussianWaves()

Pingback: Theoretical derivation of Maximum Likelihood Estimator for Poisson PDF: | GaussianWaves()

Pingback: Likelihood Function and Maximum Likelihood Estimation (MLE) | GaussianWaves()

Pingback: Minimum Variance Unbiased Estimators (MVUE) | GaussianWaves()

Pingback: Non-central Chi-squared Distribution | GaussianWaves()

Pingback: Simulation of Rayleigh Fading ( Clarke’s Model – sum of sinusoids method) | GaussianWaves()

Pingback: Chi-Squared Distribution | GaussianWaves()

Pingback: Uniform Random Variables and Uniform Distribution | GaussianWaves()

Pingback: Procedure for plotting any PDF in Matlab | GaussianWaves()

Pingback: Maximum Likelihood Decoding | GaussianWaves()

Pingback: Central Limit Theorem | GaussianWaves()

Pingback: Eb/N0 Vs BER for BPSK over Rician Fading Channel | GaussianWaves()

Pingback: Derivation of expression for a Gaussian Filter with 3 dB bandwidth | GaussianWaves()

Pingback: An Introduction to Estimation Theory | GaussianWaves()

Pingback: Intuitive derivation of Performance of an optimum BPSK receiver in AWGN channel | GaussianWaves()

Pingback: Q function and Error functions | GaussianWaves()