Random Process Archives - Page 3 of 3

Uniform random variable

Note: There is a rating embedded within this post, please visit this post to rate it.

Uniform random variables are used to model scenarios where the expected outcomes are equi-probable. For example, in a communication system design, the set of all possible source symbols are considered equally probable and therefore modeled as a uniform random variable.

The uniform distribution is the underlying distribution for an uniform random variable. A continuous uniform random variable, denoted as , take continuous values within a given interval , with equal probability. Therefore, the PDF of such a random variable is a constant over the given interval is.

This article is part of the book
● Wireless Communication Systems in Matlab (second edition), ISBN: 979-8648350779 available in ebook (PDF) format and Paperback (hardcopy) format.

$$ f_X(x) = \begin{cases}\frac{1}{b-a} & \text{when } a < x < b\\0 & \text{otherwise} \end{cases} $$

In Matlab, rand function generates continuous uniform random numbers in the interval . The rand function picks a random number in the interval in which the probability of occurrence of all the numbers in the interval are equally likely. The command rand(n,m) will generate a matrix of size . To generate a random number in the interval one can use the following expression.

a + (b-a)*rand(n,m); %Here nxm is the size of the output matrix

To test whether the numbers generated by the continuous uniform distribution are uniform in the interval , one has to generate very large number of values using the rand function and then plot the histogram. The Figure 1 shows that the simulated PDF and theoretical PDF are in agreement with each other.

a=2;b=10; %open interval (2,10)
X=a+(b-a)*rand(1,1000000);%simulate uniform RV
[p,edges]=histcounts(X,'Normalization','pdf');%estimated PDF
outcomes = 0.5*(edges(1:end-1) + edges(2:end));%possible outcomes
g=1/(b-a)*ones(1,length(outcomes)); %Theoretical PDF
bar(outcomes,p);hold on;plot(outcomes,g,'r-');
title('Probability Density Function');legend('simulated','theory');
xlabel('Possible outcomes');ylabel('Probability of outcomes');

Figure 1: Continuous uniform random variable : histogram and theoretical PDF

Figure 1: Continuous uniform random variable : histogram and theoretical PDF

On the other hand, a discrete random variable generates discrete values that are equally probable. The underlying discrete uniform distribution is denoted as , where , is a finite set of discrete elements that are equally probable as described by the probability mass function (PMF)

$$ f_X(x)= \begin{cases}\frac{1}{n} & \text{where } x \in {s_1,s_2,…,s_n } \\ 0 & otherwise \end{cases} $$

There exist several methods to generate discrete uniform random numbers and two of them are discussed here. The straightforward method is to use randi function in Matlab that can generate discrete uniform numbers in the integer set . The second method is to use rand function and ceil the result to discrete values. For example, the command to generate uniformly distributed discrete numbers from the set is

X=ceil(n*rand(1,100));

The uniformity test for discrete uniform random numbers can be performed and it is very similar to the code shown for the continuous uniform random variable case. The only difference here is the normalization term. The histogram values should not be normalized by the total area under the histogram curve as in the case of continuous random variables. Rather, the histogram should be normalized by the total number of occurrences in all the bins. We cannot normalized based on the area under the curve, since the bin values are not dense enough (bins are far from each other) for proper calculation of total area. The code snippet is given next. The resulting plot (Figure 2) shows a good match between the simulated and theoretical PMFs.

X=randi(6,100000,1); %Simulate throws of dice,S={1,2,3,4,5,6}
[pmf,edges]=histcounts(X,'Normalization','pdf');%estimated PMF
outcomes = 0.5*(edges(1:end-1) + edges(2:end));%S={1,2,3,4,5,6}
g=1/6*ones(1,6); %Theoretical PMF
bar(outcomes,pmf);hold on;stem(outcomes,g,'r-');
title('Probability Mass Function');legend('simulated','theory');
xlabel('Possible outcomes');ylabel('Probability of outcomes');

Figure 2: Discrete uniform random variable : histogram and theoretical PMFs

Note: There is a rating embedded within this post, please visit this post to rate it.

Topics in this chapter

Random Variables - Simulating Probabilistic Systems
● Introduction
● Plotting the estimated PDF
● Univariate random variables
□ Uniform random variable
□ Bernoulli random variable
□ Binomial random variable
□ Exponential random variable
□ Poisson process
□ Gaussian random variable
□ Chi-squared random variable
□ Non-central Chi-Squared random variable
□ Chi distributed random variable
□ Rayleigh random variable
□ Ricean random variable
□ Nakagami-m distributed random variable
● Central limit theorem - a demonstration
● Generating correlated random variables
□ Generating two sequences of correlated random variables
□ Generating multiple sequences of correlated random variables using Cholesky decomposition
● Generating correlated Gaussian sequences
□ Spectral factorization method
□ Auto-Regressive (AR) model

Books by the author

Wireless Communication Systems in Matlab Second Edition(PDF) Note: There is a rating embedded within this post, please visit this post to rate it. Checkout Added to cart	Digital Modulations using Python (PDF ebook) Note: There is a rating embedded within this post, please visit this post to rate it. Checkout Added to cart	Digital Modulations using Matlab (PDF ebook) Note: There is a rating embedded within this post, please visit this post to rate it. Checkout Added to cart
Hand-picked Best books on Communication Engineering Best books on Signal Processing

Derive BPSK BER – optimum receiver in AWGN channel

Key focus: Derive BPSK BER (bit error rate) for optimum receiver in AWGN channel. Explained intuitively step by step.

BPSK modulation is the simplest of all the M-PSK techniques. An insight into the derivation of error rate performance of an optimum BPSK receiver is essential as it serves as a stepping stone to understand the derivation for other comparatively complex techniques like QPSK,8-PSK etc..

Understanding the concept of Q function and error function is a pre-requisite for this section of article.

The ideal constellation diagram of a BPSK transmission (Figure 1) contains two constellation points located equidistant from the origin. Each constellation point is located at a distance from the origin, where E_s is the BPSK symbol energy. Since the number of bits in a BPSK symbol is always one, the notations – symbol energy (E_s) and bit energy (E_b) can be used interchangeably (E_s=E_b).

Assume that the BPSK symbols are transmitted through an AWGN channel characterized by variance = N₀/2 Watts. When 0 is transmitted, the received symbol is represented by a Gaussian random variable ‘r‘ with mean=S₀ = and variance =N₀/2. When 1 is transmitted, the received symbol is represented by a Gaussian random variable – r with mean=S₁= and variance =N₀/2. Hence the conditional density function of the BPSK symbol (Figure 2) is given by,

*Figure 2: Probability density function (PDF) for BPSK Symbols*

An optimum receiver for BPSK can be implemented using a correlation receiver or a matched filter receiver (Figure 3). Both these forms of implementations contain a decision making block that decides upon the bit/symbol that was transmitted based on the observed bits/symbols at its input.

When the BPSK symbols are transmitted over an AWGN channel, the symbols appears smeared/distorted in the constellation depending on the SNR condition of the channel. A matched filter or that was previously used to construct the BPSK symbols at the transmitter. This process of projection is illustrated in Figure 4. Since the assumed channel is of Gaussian nature, the continuous density function of the projected bits will follow a Gaussian distribution. This is illustrated in Figure 5.

*Figure 4: Role of correlation/Matched Filter*

After the signal points are projected on the basis function axis, a decision maker/comparator acts on those projected bits and decides on the fate of those bits based on the threshold set. For a BPSK receiver, if the a-prior probabilities of transmitted 0’s and 1’s are equal (P=0.5), then the decision boundary or threshold will pass through the origin. If the apriori probabilities are not equal, then the optimum threshold boundary will shift away from the origin.

*Figure 5: Distribution of received symbols*

Considering a binary symmetric channel, where the apriori probabilities of 0’s and 1’s are equal, the decision threshold can be conveniently set to T=0. The comparator, decides whether the projected symbols are falling in region A or region B (see Figure 4). If the symbols fall in region A, then it will decide that 1 was transmitted. It they fall in region B, the decision will be in favor of ‘0’.

For deriving the performance of the receiver, the decision process made by the comparator is applied to the underlying distribution model (Figure 5). The symbols projected on the axis will follow a Gaussian distribution. The threshold for decision is set to T=0. A received bit is in error, if the transmitted bit is ‘0’ & the decision output is ‘1’ and if the transmitted bit is ‘1’ & the decision output is ‘0’.

This is expressed in terms of probability of error as,

Or equivalently,

By applying Bayes Theorem↗, the above equation is expressed in terms of conditional probabilities as given below,

Since a-prior probabilities are equal P(0_T)= P(1_T) =0.5, the equation can be re-written as

Intuitively, the integrals represent the area of shaded curves as shown in Figure 6. From the previous article, we know that the area of the shaded region is given by Q function.

*Figure 6a, 6b: Calculating Error Probability*

Similarly,

From (4), (6), (7) and (8),

For BPSK, since E_s=E_b, the probability of symbol error (P_s) and the probability of bit error (P_b) are same. Therefore, expressing the P_s and P_b in terms of Q function and also in terms of complementary error function :

Rate this article: Note: There is a rating embedded within this post, please visit this post to rate it.

Reference

[1] Nguyen & Shwedyk, “A First course in Digital Communications”, Cambridge University Press, 1st edition.↗

Books by author

Wireless Communication Systems in Matlab Second Edition(PDF) Note: There is a rating embedded within this post, please visit this post to rate it. Checkout Added to cart	Digital Modulations using Python (PDF ebook) Note: There is a rating embedded within this post, please visit this post to rate it. Checkout Added to cart	Digital Modulations using Matlab (PDF ebook) Note: There is a rating embedded within this post, please visit this post to rate it. Checkout Added to cart
Hand-picked Best books on Communication Engineering Best books on Signal Processing

Q function and Error functions : demystified

In simple words, The Q-function gives the probability that a random variable from a normal distribution will exceed a certain threshold value. The erf function gives the probability that a normally distributed variable will fall within a certain range.

Q function

Q functions are often encountered in the theoretical equations for Bit Error Rate (BER) involving AWGN channel. A brief discussion on Q function and its relation to erfc function is given here.

Gaussian process is the underlying model for an AWGN channel.The probability density function of a Gaussian Distribution is given by

\[p(x) = \displaystyle{ \frac{1}{ \sigma \sqrt{2 \pi}} e^{ – \frac{(x-\mu)^2}{2 \sigma^2}}}\quad\quad (1) \]

Generally, in BER derivations, the probability that a Gaussian Random Variable $X \sim N ( \mu, \sigma^2) $ exceeds $x_0$ is evaluated as the area of the shaded region as shown in Figure 1.

*Figure 1: Gaussian PDF and illustration of Q function*

Mathematically, the area of the shaded region is evaluated as,

\[Pr(X \geq x_0) =\displaystyle{ \int_{x_0}^{\infty} p(x) dx = \int_{x_0}^{\infty} \frac{1}{ \sigma \sqrt{2 \pi}} e^{ – \frac{(x-\mu)^2}{2 \sigma^2}} dx } \quad\quad (2) \]

The above probability density function given inside the above integral cannot be integrated in closed form. So by change of variables method, we substitute

\[\displaystyle{ y = \frac{x-\mu}{\sigma} }\]

Then equation (2) can be re-written as,

\[\displaystyle{ Pr\left( y > \frac{x_0-\mu}{\sigma} \right ) = \int_{ \left( \frac{x_{0} -\mu}{\sigma}\right)}^{\infty} \frac{1}{ \sqrt{2 \pi}} e^{- \frac{y^2}{2}} dy } \quad\quad (3) \]

Here the function inside the integral is a normalized gaussian probability density function $Y \sim N( 0, 1)$, normalized to mean $\mu=0$ and standard deviation $\sigma=1$.

The integral on the right side can be termed as Q-function, which is given by,

\[\displaystyle{Q(z) = \int_{z}^{\infty}\frac{1}{ \sqrt{2 \pi}} e^{- \frac{y^2}{2}} dy } \quad\quad (4)\]

Here the Q function is related as,

\[\displaystyle{ Pr\left( y > \frac{x_0-\mu}{\sigma} \right ) = Q\left(\frac{x_0-\mu}{\sigma} \right ) = Q(z)} \quad\quad (5)\]

Thus Q function gives the area of the shaded curve with the transformation $y = \frac{x-\mu}{\sigma}$ applied to the Gaussian probability density function. Essentially, Q function evaluates the tail probability of normal distribution (area of shaded area in the above figure).

The Q-function gives the probability that a random variable from a normal distribution will exceed a certain threshold value.

Error function

The complementary error function represents the area under the two tails of zero mean Gaussian probability density function of variance $\sigma^2 = 1/2$. The error function gives the probability that the parameter lies outside that range.

Therefore, the complementary error function is given by

\[\displaystyle{ erfc(z) = \frac{2}{\sqrt{\pi}} \int_{z}^{\infty} e^{-x^2}} dx \quad\quad (6)\]

Hence, the error function is

\[erf(z) = 1 – erfc(z) \quad\quad (7)\]

or equivalently,

\[\displaystyle{ erf(z) = \frac{2}{\sqrt{\pi}} \int_{0}^{z} e^{-x^2} dx } \quad\quad (8) \]

The erf function gives the probability that a normally distributed variable will fall within a certain range.

Q function and Complementary Error Function (erfc)

From the limits of the integrals in equation (4) and (6) one can conclude that Q function is directly related to complementary error function (erfc). It follows from equation (4) and (6), Q function is related to complementary error function by the following relation.

\[\displaystyle{ Q(z) = \frac{1}{2} erfc \left( \frac{z}{\sqrt{2}}\right)} \quad\quad (9) \]

Some important results

Keep a note of the following equations that can come handy when deriving probability of bit errors for various scenarios. These equations are compiled here for easy reference.

If we have a normal variable $X \sim N (\mu, \sigma^2)$, the probability that $X > x$ is

\[\displaystyle{ Pr \left( X > x \right) = Q \left( \frac{x-\mu}{\sigma} \right ) } \quad\quad (10) \]

If we want to know the probability that $X$ is away from the mean by an amount ‘a’ (on the left or right side of the mean), then

\[\displaystyle{ Pr \left( X > \mu+a \right) = Pr \left( X < \mu-a \right) = Q\left(\frac{a}{\sigma} \right ) } \quad\quad (11) \]

If we want to know the probability that X is away from the mean by an amount ‘a’ (on both sides of the mean), then

\[\displaystyle{ Pr \left( \mu-a > X > \mu+a \right) = 2 Q\left(\frac{a}{\sigma} \right ) } \quad\quad (12)\]

Application of Q function in computing the Bit Error Rate (BER) or probability of bit error will be the focus of our next article.

Applications

The Q-function and the error function (erf) are important mathematical functions that arise in many fields, including probability theory, statistics, signal processing, and communications engineering. Here are some reasons why these functions are important:

Probability calculations: The Q-function and erf function are used in probability calculations involving Gaussian distributions. The Q-function gives the probability that a random variable from a normal distribution will exceed a certain threshold value. The erf function gives the probability that a normally distributed variable will fall within a certain range.
Signal processing: In signal processing, the Q-function is used to calculate the probability of bit error in digital communication systems. This is important for designing communication systems that can reliably transmit data over noisy channels.
Statistical analysis: The Q-function and erf function are used in statistical analysis to model data and estimate parameters. For example, in hypothesis testing, the Q-function can be used to calculate p-values.
Mathematical modeling: The Q-function and erf function arise naturally in mathematical models for various phenomena. For example, the heat equation in physics and the Black-Scholes equation in finance both involve the erf function.
Computational efficiency: In some cases, the Q-function and erf function provide a more efficient and accurate way of calculating certain probabilities and integrals than other methods.

Books by the author

Wireless Communication Systems in Matlab Second Edition(PDF) Note: There is a rating embedded within this post, please visit this post to rate it. Checkout Added to cart	Digital Modulations using Python (PDF ebook) Note: There is a rating embedded within this post, please visit this post to rate it. Checkout Added to cart	Digital Modulations using Matlab (PDF ebook) Note: There is a rating embedded within this post, please visit this post to rate it. Checkout Added to cart
Hand-picked Best books on Communication Engineering Best books on Signal Processing

Random Variables, CDF and PDF

Random Variable:

In a “coin-flipping” experiment, the outcome is not known prior to the experiment, that is we cannot predict it with certainty (non-deterministic/stochastic). But we know the all possible outcomes – Head or Tail. Assign real numbers to the all possible events (this is called “sample space”), say “0” to “Head” and “1” to “Tail”, and associate a variable “X” that could take these two values. This variable “X” is called a random variable, since it can randomly take any value ‘0’ or ‘1’ before performing the actual experiment.

Obviously, we do not want to wait till the coin-flipping experiment is done. Because the outcome will lose its significance, we want to associate some probability to each of the possible event. In the coin-flipping experiment, all outcomes are equally probable (given that the coin is fair and unbiased). This means that we can say that the probability of getting Head ( our random variable X = 0 ) as well that of getting Tail ( X =1 ) is 0.5 (i.e. 50-50 chance for getting Head/Tail).

This can be written as,

Cumulative Distribution Function:

Mathematically, a complete description of a random variable is given be “Cumulative Distribution Function”- F_X(x). Here the bold faced “X” is a random variable and “x” is a dummy variable which is a place holder for all possible outcomes ( “0” and “1” in the above mentioned coin flipping experiment). The Cumulative Distribution Function is defined as,

If we plot the CDF for our coin-flipping experiment, it would look like the one shown in the figure on your right.
The example provided above is of discrete nature, as the values taken by the random variable are discrete (either “0” or “1”) and therefore the random variable is called Discrete Random Variable.

If the values taken by the random variables are of continuous nature (Example: Measurement of temperature), then the random variable is called Continuous Random Variable and the corresponding cumulative distribution function will be smoother without discontinuities.

Probability Distribution function :

Consider an experiment in which the probability of events are as follows. The probabilities of getting the numbers 1,2,3,4 individually are respectively. It will be more convenient for us if we have an equation for this experiment which will give these values based on the events. For example, the equation for this experiment can be given by where . This equation ( equivalently a function) is called probability distribution function.

Probability Density function (PDF) and Probability Mass Function(PMF):

Its more common deal with Probability Density Function (PDF)/Probability Mass Function (PMF) than CDF.

The PDF (defined for Continuous Random Variables) is given by taking the first derivate of CDF.

For discrete random variable that takes on discrete values, is it common to defined Probability Mass Function.

The previous example was simple. The problem becomes slightly complex if we are asked to find the probability of getting a value less than or equal to 3. Now the straight forward approach will be to add the probabilities of getting the values which comes out to be . This can be easily modeled as a probability density function which will be the integral of probability distribution function with limits 1 to 3.

Based on the probability density function or how the PDF graph looks, PDF fall into different categories like binomial distribution, Uniform distribution, Gaussian distribution, Chi-square distribution, Rayleigh distribution, Rician distribution etc. Out of these distributions, you will encounter Gaussian distribution or Gaussian Random variable in digital communication very often.

Mean:

The mean of a random variable is defined as the weighted average of all possible values the random variable can take. Probability of each outcome is used to weight each value when calculating the mean. Mean is also called expectation (E[X])

For continuos random variable X and probability density function f_X(x)

For discrete random variable X, the mean is calculated as weighted average of all possible values (x_i) weighted with individual probability (p_i)

Variance :

Variance measures the spread of a distribution. For a continuous random variable X, the variance is defined as

For discrete case, the variance is defined as

Standard Deviation () is defined as the square root of variance

Properties of Mean and Variance:

For a constant – “c” following properties will hold true for mean

For a constant – “c” following properties will hold true for variance

PDF and CDF define a random variable completely. For example: If two random variables X and Y have the same PDF, then they will have the same CDF and therefore their mean and variance will be same.
On the otherhand, mean and variance describes a random variable only partially. If two random variables X and Y have the same mean and variance, they may or may not have the same PDF or CDF.

Gaussian Distribution :

Gaussian PDF looks like a bell. It is used most widely in communication engineering. For example , all channels are assumed to be Additive White Gaussian Noise channel. What is the reason behind it ? Gaussian noise gives the smallest channel capacity with fixed noise power. This means that it results in the worst channel impairment. So the coding designs done under this most adverse environment will give superior and satisfactory performance in real environments. For more information on “Gaussianity” refer [1]

The PDF of the Gaussian Distribution (also called as Normal Distribution) is completely characterized by its mean () and variance(),

Since PDF is defined as the first derivative of CDF, a reverse engineering tell us that CDF can be obtained by taking an integral of PDF.
Thus to get the CDF of the above given function,

Equations for PDF and CDF for certain distributions are consolidated below

Probability Distribution	Probability Density Function(PDF)	Cumulative Distribution Function (CDF)
Gaussian/Normal Distribution –

Reference :

[1] S.Pasupathy, “Glories of Gaussianity”, IEEE Communications magazine, Aug 1989 – 1, pp 38.

Topics in this chapter

Books by the author

Wireless Communication Systems in Matlab Second Edition(PDF) Note: There is a rating embedded within this post, please visit this post to rate it. Checkout Added to cart	Digital Modulations using Python (PDF ebook) Note: There is a rating embedded within this post, please visit this post to rate it. Checkout Added to cart	Digital Modulations using Matlab (PDF ebook) Note: There is a rating embedded within this post, please visit this post to rate it. Checkout Added to cart
Hand-picked Best books on Communication Engineering Best books on Signal Processing

Cookie	Duration	Description
cookielawinfo-checbox-analytics	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checbox-analytics	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checbox-functional	11 months	The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checbox-functional	11 months	The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checbox-others	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checbox-others	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-necessary	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-performance	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy	11 months	The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.