Maximum Likelihood estimation

Maximum Likelihood estimation (MLE) is an important tool in determining the actual probabilities of the assumed model of communication.

In reality, a communication channel can be quite complex and a model becomes necessary to simplify calculations at decoder side.The model should closely approximate the complex communication channel. There exist a myriad of standard statistical models that can be employed for this task; Gaussian, Binomial, Exponential, Geometric, Poisson,etc., A standard communication model is chosen based on empirical data.

Each model mentioned above has unique parameters that characterizes them. Determination of these parameters for the chosen model is necessary to make them closely model the communication channel at hand.

Suppose a binomial model is chosen (based on observation of data) for the error events over a particular channel, it is essential to determine the probability (p) of the binomial model.

If a Gaussian model (normal distribution!!!) is chosen for a particular channel then estimating \(\mu\) (mean) and \(\sigma^{2}\) (variance) are necessary so that they can be applied while computing the conditional probability of p(y received | x sent)

Similarly estimating lambda is a necessity for a Poisson distribution model.

Maximum likelihood estimation is a method to determine these unknown parameters associated with the corresponding chosen models of the communication channel.

An Example for MLE :

The following data is presented based on the observations over a Binary Symmetric Channel (BSC) (p=0.5) modeled as a Binomially distributed model.

90 codewords (each of them 10 bits wide) are transmitted over the BSC channel, of which 410 bits in total are received erroneously.Lets estimate the actual probability of success for this model.

The BSC channel is assumed to have p=0.5, meaning that the probability of error for ‘0’ and ‘1’ are equal and the probability of success is also 0.5.

Matlab code:

Output :

MLE of q is 0.544000

Therefore the probability of success for this BSC is q= 0.544 whereas the probability of error is p=1-q=0.456

Maximum Likelihood Estimation

See also:

[1] An Introduction to Estimation Theory
[2] Maximum Likelihood Decoding
[3] Probability and Random Process

Recommended Books

More Recommended Books at our Integrated Book Store