Maximum Likelihood estimation
In reality, a communication channel can be quite complex and a model becomes necessary to simplify calculations at decoder side.The model should closely approximate the complex communication channel. There exist a myriad of standard statistical models that can be employed for this task; Gaussian, Binomial, Exponential, Geometric, Poisson,etc., A standard communication model is chosen based on empirical data.
Each model mentioned above has unique parameters that characterizes them. Determination of these parameters for the chosen model is necessary to make them closely model the communication channel at hand.
Suppose a binomial model is chosen (based on observation of data) for the error events over a particular channel, it is essential to determine the probability (p) of the binomial model.
If a Gaussian model (normal distribution!!!) is chosen for a particular channel then estimating (mean) and (variance) are necessary so that they can be applied while computing the conditional probability of p(y received | x sent)
Similarly estimating lambda is a necessity for a Poisson distribution model.
Maximum likelihood estimation is a method to determine these unknown parameters associated with the corresponding chosen models of the communication channel.
An Example for MLE :
The following data is presented based on the observations over a Binary Symmetric Channel (BSC) (p=0.5) modeled as a Binomially distributed model.
90 codewords (each of them 10 bits wide) are transmitted over the BSC channel, of which 410 bits in total are received erroneously.Lets estimate the actual probability of success for this model.
The BSC channel is assumed to have p=0.5, meaning that the probability of error for ’0′ and ’1′ are equal and the probability of success is also 0.5.
% Maximum likelihood Estimation
d=410; %Number of bits in error
n=90*10; %Total number of bits sent
k=n-d; %Number of Bits NOT in error
q=0:0.002:1; %range of success probability to test likelihood
y=binopdf(k,n,q); % assuming binomial distribution
title('Maximum Likelihood Estimation');
[maxY,maxIndex]=max(y); % Finding the Max and its index
disp(sprintf('MLE of q is %f',q(maxIndex))) %print the probability corresponding to the max(y)
MLE of q is 0.544000
Therefore the probability of success for this BSC is q= 0.544 whereas the probability of error is p=1-q=0.456