Introduction to Cramer Rao Lower Bound (CRLB)
November 19, 2012 in Estimation Theory
The criteria for existence of having an Minimum Variance Unbiased Estimator (MVUE) was discussed in a previous article. To have an MVUE, it is necessary to have estimates that are unbiased and that give minimum variance (compared to the true parameter value). This is given by the following two equations
For a MVUE, it is easier to verify the first criteria (unbiased-ness) using the first equation, but verifying the second criteria (minimum variance) is tricky. We can only calculate the variance of the estimator, but how can we make sure that it is “the minimum”? How can we make sure that an estimator that is designed gives the minimum variance? There may exist other numerous unbiased estimators (which we may not know) that may give minimum variance. Other words, how do we make sure that our estimate is the best MVUE in the world? Cramer-Rao Lower Bound (CRLB) may come to our rescue.
Cramer-Rao Lower Bound (CRLB):
Harald Cramer and Radhakrishna Rao derived a way to express the lower bound on the variance of unbiased estimators that estimate deterministic parameters. This lower bound is called as the Cramer-Rao Lower Bound (CRLB).
If is an unbiased estimate of a deterministic parameter , then the relationship between the variance of the estimates ( ) and CRLB can be expressed as
CRLB tell us the best minimum variance that we can expect to get from an unbiased estimator.
Applications of CRLB include :
1) Making judgment on proposed estimators. Estimators whose variance is not close to CRLB are considered inferior.
2) To do feasibility studies as to whether a particular estimator/system can meet given specifications. It is also used to rule out impossible estimators – No estimator can beat CRLB.
3) Benchmark for comparing unbiased estimators
4) It may sometimes provide MVUE. If an unbiased estimator achieved CRLB, it means that it is a MVUE.
Feasibility Studies :
Derivation of CRLB for a particular given scenario or proposed algorithm of estimation is often found in research texts. The derived theoretical CRLB for a system/algorithm is compared with actual variance of the implemented system and conclusions are drawn. For example, in the paper titled “A Novel Frequency Synchronization Algorithm and its Cramer Rao Bound in Practical UWB Environment for MB-OFDM Systems” – a frequency offset estimation algorithm was proposed for estimating frequency offsets in multi-band orthogonal frequency division multiplexing (MB-OFDM) systems. The performance of the algorithm was studied by BER analysis (Eb/N0 Vs BER curves). Additionally,the estimator performance is further validated by comparing the simulated estimator variance with the derived theoretical CRLB for four UWB channel models.
 An Introduction to Estimation Theory
 Bias of an Estimator
 Minimum Variance Unbiased Estimators (MVUE)
 Maximum Likelihood Estimation
 Maximum Likelihood Decoding
 Probability and Random Process
 Likelihood Function and Maximum Likelihood Estimation (MLE)
 Score, Fisher Information and Estimator Sensitivity