# An Introduction to Estimation Theory

### A simple Estimation Problem : DSB-AM receiver:

In Double Side Band – Amplitude Modulation (DSB-AM), the desired message is amplitude modulated over a carrier of frequency ** f_{0}**. The following discussion is with reference to the figure 1. In the frequency domain, the spectrum of the message signal, which is a baseband signal, may look like the one shown in (a). After the modulation over a carrier frequency of

*, the spectrum of the modulated signal will look like as shown in (b). The modulated signal has spectral components centered at*

**f**_{0}**and**

*f*_{0}**.**

*-f*_{0}The modulated signal is a function of three factors :1) actual message – \( m(t) \) , 2) carrier frequency ** f_{0}** and 3) phase uncertainty

**. The modulated signal can be expressed as,**

*Φ*_{0}$$ s(t; f_0,\phi_0) = m(t) cos(2 \pi f_0 t + \phi_0)$$

To simplify things, let’s consider that the modulated signal is passed via an ideal channel (no impairments added by the channel, so we can do away with channel equalization and other complex stuffs in the receiver). The modulated signal hits the antenna located at front end of our DSBC receiver. Usually the receiver front end is employed with a band-pass filter and amplifier to put the received signal in the desired band of operation & level, as expected by the receiver. The electronics in the front end receiver adds noise to the incoming signal (modeled as white noise – \( w(t) \) ). The signal after the BPF and amplifier combination is expressed as \( x(t) \), which is a combination of our desired signal s(t) and the front end noise \( w(t) \). Thus \( x(t) \) can be expressed as

$$ x(t) = s(t) + w(t)$$

The signal \( x(t) \) is band-pass (centered around the carrier frequency ** f_{0}**). To bring \( x(t) \) back to the baseband, a mixer is employed that multiplies \( x(t) \) with a tone centered at

**(generated by a local oscillator). Actually a low pass filter is usually used after the mixer to extract the desired signal at the baseband.**

*f*_{0 }As the receiver has no knowledge about the carrier frequency, there must exist a technique/method to extract this information from the incoming signal \( x(t) \) itself. Not only the carrier frequency (** f_{0}**) but also the phase

**of the carrier need to be known at the receiver for proper demodulation. This leads us to the problem of “estimation”.**

*Φ*_{0}### Estimation of unknown parameters:

In “estimation” problem, we are confronted with estimating one or more unknown parameters based on a sequence of observed data. In our problem, the signal \( x(t) \) is the observed data and the parameters that are to be estimated are ** f_{0}** and

**.**

*Φ*_{0}Now, we add an estimation algorithm at the receiver, that takes in the signal \( x(t) \) and computes estimates of ** f_{0}** and

**.The estimated values are denoted with a cap on their respective letters.The estimation algorithm can be simply stated as follows :**

*Φ*_{0}Given \( x(t) = s(t; f_0, \phi_0) + w(t) \), estimate \( \hat{f}_0 \)and \( \hat{\phi}_0 \) that are optimal in some sense.

Since the noise \( w(t) \) is assumed to be “white”, the probability density function (PDF) of the noise is readily available at the receiver.

So far all the notations were expressed in continuous domain. To simplify calculations, let’s state the estimation problem in discrete time domain. In discrete time domain, the samples of observed signal – which is a combination of actual signal and noise is expressed As

$$ x[n] = s[n;f_0, \phi_0] + w[n]$$

The noise samples w[n] is a random variable, that randomizes every time we observe \( x[n] \). Each time when we observe the “observed” samples – \( x[n] \) , we think of it as having the same “actual” signal samples – \( s[n] \) but with different realizations of the noise samples \( w[n] \). Thus \( w[n] \) can be modeled as a Random Variable (RV). Since the underlying noise w[n] is a random variable, the estimates \( \hat{f}_0 \) and \( \hat{\phi}_0 \) that result from the estimation are also random variables.

Now the estimation algorithm can be stated as follows:

Given the observed data samples – \( x[n] = \left( x[0], x[1],x[2], … ,x[N-1] \right) \), our goal is to find estimator functions that maps the given data into estimates.

$$ \hat{f}_{0}=g_{1}(x[0], x[1],x[2],\cdots,x[N-1])=g_{1} \left( \textbf{x} \right)$$

$$ \hat{\phi}_{0} = g_{2}(x[0], x[1],x[2],\cdots,x[N-1]) = g_{2}( \textbf{x})$$

### Assessing the performance of the estimation algorithm:

Since the estimates \( \hat{f}_0 \) and \( \hat{\phi}_0 \) are random variables, they can be described by a probability density function (PDF). The PDF of the estimates depend on following factors :

1. Structure of \( s[n] \)

2. Probability model of \( w[n] \)

3. Form of estimation function \(g(x) \)

For example, the PDF of the estimate may take the following shape,

The goal of the estimation algorithm is to give an estimate that is unbiased (mean of the estimate is equal to the actual ** f_{0}**) and has minimum variance. This criteria can be expressed as,

$$ E\left\{\hat{f}_0 \right\} = f_0 $$

$$ \sigma^{2}_{\hat{f}_0}=E\left\{(\hat{f}_0 – E[\hat{f}_0])^2 \right\} \; should \; be \; minimum $$

Same type of argument will hold for the other estimate : \( \hat{\phi}_0 \)

By these criteria one can assess the performance of an estimator. The estimator described above (with the criteria) is called “Minimum Variance Unbiased Estimator” (MVUE).

### See also:

[1] Minimum Variance Unbiased Estimators (MVUE)

[2] Bias of a Minimum Variance Estimator

[3] Maximum Likelihood Estimation

[4] Maximum Likelihood Decoding

[5] Probability and Random Process

Pingback: Minimum Variance Unbiased Estimators (MVUE) | GaussianWaves()

Pingback: Maximum Likelihood Decoding | GaussianWaves()

Pingback: Bias of a Minimum Variance Estimator | GaussianWaves()

Pingback: Likelihood Function and Maximum Likelihood Estimation (MLE) | GaussianWaves()

Pingback: Theoretical derivation of Maximum Likelihood Estimator for Poisson PDF: | GaussianWaves()

Pingback: Theoretical derivation of MLE for Exponential Distribution: | GaussianWaves()

Pingback: Theoretical derivation of MLE for Gaussian Distribution: | GaussianWaves()

Pingback: Score, Fisher Information and Estimator Sensitivity | GaussianWaves()

Pingback: Introduction to Cramer Rao Lower Bound (CRLB) | GaussianWaves()

Pingback: Cramer Rao Lower Bound for Phase Estimation | GaussianWaves()

Pingback: Best Linear Unbiased Estimator (BLUE) – Introduction | GaussianWaves()

Pingback: Hard and Soft decision decoding | GaussianWaves()

Pingback: Maximum Likelihood estimation | GaussianWaves()