Estimation Theory

Theoretical derivation of MLE for Gaussian Distribution:

by

[ratings] As a pre-requisite, check out the previous article on the logic behind deriving the maximum likelihood estimator for a given PDF. Let X=(x1,x2,…, xN) are the samples taken from Gaussian distribution given by Calculating the Likelihood The log likelihood is given by, Differentiating and equating to zero to find the maxim (otherwise equating the … Read more

Theoretical derivation of MLE for Exponential Distribution:

by

[ratings] As a pre-requisite, check out the previous article on the logic behind deriving the maximum likelihood estimator for a given PDF. Let X=(x1,x2,…, xN) are the samples taken from Exponential distribution given by $latex f(x;\theta) = \theta e^{-\theta x } &s=2 $ Calculating the Likelihood The log likelihood is given by, Differentiating and equating … Read more

Theoretical derivation of Maximum Likelihood Estimator for Poisson PDF:

by

[ratings] Suppose X=(x1,x2,…, xN) are the samples taken from a random distribution whose PDF is parameterized by the parameter $latex \theta $. If the PDF of the underlying parameter satisfies some regularity condition (if the log of the PDF is differentiable) then the likelihood function is given by Here $latex f_N(x_N;\theta) $ is the PDF … Read more

Maximum Likelihood Estimation (MLE) : Understand with example

by

Key focus: Understand maximum likelihood estimation (MLE) using hands-on example. Know the importance of log likelihood function and its use in estimation problems. Maximum Likelihood Estimation (MLE) is a statistical method used to estimate the parameters of a statistical model. The core idea behind MLE is to find the parameter values that maximize the likelihood … Read more

Estimator Bias

by

Estimator bias: Systematic deviation from the true value, either consistently overestimating or underestimating the parameter of interest. Estimator Bias: Biased or Unbiased Consider a simple communication system model where a transmitter transmits continuous stream of data samples representing a constant value – ‘A’. The data samples sent via a communication channel gets added with White … Read more

Estimation Theory : an introduction

by

Key focus: Understand the basics of estimation theory with a simple example in communication systems. Know how to assess the performance of an estimator. A simple estimation problem : DSB-AM receiver In Double Side Band – Amplitude Modulation (DSB-AM), the desired message is amplitude modulated over a carrier of frequency f0. The following discussion is … Read more