Estimation Theory

Theoretical derivation of MLE for Exponential Distribution:

by

As a pre-requisite, check out the previous article on the logic behind deriving the maximum likelihood estimator for a given PDF. Let X=(x1,x2,…, xN) are the samples taken from Exponential distribution given by $latex f(x;\theta) = \theta e^{-\theta x } $ Calculating the Likelihood The log likelihood is given by, Differentiating and equating to zero … Read more

Theoretical derivation of Maximum Likelihood Estimator for Poisson PDF:

by

Suppose X=(x1,x2,…, xN) are the samples taken from a random distribution whose PDF is parameterized by the parameter $latex \theta $. If the PDF of the underlying parameter satisfies some regularity condition (if the log of the PDF is differentiable) then the likelihood function is given by Here $latex f_N(x_N;\theta) $ is the PDF of … Read more

Maximum Likelihood Estimation (MLE) : Understand with example

by

Key focus: Understand maximum likelihood estimation (MLE) using hands-on example. Know the importance of log likelihood function and its use in estimation problems. Maximum Likelihood Estimation (MLE) is a statistical method used to estimate the parameters of a statistical model. The core idea behind MLE is to find the parameter values that maximize the likelihood … Read more

Estimator Bias

by

Estimator bias: Systematic deviation from the true value, either consistently overestimating or underestimating the parameter of interest. Estimator Bias: Biased or Unbiased Consider a simple communication system model where a transmitter transmits continuous stream of data samples representing a constant value – ‘A’. The data samples sent via a communication channel gets added with White … Read more