Fisher information and score — estimator sensitivity formula (Matlab)
Score function and Fisher information: how estimator sensitivity is measured, the link to CRLB, and how to compute it for common observation models.
Estimation
Theoretical derivation of MLE for Gaussian Distribution:
As a pre-requisite, check out the previous article on the logic behind deriving the maximum likelihood estimator for a given PDF. Let X=(x1,x2,…, xN) are the samples taken from Gaussian distribution given by Calculating the Likelihood The log likelihood is given by, Differentiating and equating to zero to find the maxim (otherwise equating the score … Read more
Theoretical derivation of MLE for Exponential Distribution:
As a pre-requisite, check out the previous article on the logic behind deriving the maximum likelihood estimator for a given PDF. Let X=(x1,x2,…, xN) are the samples taken from Exponential distribution given by $latex f(x;\theta) = \theta e^{-\theta x } $ Calculating the Likelihood The log likelihood is given by, Differentiating and equating to zero … Read more
Theoretical derivation of Maximum Likelihood Estimator for Poisson PDF:
Suppose X=(x1,x2,…, xN) are the samples taken from a random distribution whose PDF is parameterized by the parameter $latex \theta $. If the PDF of the underlying parameter satisfies some regularity condition (if the log of the PDF is differentiable) then the likelihood function is given by Here $latex f_N(x_N;\theta) $ is the PDF of … Read more
Maximum Likelihood Estimation (MLE) : Understand with example
Key focus: Understand maximum likelihood estimation (MLE) using hands-on example. Know the importance of log likelihood function and its use in estimation problems. Maximum Likelihood Estimation (MLE) is a statistical method used to estimate the parameters of a statistical model. The core idea behind MLE is to find the parameter values that maximize the likelihood … Read more
Estimator Bias
Estimator bias: Systematic deviation from the true value, either consistently overestimating or underestimating the parameter of interest. Estimator Bias: Biased or Unbiased Consider a simple communication system model where a transmitter transmits continuous stream of data samples representing a constant value – ‘A’. The data samples sent via a communication channel gets added with White … Read more
Estimation theory intro — unbiased estimators, variance, and CRLB path
Introduction to estimation theory: what an estimator is, bias vs variance, and how the path leads to Fisher information and the Cramér–Rao bound.