Linear algebra for DSP:
Positive-definiteness tests →
BLUE estimator →
Energy (L2) ideas
In one sentence: A symmetric matrix is positive definite if all eigenvalues are positive (equivalently, a Cholesky factor exists) — required before treating it as a covariance or Hessian.
Matlab/Python simulation: Check with eig / chol or numpy.linalg.eigvalsh / cholesky before you invert a covariance.
In order to perform Cholesky Decomposition of a matrix, the matrix has to be a positive definite matrix. I have listed down a few simple methods to test the positive definiteness of a matrix.
Methods to test Positive Definiteness:
Remember that the term positive definiteness is valid only for symmetric matrices.
Test method 1: Existence of all Positive Pivots
For a matrix to be positive definite, all the pivots of the matrix should be positive. Hmm.. What is a pivot ?
Pivots:
Pivots are the first non-zero element in each row of a matrix that is in Row-Echelon form. Row-Echelon form of a matrix is the final resultant matrix of Gaussian Elimination technique.
In the following matrices, pivots are encircled.
A positive definite matrix will have all positive pivots. Only the second matrix shown above is a positive definite matrix. Also, it is the only symmetric matrix.
Test method 2: Determinants of all upper-left sub-matrices are positive:
Determinant of all$latex k \times k$ upper-left sub-matrices must be positive.
Break the matrix in to several sub matrices, by progressively taking$latex k \times k$ upper-left elements. If the determinants of all the sub-matrices are positive, then the original matrix is positive definite.
Is the following matrix Positive Definite?
Find the determinants of all possible$latex k \times k$ upper sub-matrices.
Test method 3: All Positive Eigen Values
If all the Eigen values of the symmetric matrix are positive, then it is a positive definite matrix.
Is if following matrix Positive definite ?
Since, not all the Eigen Values are positive, the above matrix is NOT a positive definite matrix.
There exist several methods to determine positive definiteness of a matrix. The method listed here are simple and can be done manually for smaller matrices.
External resource:
1) Online tool to generate Eigen Values and Eigen Vectors↗
FAQ
Why do DSP and estimation care about positive-definite matrices? Covariance matrices, Gram matrices, and many Hessians must be symmetric positive definite (SPD) for Cholesky-based solvers, for unique minima of quadratic costs, and for well-behaved Kalman/BLUE updates. If a computed covariance loses definiteness to roundoff, estimators can fail or invent negative variances.
What is a quick numerical test for positive definiteness? Attempt a Cholesky factorization. If it succeeds (within numerical tolerance), the matrix is SPD for practical purposes; if it fails, the matrix is not positive definite. Eigenvalue checks ($latex \lambda_{\min}>0$) are clearer pedagogically but slower for large dense matrices.
What is the difference between positive definite and positive semi-definite? Positive definite requires all eigenvalues strictly greater than zero (or all leading principal minors positive). Positive semi-definite allows zeros — the matrix is singular, quadratic forms are nonnegative but not strictly positive, and Cholesky in the strict sense fails.