I will utilize the test method 2 to implement a small matlab code to check if a matrix is positive definite.The test method 2 relies on the fact that for a positive definite matrix, the determinants of all upper-left sub-matrices are positive.The following Matlab code uses an inbuilt Matlab function -‘det’ – which gives the determinant of an input matrix.
Furthermore, the successive upper \(k \times k\) sub-matrices are got by using the following notation
subA=A(1:i,1:i)
I will explain how this notation works to give the required sub-matrices.
Select elements from 1st row-1st column to 3rd row-2nd column
Matlab Code to test if a matrix is positive definite:
function x=isPositiveDefinite(A)
%Function to check whether a given matrix A is positive definite
%Author Mathuranathan for https://www.gaussianwaves.com
%Returns x=1, if the input matrix is positive definite
%Returns x=0, if the input matrix is not positive definite
%Throws error if the input matrix is not symmetric
%Check if the matrix is symmetric
[m,n]=size(A);
if m~=n,
error('A is not Symmetric');
end
%Test for positive definiteness
x=1; %Flag to check for positiveness
for i=1:m
subA=A(1:i,1:i); %Extract upper left kxk submatrix
if(det(subA)<=0); %Check if the determinent of the kxk submatrix is +ve
x=0;
break;
end
end
if x
display('Given Matrix is Positive definite');
else
display('Given Matrix is NOT positive definite');
end
end
Sample run:
>> A=[1 2 3; 4 5 6; 7 8 9]
\(A =\begin{bmatrix}
1 & 2 & 3\\
4 & 5 & 6\\
7 & 8 & 9\end{bmatrix}\)
>> x=isPositiveDefinite(A)
Given Matrix is NOT positive definite
x = 0
------------------------------------------
>> A=[25 15 -5; 15 18 0;-5 0 11]
\(A =\begin{bmatrix}
25 & 15 & -5\\
15 & 18 & 0\\
-5 & 0 & 11 \end{bmatrix}\)
>> x=isPositiveDefinite(A)
Given Matrix is Positive definite
x = 1
------------------------------------------
>> A=[1 2 3; 4 5 6]
\(A =\begin{bmatrix}
1 & 2 & 3\\
4 & 5 & 6 \end{bmatrix}\)
>> x=isPositiveDefinite(A)
Error using isPositiveDefinite (line 11)
A is not Symmetric
------------------------------------------
Key focus: Know the expressions to solve triangular matrix using forward and backward substituting techniques and the FLOPS required for solving it.
Forward Substitution:
Consider a set of equations in a matrix form , where A is a lower triangular matrix with non-zero diagonal elements. The equation is re-written in full matrix form as
It can be solved using the following expressions
From the DSP implementation point of view, computation of requires one FLoating Point Operation per Second (FLOPS) – only one division. Computing will require 3 FLOPS – 1 multiplication, 1 division and 1 subtraction, will require 5 FLOPS – 2 multiplications, 1 division and two subtractions. Thus the computation of will require FLOPS.
Thus the overall FLOPS required for forward substitution is FLOPS
Backward substitution:
Consider a set of equations in a matrix form , where A is a upper triangular matrix with non-zero diagonal elements. The equation is re-written in full matrix form as
Solved using the following algorithm
This one also requires FLOPS.
Rate this article: Note: There is a rating embedded within this post, please visit this post to rate it.
In order to perform Cholesky Decomposition of a matrix, the matrix has to be a positive definite matrix. I have listed down a few simple methods to test the positive definiteness of a matrix.
Methods to test Positive Definiteness:
Remember that the term positive definiteness is valid only for symmetric matrices.
Test method 1: Existence of all Positive Pivots
For a matrix to be positive definite, all the pivots of the matrix should be positive. Hmm.. What is a pivot ?
Pivots:
Pivots are the first non-zero element in each row of a matrix that is in Row-Echelon form. Row-Echelon form of a matrix is the final resultant matrix of Gaussian Elimination technique.
In the following matrices, pivots are encircled.
A positive definite matrix will have all positive pivots. Only the second matrix shown above is a positive definite matrix. Also, it is the only symmetric matrix.
Test method 2: Determinants of all upper-left sub-matrices are positive:
Determinant of all upper-left sub-matrices must be positive.
Break the matrix in to several sub matrices, by progressively taking upper-left elements. If the determinants of all the sub-matrices are positive, then the original matrix is positive definite.
Is the following matrix Positive Definite?
Find the determinants of all possible upper sub-matrices.
Test method 3: All Positive Eigen Values
If all the Eigen values of the symmetric matrix are positive, then it is a positive definite matrix.
Is if following matrix Positive definite ?
Since, not all the Eigen Values are positive, the above matrix is NOT a positive definite matrix.
There exist several methods to determine positive definiteness of a matrix. The method listed here are simple and can be done manually for smaller matrices.
Rate this article: Note: There is a rating embedded within this post, please visit this post to rate it.
Note: There is a rating embedded within this post, please visit this post to rate it.
Matrix inversion is seen ubiquitously in signal processing applications. For example, matrix inversion is an important step in channel estimation and equalization. For instance, in GSM normal burst, 26 bits of training sequence are put in place with 114 bits of information bits. When the burst travels over the air interface (channel), it is subject to distortions due to channel effect like Inter Symbol Interference (ISI). It becomes necessary to estimate the channel impulse response (H) and equalize the effects of the channel, before attempting to decode/demodulate the information bits. The training bits are used to estimate the channel impulse response.
If the transmitted signal “x” travels over a multipath fading channel (H) with AWGN noise “w”, the received signal is modeled as
A Minimum Mean Square Error (MMSE) linear equalizer employed to cancel out the effects of ISI, attempts to minimize the error between equalizer output – “” and the transmitted signal ““. If the AWGN noise power is , then the equalizer is represented by the following equation[1].
Note that the expression involves the computation of matrix inversion – .
Matrix inversion is a tricky subject. Not all matrices are invertible. Furthermore, ordinary matrix inversion technique of finding the adjoint of a matrix and using it to invert the matrix will consume lots of memory and computation time. Physical layer algorithm (PHY) designers typically use Cholesky decomposition to invert the matrix. This helps to reduce the computational complexity of matrix inversion.
Key focus : Essential matrix algebra: formation of matrices, determinants, rank, inverse & transpose of matrix and solving simultaneous equations.
I thought of making a post on Cholesky Decomposition, which is a very essential technique in digital signal processing applications like generating correlated random variables, solving linear equations, channel estimation etc.., But jumping straight to the topic of Cholesky Decomposition will leave many flummoxed/confused. So I decided to touch on some essentials in basic matrix algebra before taking up advanced topics.
Prerequisite:
Basic knowledge on topics like formation of matrices, determinants, rank of a matrix, inverse & transpose of matrix, solving a system of simultaneous equation using matrix algebra.
The prerequisites listed above being fulfilled, you will learn different types of matrices in this post.
1. Vector
A matrix with only one row or one column.
2. Transpose of a Matrix
Transpose of a matrix is formed by interchanging the elements from row to columns. For example, the first row of the matrix becomes first column in the transpose matrix, second row of the matrix becomes second column in the transpose matrix and so on.
3. Square Matrix:
Matrix with equal number of rows and columns.
(Note: The sample matrices shown below are of 3×3 dimension. They can be readily extended to nxn dimension)
Square Matrix is further classified into
4. Symmetric Matrix :
If a square matrix and its transpose are equal, then it is called a symmetric square matrix or simply symmetric matrix.
5. Diagonal Matrix
A special kind of symmetric matrix, with zeros in off-diagonal locations.
6. Scalar matrix
A special kind of diagonal matrix, with the equal values at the diagonal
7. Identity Matrix
It is a special type of scalar matrix, where the leading diagonals are one. It is denoted by I
8. Inverse of a Matrix or the notion of non-Singular Matrix:
Inverse matrices are defined for square matrices. For non-square matrices, inverses do not exist. The product of a square matrix and its inverse yields an identity matrix.
For a square matrix A.
Here denotes the inverse of square matrix .
It is possible to have a square matrix but not invertible. Such non-invertible square matrices are called
Singular Matrices. Matrix that are invertible are called non-singular matrices.
9. Singular Matrix:
A non-invertible square matrix. Inverse does not exist.
10. Orthogonal Matrix
An invertible square matrix, whose transpose and inverse are equal. If for an invertible square matrix A,
then, the matrix or equivalently is called an orthogonal matrix.
11. Complex Matrix :
A matrix with complex elements.
12. Complex Square Matrix
A complex matrix with with equal number of rows and columns
13. Conjugate Transpose or Hermitian Transpose:
Similar to the transpose of a matrix defined above. But the only difference is : put the complex conjugate form of the element when interchanging from rows to columns. For a complex square matrix , the conjugate transpose is denoted by
14. Hermitian Matrix:
A complex square matrix whose conjugate transpose is equal to its own. For a complex square matrix , the Hermitian Matix is denoted by . For a Hermitian matrix, the element of the complex square matrix located at i-th row and j-th column will be equal to the complex conjugate of the element located at j-th row and i-th column.
That is,
15. Triangular Matrix:
A special type of square matrix. Further classified into : lower triangular and upper triangular matrix. For a lower triangular matrix, all the elements above the main diagonal are zeros.For an upper triangular matrix, all the elements below the main diagonal are zeros.
16. Positive-Definite Matrix:
It is defined for both real matrices and complex matrices. It is matrix equivalent to positivity of real numbers. For example, the numbers +3,+5 and +6 are definitely positive. Similarly a positive definite matrix is defined to be definitely positive if it satisfies the following condition.
For real case: A n x n symmetric square real matrix – is said to be positive definite for all if and only if,
For complex case: A n x n symmetric square complex matrix – is said to be positive definite for all if and only if,
17. Positive-semi-definite Matrix:
It is defined for both real matrices and complex matrices. It is matrix equivalent to semi-positivity of real numbers. For example, the numbers +3,+5 and +6 are definitely positive. But the set is not positive definite since it includes a zero element which is neither positive nor negative. Similarly a positive semi-definite matrix is defined to be semi-definitely positive if it satisfies the following condition.
For real case: A n x n symmetric square real matrix – is said to be positive semi-definite for all if and only if,
For complex case: A n x n symmetric square complex matrix – is said to be positive semi-definite for all if and only if,
Note : The notation stands for a set of real numbers. The notation denotes a set of n real numbers in n-dimensional space.
18. Negative-Definite Matrix:
It is defined for both real matrices and complex matrices. It is matrix equivalent to negativity of real numbers. For example, the numbers -3,-5 and -6 are definitely negative. Similarly a negative definite matrix is defined to be definitely negative if it satisfies the following condition.
For real case: A n x n symmetric square real matrix – is said to be negative definite for all if and only if,
For complex case: A n x n symmetric square complex matrix – is said to be negative definite for all if and only if,
Note : The notation stands for a set of real numbers. The notation denotes a set of n real numbers in n-dimensional space.
19. Negative-semi-definite Matrix:
It is defined for both real matrices and complex matrices. It is matrix equivalent to semi-negativity of real numbers. For example, the numbers -3,-5 and -6 are definitely negative. But the set is not negative definite since it includes a zero element which is neither positive nor negative. Similarly a negative semi-definite matrix is defined to be semi-definitely negative if it satisfies the following condition.
For real case: A n x n symmetric square real matrix – is said to be negative semi-definite for all if and only if,
For complex case: A n x n symmetric square complex matrix – is said to be negative semi-definite for all if and only if,
Note : The notation stands for a set of real numbers. The notation denotes a set of n real numbers in n-dimensional space.
20. Indefinite Matrix :
A Matrix, that is neither positive definite, positive semi-definite, negative definite nor negative semi-definite is called an indefinite Matrix.
Rate this article: Note: There is a rating embedded within this post, please visit this post to rate it.
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
Cookie
Duration
Description
cookielawinfo-checbox-analytics
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checbox-analytics
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checbox-functional
11 months
The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checbox-functional
11 months
The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checbox-others
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checbox-others
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-necessary
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-performance
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy
11 months
The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.