Numerical Linear Algebra for Signal Processing

Introduction to the fundamentals of numerical linear algebra with the application to signal processing problems.

Floating-point arithmetics: IEEE standard, error of floating-point arithmetic.

Preliminaries from linear algebra: singular value decomposition (SVD), projectors, matrix norms, Householder reflection, Givens rotation.

QR factorization: Gram-Schmidt orthogonalization, Householder triangularization, applications of QR factorization. Back substitution: solving a triangular equation system, inversion of a triangular matrix and application to channel equalization.

Least squares: least squares with Cholesky factorization, QR factorization, and SVD; rank-deficient least squares, application to least squares estimation.

Condition of a problem: norm-wise & component-wise condition numbers, condition number of basic operations, condition of inner product, matrix-vector product, unitary matrix, and equation system.

Stability of an algorithm: backward stability, accuracy based on backward stability; stability of floating-point arithmetic, algebraic operations, Householder triangularization, back substitution; stability of solving equation systems via Householder triangularization.

Systems of equations: Gaussian elimination, pivoting, stability of Gaussian elimination; Cholesky factorization, pivoting, stability.

Eigenvalues: Hessenberg form; Rayleigh quotient iteration, QR algorithm, application to principal component analysis; SVD, bi-diagonal form, implicit Q-theorem, Golub-Reinsch SVD, application to blind channel estimation.