Numerical Linear Algebra

Numerical Linear Algebra 数值线性代数 数值线性代数主要涉及高效处理向量和矩阵的本质思想，这些思想可以拓展应用到函数和算子。从这个意义上来说，数值线性代数实际上是泛函分析，但她始终强调实用的算法思想及其数学原理。如果说有其他数学科目像微积分和微分方程一样基础的话，那就是数值线性代数。 The study of applying linear algebra fast and efficiently with computers.

• Prerequisites:   Mathematical Analysis, Advanced Algebra, Matlab Programming

• Highlights:   专业核心课程，深入探讨数值线性代数

• Classroom quizzes:

  • Quiz I (Lectures 1 – 4)

  • Quiz II (Lectures 5 – 10)

• Final exam

• Grading Policy: Assignments 30% + Classroom quizzes 20% + Final exam 50%

Instructor

• Kui Du, Email: kuidu@xmu.edu.cn   Office: B603 Math Building, Haiyun Campus

• Teaching Assistants: Jia Huang 黄嘉, Email: 2645065968@qq.com Jia-Jun Fan 范家俊 Email: jiajunfan@stu.xmu.edu.cn

References

• Numerical Linear Algebra, Lloyd N. Trefethen and David Bau, III, SIAM, 1997. Twenty-fifth Anniversary Edition, 2022

• Applied Numerical Linear Algebra, James Demmel, SIAM, 1997

• Matrix Analysis and Applied Linear Algebra, Study and Solutions Guide, Carl D. Meyer, 2nd Edition, SIAM, 2023

• Matrix Analysis and Computations, Zhong-Zhi Bai and Jian-Yu Pan, SIAM, 2021

• Matrix Computations, Gene H. Golub and Charles F. Van Loan, 4th Edition, Johns Hopkins University Press, 2013

• Numerical Linear Algebra An Introduction, Holger Wendland, Cambridge University Press, 2018

• Linear Algebra, Data Science, and Machine Learning, Jeff Calder and Peter J. Olver, Springer, 2025

• Iterative Methods and Preconditioners for Systems of Linear Equations, Gabriele Ciaramella and Martin J. Gander, SIAM, 2022

• Iterative Methods for Sparse Linear Systems, Yousef Saad, 2nd Edition, SIAM, 2003

• Iterative Krylov Methods for Large Linear Systems, Henk Van der Vorst, Cambridge University Press, 2003

• Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, R. Barrett et al., 2nd Edition, SIAM, 1994. Matlab, Fortran, and C++ codes

• Accuracy and Stability of Numerical Algorithms, Nicholas J. Higham, 2nd Edition, SIAM, 2002

• 数值线性代数, 曹志浩, 复旦大学出版社, 1996

• 数值线性代数, 徐树方, 高立, 张平文, 第二版, 北京大学出版社, 2013

• 数值线性代数, 高卫国, 魏轲, 柏兆俊, 高等教育出版社, “101计划”核心教材数学领域, 2025

Lecture Notes

• Lecture 1: Inner product, Orthogonality, Vector/Matrix norms

• Lecture 2: Singular value decomposition (SVD)

• Lecture 3: Projector, Classical/Modified Gram–Schmidt orthogonalization, QR factorization

• Lecture 4: Householder reflector, Givens rotation, Least squares problem

• Lecture 5: LU factorization, Cholesky factorization, Gaussian elimination with pivoting

• Lecture 6: Stationary iterative methods

• Lecture 7: Eigenvalue problem

• Lecture 8: Power/Inverse iteration, Rayleigh quotient iteration

• Lecture 9: QR algorithm

• Lecture 10: Jacobi method, Bisection method, Divide-and-conquer method

• Lecture 11: Krylov subspace, Generalized minimal residual method

• Lecture 12: Conjugate gradients    (Proof of Theorem 1)

• Lecture 13: Biorthogonalization methods

• Lecture 14: Krylov subspace methods for least squares problems

• Lecture 15: Krylov subspace methods for eigenvalue problems

• Lecture 16: From Lanczos to Gauss quadrature

• Lecture 17: FFT and structured matrices

• Lecture 18: Multigrid

• Lecture 19: Conditioning of a problem

• Lecture 20: Backward stability of an algorithm

Assignments

• Assignment for Lecture 1 (100 points)

• Assignment for Lecture 2 (120 points)

• Assignment for Lecture 3 (100 points)

• Assignment for Lecture 4 (120 points)

• Assignment for Lecture 5 (80 points)

• Assignment for Lecture 6 (50 points)

• Assignment for Lecture 7 (90 points)

• Assignment for Lecture 8 (30 points)

• Assignment for Lecture 9 (40 points)

• Assignment for Lecture 10 (60 points)

• Assignment for Lecture 11 (60 points)

• Assignment for Lecture 12 (80 points)

• Assignment for Lecture 13 (30 points)

• Assignment for Lecture 14 (40 points)

• Assignment for Lecture 15 (20 points)

• Assignment for Lecture 16 (20 points)

• Assignment for Lecture 17 (60 points)

• Assignment for Lecture 18 (20 points)

• Assignment for Lecture 19 (50 points)

• Assignment for Lecture 20 (30 points)

Other

• The Matrix Cookbook, Kaare Brandt Petersen and Michael Syskind Pedersen

• Image Compression with Singular Value Decomposition, Tim Baumann

• 矩阵计算 / 数值线性代数, 华东师范大学, 潘建瑜

• Matrix Computations / Numerical Linear Algebra, U.C. Berkeley, James Demmel

• Computational Aspects of Matrix Theory, University of Minnesota, Yousef Saad

• PETSc: the Portable, Extensible Toolkit for Scientific Computation