Foundations & Data Representation
What is the difference between a scalar, a vector, a matrix, and a tensor?
Why is linear algebra considered the language of machine learning and data science?
How is a dataset represented as a matrix in machine learning?
What is a vector and how is it used in machine learning?
Explain the difference between a scalar and a vector.
What is a matrix and why is it central to linear algebra?
Explain the concept of a tensor in the context of machine learning.
What is the difference between a row vector and a column vector, and why does the distinction matter?
What determines the shape or dimensions of a matrix, and how do you read them?
Vectors & Inner Products
What is a unit vector and how do you normalize a vector?
What is the dot product and what is its geometric meaning?
What does it mean for two vectors to be orthogonal?
How does the dot product relate to the cosine of the angle between two vectors?
What is a linear combination of vectors?
What is the cross product of vectors and when is it used?
How do you calculate the norm of a vector and what does it represent?
What do span and linear independence mean for a set of vectors?
What is cosine similarity and why is it useful for embeddings or text?
What is the difference between the inner product and the outer product?
What does it mean to project one vector onto another?
Matrix Operations
What is the transpose of a matrix and why does (AB)ᵀ = BᵀAᵀ?
How is matrix multiplication defined and why is it row-times-column?
Why is matrix multiplication not commutative?
What are the dimension and conformability rules for multiplying two matrices?
What is the difference between the Hadamard (element-wise) product and matrix multiplication?
How do you perform matrix addition and subtraction?
What are the properties of matrix multiplication?
How can matrix multiplication be viewed as computing a batch of dot products?
What is the trace of a matrix and what are its properties?
How can matrix multiplication be interpreted as the composition of linear maps?
How does a matrix-vector product represent a linear transformation?
What is the computational cost or complexity of core matrix operations like multiplication?
Matrix Types & Structure
What is a symmetric matrix?
What is the identity matrix and what role does it play?
What is the difference between a sparse and a dense matrix?
What is a diagonal matrix and how is it used in linear algebra?
What is a triangular matrix and why is it convenient for solving linear systems?
What is a permutation matrix and what does multiplying by one do?
What does the covariance matrix represent as a matrix object, and what are its structural properties?
Eigenvalues & Eigenvectors
What are eigenvalues and eigenvectors, and what is their geometric meaning?
How do the trace and determinant relate to the eigenvalues of a matrix?
Explain what it means for a matrix to be diagonalizable.
What are the key properties of eigenvalues, and how can they be manipulated?
Explain Eigen decomposition and its importance or applications.
What is the difference between algebraic and geometric multiplicity of an eigenvalue?
What does the spectral theorem say about symmetric matrices?
How can PageRank or a Markov chain be framed as an eigenvector problem?
What is the characteristic polynomial and how does it relate to finding eigenvalues?
Can a real matrix have complex eigenvalues, and what does that mean geometrically?
How do eigenvalues determine the behavior of repeated matrix powers, such as in a Markov chain converging to a steady state?
Linear Systems & Least Squares
Explain the least squares method from a linear algebra perspective, focusing on its geometric interpretation such as projection.
How do you solve a system of linear equations Ax = b, and when does a solution exist or is unique?
What is Gaussian elimination and what are pivots and RREF?
Describe the difference between under-determined and over-determined linear systems, and how solutions are approached for each.
What are normal equations, and when are they used in the context of solving linear systems or least squares?
What is the pseudoinverse and how does it give a least-squares solution when there is no exact solution?
What is orthogonal projection onto a subspace and how does it connect to least squares?
Orthogonality & Basis
What are orthogonal and orthonormal matrices, and what are their key properties?
What is Gram–Schmidt orthogonalization?
How do geometric transformations like rotation, scaling, shear, and reflection correspond to matrices?
What is an orthonormal basis?
What is a change of basis?
Explain rotation parametrizations and their pros and cons.
How does an orthonormal basis simplify computing coordinates and projections?
Vector Spaces & Subspaces
What is the rank of a matrix and why is it important?
What are a basis and the dimension of a vector space?
What is the rank of a matrix and why does row rank equal column rank?
What is the null space (kernel) of a matrix and what does it tell you about solutions to Ax = 0?
What is the column space of a matrix and how does it relate to solving Ax = b?
What is a subspace of a vector space and what conditions must it satisfy?
What does it mean for a matrix to be full rank versus rank deficient, and why does it matter?
What conditions must a set of vectors satisfy to form a basis for a space?
What are the four fundamental subspaces and what do they mean?
What is an orthogonal complement of a subspace?
Determinants & Inverses
What is the determinant and what is its geometric meaning?
What is the matrix inverse and when does it exist?
What is the difference between a singular and an invertible matrix?
Why does a determinant of zero mean a matrix is singular?
What does the inverse of a product (AB)⁻¹ equal, and why does the order reverse?
How do elementary row operations affect the value of the determinant?
What is the determinant of a product of matrices, and how does it relate to the individual determinants?
What does the sign of the determinant tell you about a transformation?
Vector Norms & Distance
How do you derive a distance metric such as Euclidean distance from a vector norm?
What are vector norms and can you explain the difference between L1, L2, and L∞ norms?
What is a general Lp norm and how do the different values of p change its behavior?
What is the triangle inequality and why must a valid norm satisfy it?
What is the Frobenius norm of a matrix and how is it interpreted?
What is the spectral (operator) norm and how does it relate to singular values?
Matrix Factorizations
What is LU decomposition and how is it used to solve linear systems?
What is QR decomposition and how is it used for least squares?
Why are matrix factorizations preferred over explicit inverses for numerical computations?
What is Cholesky decomposition and when can you use it?
How does SVD relate to the eigendecomposition of AᵀA and AAᵀ?
Why does the SVD exist for any matrix?
What is the condition number of a matrix and why does it matter for numerical stability?
What floating-point pitfalls arise in linear algebra computations, and how do you guard against them?
Applied Machine Learning
How can a neural network layer be written as a linear algebra operation Wx + b?
How is PCA expressed as an eigen or SVD problem on the covariance matrix?
How does matrix factorization underpin recommendation systems?
How would you propose a method for dimensionality reduction using linear algebra techniques?
Describe a scenario where linear algebra could be used to improve model accuracy.
How would you use matrices to model relational data in databases?
How would you apply linear algebra to image processing tasks?
Definiteness & Quadratic Forms
Define positive definite matrices and explain their properties and importance in machine learning.
What do the eigenvalues of a matrix tell you about its definiteness?
Why are covariance and Gram matrices positive-semi-definite?
What is a quadratic form xᵀAx and what does it tell you about a matrix?
Singular Value Decomposition
What is the Singular Value Decomposition (SVD), and why is it a fundamental tool in data science and machine learning?
How is SVD used behind PCA, dimensionality reduction, and noise reduction?
What are singular values and singular vectors, and what do they represent?
What is low-rank approximation via truncated SVD and what does the Eckart–Young theorem say?
What is whitening a dataset and how is it expressed with linear algebra?
How does the SVD give the pseudoinverse of a matrix?
What is the relationship between the rank of a matrix and the number of nonzero singular values?