Interactive Educational Modules in
Scientific Computing

Singular Value Decomposition

This module illustrates a geometric interpretation of the singular value decomposition of a matrix A. The singular value decomposition has the form A = U Σ VT, where U and V are orthogonal matrices whose columns are called left and right singular vectors, respectively, and Σ is a diagonal matrix with nonnegative diagonal entries called singular values of A. If we think of A as transforming the unit sphere into an ellipsoid, then the singular values and left singular vectors are the lengths and directions, respectively, of the principal axes of the ellipsoid, and the right singular vectors are preimages of the left singular vectors.

The user enters a matrix in the display on the lower left or can use a preset example. The user then selects candidate left singular vectors by clicking on the graph of the image space on the right, and their preimages will be shown in the graph on the left. The placement of the vectors in the image space can be adjusted by dragging them around the ellipse, with their preimages moving accordingly around the unit circle on the left. For any given choice of image vectors, the corresponding U, Σ, and V matrices are shown below, along with the norm of the difference between their product and the original matrix A, as a measure of whether we have a true SVD.

The matrices U and V are orthogonal when the corresponding vectors in the graphs are perpendicular. A true SVD results when the image vectors on the right are aligned with the principal axes of the ellipse, in which case both the image and preimage vectors are orthogonal, the diagonal entries of Σ are extremal, and the norm of the difference is zero. The SVD is unique only up to the ordering of the singular values and the signs of the singular vectors, so more than one arrangement of the vectors along the principal axes yields a valid SVD. For some choices of vectors, the norm of the difference can be zero without yielding a true SVD, because either U or V is not orthogonal.

References:

  1. Michael T. Heath, Scientific Computing, An Introductory Survey, 2nd edition, McGraw-Hill, New York, 2002. See Section 3.6.
  2. Lloyd N. Trefethen and David Bau, Numerical Linear Algebra, SIAM, Philadelphia, 1997. See Figure 4.1.

Developers: Sukolsak Sakshuwong and Michael Heath