Eigendecomposition and Singular Value Decomposition (SVD) are foundational, yet their numerical behaviour diverges dramatically under stress. In applications like muon tomography or gravitational wave analysis, where data matrices can be nearly singular or non-normal, the choice is not academic, it dictates the reliability of the result. While eigendecomposition provides a profound look into a system's dynamics through its eigenvalues, its numerical fragility often makes it unsuitable for direct application on raw, high-dimensional data.

The core issue lies in the basis. For any real matrix $A$, SVD provides two sets of orthogonal bases, the left and right singular vectors. Eigendecomposition, however, only guarantees an orthogonal basis of eigenvectors if the matrix is normal ($AA^T = A^T A$). For a general non-normal matrix—a common occurrence in real-world systems—the eigenvectors can be nearly linearly dependent. This lack of orthogonality is a primary source of numerical instability.

The stability of an eigendecomposition $A = PDP^{-1}$ depends critically on the condition number of the eigenvector matrix $P$, denoted $\kappa(P)$. If the eigenvectors are not orthogonal, $P$ can be ill-conditioned, meaning $\kappa(P) = \|P\|\|P^{-1}\|$ is large. A perturbation $\delta A$ to the matrix $A$ can cause a change in an eigenvalue $\lambda_i$ that is magnified by this condition number. The system's sensitivity explodes not because of the conditioning of $A$ itself, but because of the non-orthogonality of its eigenvectors.

SVD, defined as $A = U\Sigma V^T$, sidesteps this problem entirely. The left and right singular vectors, columns of $U$ and $V$, are guaranteed to be orthogonal bases regardless of whether $A$ is normal. Consequently, the SVD computation is always backward stable; the computed decomposition is the exact decomposition of a nearby matrix $A + \delta A$. This is not guaranteed for the eigendecomposition of a non-normal matrix.

Another failure mode for eigendecomposition is sensitivity to the spectral gap. If eigenvalues are clustered closely together, their corresponding eigenvectors become extremely sensitive to perturbations. Even for a normal matrix, a small change in $A$ can cause the eigenvectors for a cluster of eigenvalues to rotate wildly within the subspace they span. SVD is less susceptible to this, as the singular values are generally more stable.

The formal connection between the two decompositions is well-known. The singular values of $A$ are the square roots of the eigenvalues of the Gram matrix $A^T A$ (or $A A^T$). The right singular vectors ($V$) are the eigenvectors of $A^T A$, and the left singular vectors ($U$) are the eigenvectors of $A A^T$.

This relationship might suggest a simple algorithm for finding the SVD: form $A^T A$ and compute its eigendecomposition. This is a catastrophic mistake from a numerical standpoint. The process of forming $A^T A$ squares the condition number of the original matrix $A$.

Modern SVD algorithms, like variants of the Golub-Kahan-Reinsch algorithm, work directly on the matrix $A$. They avoid the explicit formation of $A^T A$, thereby preventing this catastrophic loss of precision. They operate through a series of orthogonal transformations (like Householder reflections) that preserve the singular values and maintain numerical stability.

For any application demanding high fidelity, the message is clear. While eigendecomposition offers irreplaceable insights for normal matrices and theoretical dynamics, for practical, numerical work on general matrices, SVD provides a foundation of stability that eigendecomposition cannot promise.

Let's test your understanding of these critical distinctions.

Choosing the right decomposition method is a fundamental aspect of robust numerical analysis, ensuring that the computed results accurately reflect the underlying system, not the artifacts of calculation.