Problem
Differentiating through eig and svd is fragile, especially for complex matrices and whenever eigenvalues or singular values are repeated or nearly repeated. In those cases the associated basis vectors are not uniquely defined, so exposing derivatives of eigenvectors or singular vectors can produce unstable, misleading, or undefined results. Although it is possible to “stabilize” raw vectors using heuristics such as sorting, sign or phase fixing, rank truncation, or mode selection, these only mask part of the problem and still encourage downstream code to rely on derivatives of U and V that are fundamentally gauge-dependent.
Proposed solution
Add a safer spectral API that supports differentiable spectral quantities, such as eigenvalues or singular values, without exposing raw basis vectors as differentiable outputs. The library could still use stabilization heuristics internally, or provide representative basis vectors for plotting and inspection, but those values should be returned through an explicitly nondifferentiable API. This would preserve practical workflows that need mode shapes or bases for analysis, while discouraging risky autodiff code that treats stabilized U and V as if they were generally safe differentiable objects.
More detail
The key issue is not that stabilization is impossible, but that it gives a false sense of safety. For example, sorting can make outputs look more consistent, sign or phase conventions can reduce jumps, and truncation can suppress obvious numerical blow-ups; this is roughly the kind of approach used in EquationFreeGit. However, these strategies do not remove the underlying non-uniqueness of basis vectors near repeated or clustered spectra, and they do not make derivatives of individual columns of U or V mathematically reliable in general. If the API returns stabilized vectors as ordinary differentiable outputs, users are still encouraged to build downstream logic on top of them, which makes the interface a footgun even when the implementation is careful. A better contract would be to expose the stable differentiable part, such as spectral values and possibly subspace-level objects, while treating basis access as value-only so risky uses of U and V are not silently legitimized.
One possible API shape would be:
spec = spectralDecomposition(A);
s = spec.singularValues(); % differentiable
lam = spec.eigenvalues(); % differentiable when supported
sub = spec.invariantSubspace(selector); % differentiable subspace handle
B = sub.basis(); % nondifferentiable basis object
v = sub.mode(k); % nondifferentiable representative mode
P = sub.projector(); % optional differentiable projector
y = sub.project(x); % optional differentiable projection
For the nondifferentiable outputs, the API could make that explicit instead of silently returning ordinary autodiff values. For example:
B = sub.basis();
double(B) % allowed, returns numeric values for plotting/inspection
getvalue(B) % allowed
getderivs(B) % error: basis() is nondifferentiable
or via a dedicated wrapper type such as:
B = NonDifferentiableValue(values, 'basis vectors are gauge-dependent');
This would let users inspect, plot, or post-process representative basis vectors, while making it clear that these objects are not safe inputs to further autodiff computations.
Problem
Differentiating through eig and svd is fragile, especially for complex matrices and whenever eigenvalues or singular values are repeated or nearly repeated. In those cases the associated basis vectors are not uniquely defined, so exposing derivatives of eigenvectors or singular vectors can produce unstable, misleading, or undefined results. Although it is possible to “stabilize” raw vectors using heuristics such as sorting, sign or phase fixing, rank truncation, or mode selection, these only mask part of the problem and still encourage downstream code to rely on derivatives of U and V that are fundamentally gauge-dependent.
Proposed solution
Add a safer spectral API that supports differentiable spectral quantities, such as eigenvalues or singular values, without exposing raw basis vectors as differentiable outputs. The library could still use stabilization heuristics internally, or provide representative basis vectors for plotting and inspection, but those values should be returned through an explicitly nondifferentiable API. This would preserve practical workflows that need mode shapes or bases for analysis, while discouraging risky autodiff code that treats stabilized U and V as if they were generally safe differentiable objects.
More detail
The key issue is not that stabilization is impossible, but that it gives a false sense of safety. For example, sorting can make outputs look more consistent, sign or phase conventions can reduce jumps, and truncation can suppress obvious numerical blow-ups; this is roughly the kind of approach used in EquationFreeGit. However, these strategies do not remove the underlying non-uniqueness of basis vectors near repeated or clustered spectra, and they do not make derivatives of individual columns of U or V mathematically reliable in general. If the API returns stabilized vectors as ordinary differentiable outputs, users are still encouraged to build downstream logic on top of them, which makes the interface a footgun even when the implementation is careful. A better contract would be to expose the stable differentiable part, such as spectral values and possibly subspace-level objects, while treating basis access as value-only so risky uses of U and V are not silently legitimized.
One possible API shape would be:
For the nondifferentiable outputs, the API could make that explicit instead of silently returning ordinary autodiff values. For example:
or via a dedicated wrapper type such as:
This would let users inspect, plot, or post-process representative basis vectors, while making it clear that these objects are not safe inputs to further autodiff computations.