NOTES | HOME
$$ \newcommand{\RR}{\mathbb{R}} \newcommand{\GG}{\mathbb{G}} \newcommand{\PP}{\mathbb{P}} \newcommand{\PS}{\mathcal{P}} \newcommand{\SS}{\mathbb{S}} \newcommand{\NN}{\mathbb{N}} \newcommand{\ZZ}{\mathbb{Z}} \newcommand{\CC}{\mathbb{C}} \newcommand{\HH}{\mathbb{H}} \newcommand{\ones}{\mathbb{1\hspace{-0.4em}1}} \newcommand{\alg}[1]{\mathfrak{#1}} \newcommand{\mat}[1]{ \begin{pmatrix} #1 \end{pmatrix} } \renewcommand{\bar}{\overline} \renewcommand{\hat}{\widehat} \renewcommand{\tilde}{\widetilde} \newcommand{\inv}[1]{ {#1}^{-1} } \newcommand{\eqdef}{\overset{\text{def}}=} \newcommand{\block}[1]{\left(#1\right)} \newcommand{\set}[1]{\left\{#1\right\}} \newcommand{\abs}[1]{\left|#1\right|} \newcommand{\trace}[1]{\mathrm{tr}\block{#1}} \newcommand{\vol}[1]{\mathrm{vol}\block{#1}} \newcommand{\norm}[1]{ \left\| #1 \right\| } \newcommand{\modulus}[1]{ \left| #1 \right| } \newcommand{\argmin}[1]{ \underset{#1}{\mathrm{argmin}} } \newcommand{\argmax}[1]{ \underset{#1}{\mathrm{argmax}} } \newcommand{\st}{\ \mathrm{s.t.}\ } \newcommand{\sign}[1]{\mathrm{sign}\block{#1}} \newcommand{\half}{\frac{1}{2}} \newcommand{\inner}[1]{\left\langle #1 \right\rangle} \newcommand{\dd}{\mathrm{d}} \newcommand{\ddd}[2]{\frac{\partial #1}{\partial #2} } \newcommand{\db}{\dd^b} \newcommand{\ds}{\dd^s} \newcommand{\dL}{\dd_L} \newcommand{\dR}{\dd_R} \newcommand{\Ad}{\mathrm{Ad}} \newcommand{\ad}{\mathrm{ad}} \newcommand{\LL}{\mathcal{L}} \newcommand{\Krylov}{\mathcal{K}} \newcommand{\Span}[1]{\mathrm{Span}\block{#1}} \newcommand{\diag}{\mathrm{diag}} \newcommand{\tr}{\mathrm{tr}} \newcommand{\sinc}{\mathrm{sinc}} \newcommand{\cat}[1]{\mathcal{#1}} \newcommand{\Ob}[1]{\mathrm{Ob}\block{\cat{#1}}} \newcommand{\Hom}[1]{\mathrm{Hom}\block{\cat{#1}}} \newcommand{\op}[1]{\cat{#1}^{op}} \newcommand{\hom}[2]{\cat{#1}\block{#2}} \newcommand{\id}{\mathrm{id}} \newcommand{\Set}{\mathbb{Set}} \newcommand{\Cat}{\mathbb{Cat}} \newcommand{\Hask}{\mathbb{Hask}} \newcommand{\lim}{\mathrm{lim}\ } \newcommand{\funcat}[1]{\left[\cat{#1}\right]} \newcommand{\natsq}[6]{ \begin{matrix} & #2\block{#4} & \overset{#2\block{#6}}\longrightarrow & #2\block{#5} & \\ {#1}_{#4} \hspace{-1.5em} &\downarrow & & \downarrow & \hspace{-1.5em} {#1}_{#5}\\ & #3\block{#4} & \underset{#3\block{#6}}\longrightarrow & #3\block{#5} & \\ \end{matrix} } \newcommand{\comtri}[6]{ \begin{matrix} #1 & \overset{#4}\longrightarrow & #2 & \\ #6 \hspace{-1em} & \searrow & \downarrow & \hspace{-1em} #5 \\ & & #3 & \end{matrix} } \newcommand{\natism}[6]{ \begin{matrix} & #2\block{#4} & \overset{#2\block{#6}}\longrightarrow & #2\block{#5} & \\ {#1}_{#4} \hspace{-1.5em} &\downarrow \uparrow & & \downarrow \uparrow & \hspace{-1.5em} {#1}_{#5}\\ & #3\block{#4} & \underset{#3\block{#6}}\longrightarrow & #3\block{#5} & \\ \end{matrix} } \newcommand{\cone}[1]{\mathcal{#1}} $$

Singular Value Decomposition

Mostly about the SVD/polar decomposition differentiation.

  1. SVD
  2. Derivative
  3. Polar Decomposition
    1. Jacobian Matrix

SVD

Any square matrix \(A\) can be decomposed as:

\[A = USV^T\]

where \(U, V \in O(n)\) are orthogonal matrices and \(S\) is diagonal and positive. It is unique up to a permutation of the unique diagonal values of \(S\), called the singular values.

If \(A\) is not square, the SVD still exists but \(S\) is no longer square.

Derivative

From \(A = USV^T\), we get:

\[\begin{align} \dd A &= \dd U\ S V^T + U\ \dd S\ V^T + U S\ \dd V^T\\ &= U\ \db U\ S V^T + U\ \dd S\ V^T + U S\ \db V^T\ V^T\\ \end{align}\]

where \(\dd U = U.\db U = \ds U.U\) is the (left/right) trivialization of tangent vector \(\dd U \in T_U O(n)\), in this case with \(\db U, \ds U \in \alg{o(n)}\) i.e. skew-symmetric matrices (see Lie groups for details).

From the above we get:

\[\begin{align} U^T \dd A V &= \db U\ S + \dd S + S\ \db V^T\\ &= \underbrace{\dd S}_{\text{diagonal}} \oplus \underbrace{\block{\db U\ S + S\ \db V^T}}_{\text{zero diag.}}\\ \end{align}\]

from which we may already obtain \(\dd S\) from the diagonal. Let us now call \(R = \db U\ S + S\ \db V^T = \db U\ S - S \db V\) and fiddle around a bit to obtain:

\[\begin{align} RS &= \db U\ S^2 - S\ \db V\ S \\ (RS)^T &= S\ \db V\ S - S^2 \db U \\ \end{align}\]

From this we obtain:

\[RS + (RS)^T = \db U\ S^2 - S^2 \db U\]

which we may rewrite as:

\[RS + (RS)^T = \hat{S} \circ \db U\]

where \(\hat{S} = \block{s_j^2 - s_i^2}_{i, j}\) is skew-symmetric. We then obtain \(\db U\) as:

\[\db U = \check{S}\circ \block{RS + (RS)^T}\]

where \(\check{S}_{i,j} = \begin{cases}\frac{1}{s_j^2 - s_i^2} & i \neq j \\ 0 & i = j\end{cases}\)

TODO what if \(s_i = s_j\)

Similarly, we get \(\block{SR + (SR)^T} = \hat{S} \circ \db V\) and obtain \(\db V\) as

\[\db V = \check{S}\circ \block{SR + (SR)^T}\]

Polar Decomposition

TODO REDO

We want to compute \(f: A \mapsto UV^T\) to get the closest orientation to a given invertible linear map. The derivative gives:

\[\begin{align} \dd f(A).\dd A &= \dd U\ V^T + U\ \dd V^T \\ &= U\ \db U V^T + U\ \db V^T V^T \\ &= U\underbrace{\block{\db U + \db V^T}}_Y V^T \\ \end{align}\]

from the above, we obtain \(Y\) as:

\[\begin{align} Y &= W + Z + \block{W - Z}^T\\ &= 2Z \end{align}\]

so that we only have to identify \(Z\) from the skew-symmetric part \(R_-\), which is well-defined when \(S > 0\). We end up with:

\[\dd f(A).\dd A = 2 U Z V^T\]

which we can express as a spatial derivative as:

\[\ds f(A).\dd A = 2 U Z U^T = 2 \Ad_U Z\]

Jacobian Matrix

The whole sequence of computations can be decomposed as follows:

\[\dd A \overset{G}{\mapsto} U^T\ \dd A\ V \overset{\pi}{\mapsto} R_- \overset{F}{\mapsto} Z \overset{2\Ad_U} \mapsto \ds f(A).\dd A\]

The operator \(G\) is simply \(\dd L_{U^T}\ \dd R_V\). The operator \(\pi\) projects orthogonally over the set of skew-symmetric matrices, and the operator \(F\) selects/identifies \(z_1, z_2, z_3\) from the coordinates of \(R_-\).