Appreciation of the Identity $ ( ( A )) = ( Tr ( A ))$

Matrix Exponential

Of course, to appreciate this identity, some preparation is required. First, how should we understand \exp(\boldsymbol{A})? Generally, it is defined according to the standard Taylor series expansion of e^x: \exp(\boldsymbol{A})=\sum_{n=0}^{\infty}\frac{\boldsymbol{A}^n}{n!} Here \boldsymbol{A} is a k\times k matrix. It can be proven that this definition converges for any matrix, making it a well-defined concept.

With this definition, we can directly write the solution to a system of linear differential equations with constant coefficients: \frac{d}{dt}\boldsymbol{x}=\boldsymbol{A}\boldsymbol{x}\quad\Rightarrow \quad \boldsymbol{x}=\exp(t\boldsymbol{A})\boldsymbol{x}_0 Of course, this result is primarily of theoretical value because, in actual calculations, you would still laboriously have to compute \boldsymbol{A}^2, \boldsymbol{A}^3, \dots one by one.

Is there a simpler calculation scheme? Yes, it becomes simpler if \boldsymbol{A} is diagonalizable. Diagonalizability means: \boldsymbol{A}=\boldsymbol{P}\boldsymbol{\Lambda}\boldsymbol{P}^{-1} where \boldsymbol{P} is an invertible matrix and \boldsymbol{\Lambda}=\text{diag}(\lambda_1,\dots,\lambda_k) is a diagonal matrix. This case is simple because: \boldsymbol{A}^n=\boldsymbol{P}\boldsymbol{\Lambda}^n\boldsymbol{P}^{-1} Since \boldsymbol{\Lambda} is a diagonal matrix, \boldsymbol{\Lambda}^n is obtained simply by raising each diagonal element to the n-th power. Thus: \begin{aligned}\exp(\boldsymbol{A})=&\exp(\boldsymbol{P}\boldsymbol{\Lambda}\boldsymbol{P}^{-1})\\ =&\boldsymbol{P}\left(\sum_{n=0}^{\infty}\frac{\boldsymbol{\Lambda}^n}{n!}\right)\boldsymbol{P}^{-1}\\ =&\boldsymbol{P}\exp(\boldsymbol{\Lambda})\boldsymbol{P}^{-1} \end{aligned} Here \exp(\boldsymbol{\Lambda})=\text{diag}\left(e^{\lambda_1},\dots,e^{\lambda_k}\right).

However, it should be noted that although the definition of the matrix exponential mimics the exponential series of real numbers, for any two matrices \boldsymbol{A} and \boldsymbol{B}, in general: \exp(\boldsymbol{A}+\boldsymbol{B})\neq \exp(\boldsymbol{A})\exp(\boldsymbol{B}) A sufficient condition for equality is \boldsymbol{A}\boldsymbol{B}=\boldsymbol{B}\boldsymbol{A}, i.e., the multiplication is commutative. In other words, when applying operational laws of real numbers to multiple matrices, a commutativity condition is often required.

Since \boldsymbol{A} and -\boldsymbol{A} obviously commute, we have: \boldsymbol{I}=\exp(\boldsymbol{A}-\boldsymbol{A})=\exp(\boldsymbol{A})\exp(-\boldsymbol{A}) This means that \exp(\boldsymbol{A}) is always invertible, and its inverse is \exp(-\boldsymbol{A}).

Matrix Functions

In fact, through power series, many real-valued series can be extended to matrices, thereby becoming matrix functions. For example: \begin{aligned}\sin(\boldsymbol{A})=&\boldsymbol{A}-\frac{\boldsymbol{A}^3}{3!}+\frac{\boldsymbol{A}^5}{5!}-\frac{\boldsymbol{A}^7}{7!}+\dots\\ \cos(\boldsymbol{A})=&\boldsymbol{I}-\frac{\boldsymbol{A}^2}{2!}+\frac{\boldsymbol{A}^4}{4!}-\frac{\boldsymbol{A}^6}{6!}+\dots \end{aligned} Similarly, if \boldsymbol{A}\boldsymbol{B}=\boldsymbol{B}\boldsymbol{A}, then: \sin(\boldsymbol{A}+\boldsymbol{B})=\sin(\boldsymbol{A})\cos(\boldsymbol{B})+\sin(\boldsymbol{B})\cos(\boldsymbol{A})

Having discussed the exponential function, it is natural to consider the logarithm. There are generally two definitions for the matrix logarithm. The first definition is: if a matrix \boldsymbol{B} satisfies \exp(\boldsymbol{B})=\boldsymbol{A}, then \boldsymbol{B} is called a logarithm of matrix \boldsymbol{A}.

However, under this definition, the matrix logarithm is not unique, even when restricted to the domain of real matrices. For example, for \boldsymbol{A}=\begin{pmatrix}\cos\alpha & -\sin\alpha \\ \sin\alpha & \cos\alpha\end{pmatrix}, any matrix of the form (\alpha + 2\pi n)\begin{pmatrix}0 & -1 \\ 1 & 0\end{pmatrix} is its logarithm, where n is any integer.

The second definition mimics the Taylor series expansion for the real logarithm: \ln (\boldsymbol{I}+\boldsymbol{A}) = \sum_{n=1}^{\infty}(-1)^{n-1}\frac{\boldsymbol{A}^n}{n} This definition is straightforward and yields a unique result, but the convergence condition is restricted to \|\boldsymbol{A}\|_2 < 1, where \|\cdot\|_2 is the spectral norm of the matrix (refer to the section on "Matrix Norms" in "Lipschitz Constraints in Deep Learning"). When the constraint is satisfied, the logarithm defined this way satisfies: \exp(\ln (\boldsymbol{I}+\boldsymbol{A})) = \boldsymbol{I}+\boldsymbol{A} In other words, the definition is self-consistent.

The diagonalization technique discussed earlier for the matrix exponential also applies to any matrix function defined via a power series, such as: \ln \left(\boldsymbol{I}+\boldsymbol{P}\boldsymbol{\Lambda}\boldsymbol{P}^{-1}\right) = \boldsymbol{P}\ln(\boldsymbol{I}+\boldsymbol{\Lambda})\boldsymbol{P}^{-1} Here \ln(\boldsymbol{I}+\boldsymbol{\Lambda})=\text{diag}\big(\ln(1+\lambda_1),\dots,\ln(1+\lambda_k)\big).

\det(\exp(\boldsymbol{A})) = \exp(\text{Tr}(\boldsymbol{A}))

After all this preparation, we can finally enter the main theme. For the identity [eq:main], if the matrix is diagonalizable, the proof is not difficult. Because: \begin{aligned}\text{LHS}=&\det(\exp(\boldsymbol{A}))\\ =&\det\big(\boldsymbol{P}\exp(\boldsymbol{\Lambda})\boldsymbol{P}^{-1}\big)\\ =&\det(\boldsymbol{P}) \det(\exp(\boldsymbol{\Lambda})) \det(\boldsymbol{P}^{-1})\\ =&\det(\exp(\boldsymbol{\Lambda}))\\ =&e^{\lambda_1 + \dots + \lambda_k} \end{aligned} And: \begin{aligned}\text{RHS}=&\exp(\text{Tr}(\boldsymbol{A}))\\ =&\exp(\text{Tr}(\boldsymbol{P}\boldsymbol{\Lambda}\boldsymbol{P}^{-1}))\\ =&\exp(\text{Tr}(\boldsymbol{P}^{-1}\boldsymbol{P}\boldsymbol{\Lambda}))\quad [\text{The order is swapped because for square matrices } \boldsymbol{A}, \boldsymbol{B}, \text{Tr}(\boldsymbol{A}\boldsymbol{B})=\text{Tr}(\boldsymbol{B}\boldsymbol{A})]\\ =&\exp(\text{Tr}(\boldsymbol{\Lambda}))\\ =&e^{\lambda_1 + \dots + \lambda_k} \end{aligned}

What if the matrix is not diagonalizable? If one can prove that the set of diagonalizable matrices is dense in the set of all matrices, then the remaining part can be supplemented using limits, but this is clearly quite tedious. Is there a more elegant, seamless proof? Yes! And it is very ingenious, which is the primary reason I am writing this article.

This ingenious proof requires us to consider a function with a parameter t: f(t)=\det(\exp(t\boldsymbol{A}))\label{eq:detexp} Then we find its derivative (this involves the derivative of a determinant; for the method, please refer to Derivative of Determinants): \begin{aligned}\frac{d}{dt}f(t)=&f(t)\text{Tr}\left(\exp(-t\boldsymbol{A})\frac{d}{dt}\exp(t\boldsymbol{A})\right)\\ =&f(t)\text{Tr}\left(\exp(-t\boldsymbol{A})\exp(t\boldsymbol{A})\boldsymbol{A}\right)\\ =&f(t)\text{Tr}(\boldsymbol{A})\end{aligned} Note that \text{Tr}(\boldsymbol{A}) is just a number, so we have obtained an ordinary differential equation for f(t)! Its solution is: f(t)=C\exp(t\,\text{Tr}(\boldsymbol{A})) From equation [eq:detexp], we can see that f(0)=\det(\boldsymbol{I})=1, which implies C=1. Thus f(t)=\exp(t\,\text{Tr}(\boldsymbol{A})), and we have proven: \det(\exp(t\boldsymbol{A}))=\exp(t\,\text{Tr}(\boldsymbol{A})) Setting t=1, we obtain the identity [eq:main].

Isn’t it wonderful?

By taking the logarithm of both sides of [eq:main] and letting \exp(\boldsymbol{A})=\boldsymbol{B}, we can obtain another common form of the identity: \ln\det(\boldsymbol{B}) = \text{Tr}(\ln (\boldsymbol{B})) That concludes our appreciation for this time. Thank you for reading.

References:
https://math.stackexchange.com/questions/1487773/the-identity-deta-exptrlna-for-a-general

Original article address: https://kexue.fm/archives/6377

For more details on reprinting, please refer to: Scientific Space FAQ