Previously, I promised to write some of my own understandings of matrices. In fact, Meng Yan, in his three articles titled "Understanding Matrices," has already used a very intuitive method to tell us about some properties and ideas regarding matrices and linear algebra. Most of my understanding of matrices also originates from his articles. Of course, to better understand linear algebra, I have also read many related books in search of an intuitive way of understanding. Meng Yan’s blog has not been updated for a long time; here, I would like to borrow his title to narrate my own understanding of matrices.
Of course, I do not intend to pursue highly abstract issues such as spaces and operators. I simply want to express my views on some commonly used tools in linear algebra, such as matrices and determinants. At the same time, the article is named "Understanding Matrices," which means this is not an introductory tutorial on matrices, but rather a discussion with readers who already have a certain foundation in linear algebra about other ways to understand matrices, and nothing more. I estimate that basically any reader who has studied linear algebra will be able to understand this article.
First, we cannot help but trace back to a fundamental question: What is a matrix?
Let us recall how matrices are generated. A matrix can be seen as an ordered combination of vectors, which suggests that a matrix can be compared to a vector. But how are vectors generated? A vector is an ordered combination of numbers, which points our research direction toward the question: "What is a number?" For example, what is the number 1? It can represent 1 meter, 1 kilogram, 1 minute, 1 degree Celsius, or even 1 apple. Why does it have so many representational meanings? The answer is simple: because in essence, it is nothing; it is just the number 1, a symbol, an abstract concept. Precisely because it is abstract, it can be endowed with various intuitive meanings! Returning to the matrix itself, we realize that the reason the matrix is so powerful is because of that very dry definition in textbooks—a matrix is a table of numbers with m rows and n columns! It abstracts the matrix, allowing it to "evolve." It is a more generalized concept: a vector can be seen as a matrix, and even a single number can be seen as a matrix, and so on.
Understanding from an Algebraic Perspective
Of course, the above statement is vague. We still need to know exactly what it is used for. This can be analyzed from algebraic and geometric perspectives, as achieving the integration of numbers and shapes is most perfect. First, we know that the most basic elements of mathematics are numbers, strictly speaking, natural numbers such as 0, 1, 2, …; with numbers, we can do many things. However, numbers are individual, and we often need to process similar operations in batches, such as calculating the four expressions 1+2, 1+3, 2+3, 4+5 simultaneously. Whether from the perspective of recording or research, studying them separately is quite tedious. Thus, a "batch" notation was created. We denote it as (1,1,2,4)+(2,3,3,5), using two different symbols to record them, such as \vec{A}=(1,1,2,4), \vec{B}=(2,3,3,5), and we can denote it as \vec{A}+\vec{B}. This provides convenience in both research and recording. Thus, something we call a "vector" was born. That is to say, from an algebraic perspective, vectors were created to study batch operations. However, vectors did not solve all problems of batch operations. For example, consider a system of linear equations with three variables: \begin{aligned} a_{11}x_{1}+a_{12}x_{2}+a_{13}x_{3}=b_{1}\\ a_{21}x_{1}+a_{22}x_{2}+a_{23}x_{3}=b_{2}\\ a_{31}x_{1}+a_{32}x_{2}+a_{33}x_{3}=b_{3} \end{aligned}
Using vectors alone, we still cannot study this type of problem well. Therefore, we must find a way to create new notations. Since the coefficients on the left have a certain arrangement order and a unified form, we might as well write them out separately: \begin{pmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{pmatrix} and represent it with a simple symbol A. Then, we write the unknowns and the constants on the right as vector forms x=[x_{1},x_{2},x_{3}]^{\top} and b=[b_{1},b_{2},b_{3}]^{\top} (adding the superscript \top to denote a column vector). We expect the above system of equations to be written in a simple form: Ax=b
From this, we can define the multiplication of a 3rd-order square matrix by a 3-dimensional column vector. This is a pure definition, a definition made for the convenience of our recording and research. On this basis, we can study more things, such as what happens when a matrix is multiplied by a matrix?
Similarly, the matrices studied here refer to n-th order square matrices, which are the core. We must first study the core issues thoroughly; otherwise, if we consider all complex situations from the beginning, it is easy to fall into confusion and not know what to do.
Before studying general multiplication, let us understand the issue of operational laws. We know that in real numbers, addition satisfies the associative and commutative laws, and multiplication satisfies the associative, commutative, and distributive laws. Which laws can be migrated to matrix multiplication? The commutative law cannot be assumed a priori; it is a matter of definition. I could define it to hold or not to hold, but for the convenience of calculation, we still hope it satisfies more operational laws. Therefore, we first consider the associative law, hoping it can satisfy this law. That is: (AB)x=A(Bx) where Bx is already an operation we are familiar with (by definition), which will result in a column vector, so we can also easily calculate A(Bx). Intuitively, AB should also be an n-th order square matrix. We can first set it up, then operate it with the column vector x, and finally correspond the results on both sides one by one to obtain the expressions for each element in the n-th order square matrix AB. We eventually find that it is exactly the expression defined in our textbooks.
Taking a 2nd-order square matrix as an example, let A= \begin{pmatrix} a & b\\ c & d \end{pmatrix},\quad B=\begin{pmatrix} e & f\\ g & h \end{pmatrix},\quad AB=\begin{pmatrix} p & q\\ r & s \end{pmatrix} and let x=[x,y]^{\top}, then (AB)x is equal to [px+qy,rx+sy]^{\top} And since Bx=[ex+fy,gx+hy]^{\top}, then A(Bx)=[aex+afy+bgx+bhy,cex+cfy+dgx+dhy]^{\top}.
Then, according to the correspondence of each element, we get p=ae+bg, q=af+bh, r=ce+dg, s=cf+dh. This completes the definition of 2nd-order square matrix multiplication.
Now we can say from an algebraic perspective that the matrix is an "ultimate weapon" for simplifying batch linear operations! This is a relatively intuitive and useful algebraic meaning of the matrix.
If we consider the commutative law according to this definition, we will find that matrices generally do not obey the commutative law. This is undoubtedly a regret. But it doesn’t matter; the fact that it obeys the associative law has already endowed this tool with great power. For example, for the linear system Ax=y, we have By=B(Ax)=(BA)x. If we find a way to find a matrix B such that BA=I, then it would be wonderful, because I would only need to apply matrix B to vector y to get the solution to the system. In fact, such a matrix B exists; this is the inverse matrix. Without the associative law, none of this would be possible!
Since this is a "batch version" of basic real number operations (linear operations), we can naturally extend some real number formulas into matrix versions (as long as the commutative law is not involved). For example, in real numbers, we have the formula: \frac{1}{1-x}=1+x+x^{2}+x^{3}+\dots\approx 1+x
Then, when we find the inverse of a matrix, we also have a similar formula: (I-A)^{-1}=I+A+A^{2}+A^{3}+\dots\approx I+A where I is the identity matrix and A is a "relatively small" matrix. As for how "relatively small" is defined, it is not yet clear, but it can be thought of as the determinant of the matrix being small. Similarly, according to \sqrt{1+x}\approx 1+\frac{1}{2}x we can correspondingly give (I+\frac{1}{2}A)^{2}\approx I+A, which is an approximate formula for finding the "square root" of a matrix.
Thanks to the matrix multiplication we defined, batch operations can be performed directly using the calculation formulas for single quantities, without us having to painstakingly rack our brains to conceive new formulas. This is where the power of the matrix lies! It possesses a miraculous beauty in solving many linear problems. The simplest example is that the solution to the linear system Ax=y is x=A^{-1}y. Solving the system of equations is just like solving a univariate equation with a simple form! There are also some definitions regarding exponents, etc., which will be introduced when applied in the future. They are all like exquisite "works of art"!
In the next installment, we will understand matrices from a geometric perspective. Of course, most of the content therein has already been mentioned in Meng Yan’s articles; I am merely revisiting old theories, and I hope readers will not find it tedious.
When reprinting, please include the original address of this article: https://kexue.fm/archives/1765
For more detailed reprinting matters, please refer to: "Scientific Space FAQ"