Matrix: An \(m\times n\) matrix \(A\) is an \(m\)-by-\(n\) array of scalars from a field (for example, real numbers) of the form \[A=\left[\begin{array}{cccc} a_{11}&a_{12}&\cdots &a_{1n}\\ a_{21}&a_{22}&\cdots &a_{2n}\\ \vdots&\vdots&\ddots &\vdots\\ a_{m1}&a_{m2}&\cdots &a_{mn} \end{array}\right].\] The order (or size) of \(A\) is \(m\times n\) (read as m by n) if \(A\) has \(m\) rows and \(n\) columns. The \((i,j)\)-entry of \(A=[a_{i,j}]\) is \(a_{i,j}\).

For example, \(A=\left[\begin{array}{rrr}1&2&0\\-3&0&-1\end{array} \right]\) is a \(2\times 3\) real matrix. The \((2,3)\)-entry of \(A\) is \(-1\).

Useful matrices:

• A zero matrix, denoted by \(O\) or \(O_{m,n}\), is an \(m\times n\) matrix whose all entries are zero.


• A square matrix is a matrix is a matrix whose number of rows and number of columns are the same.


• A diagonal matrix is a square \(n\times n\) matrix whose nondiagonal entries are zero.


• The identity matrix of order \(n\), denoted by \(I_n\), is the \(n\times n\) diagonal matrix whose diagonal entries are 1. For example, \(I_3=\left[\begin{array}{rrr}1&0&0\\0&1&0\\0&0&1 \end{array} \right]\) is the \(3\times 3\) identity matrix.


• An \(n\times 1\) matrix is called a column matrix or an \(n\)-dimensional (column) vector, denoted by lowercase letters such \(x\), \(\boldsymbol x\), or \(\overrightarrow{x}\). For example, \(\overrightarrow{x}=\left[\begin{array}{r}0\\1\\2 \end{array} \right]\) is a \(3\)-dimensional vector which represents the position vector of the point \((0,1,2)\) in the 3-space \(\mathbb R^3\) (i.e., a directed line segment from the origin \((0,0,0)\) to the point \((0,1,2)\)).

Position vector of a point in the 2-space \(\mathbb R^2\)

Matrix operations:

Transpose: The transpose of an \(m\times n\) matrix \(A\), denoted by \(A^T\), is an \(n\times m\) matrix whose columns are corresponding rows of \(A\), i.e., \((A^T)_{ij}=A_{ji}\).
Example. If \(A=\left[\begin{array}{rrr}1&2&0\\-3&0&-1\end{array} \right]\), then \(A^T=\left[\begin{array}{rr}1&-3\\2&0\\0&-1\end{array} \right]\).

Scalar multiplication: Let \(A\) be a matrix and \(c\) be a scalar. The scalar multiple, denoted by \(cA\), is the matrix whose entries are \(c\) times the corresponding entries of \(A\).
Example. If \(A=\left[\begin{array}{rrr}1&2&0\\-3&0&-1\end{array} \right]\), then \(-2A=\left[\begin{array}{rrr}-2&-4&0\\6&0&2\end{array} \right]\).

Sum: If \(A\) and \(B\) are \(m\times n\) matrices, then the sum \(A+B\) is the \(m\times n\) matrix whose entries are the sum of the corresponding entries of \(A\) and \(B\), i.e., \((A+B)_{ij}=A_{ij}+B_{ij}\).
Example. If \(A=\left[\begin{array}{rrr}1&2&0\\-3&0&-1\end{array} \right]\) and \(B=\left[\begin{array}{rrr}0&-2&0\\3&0&2\end{array} \right]\), then \(A+B=\left[\begin{array}{rrr}1&0&0\\0&0&1\end{array} \right]\).

Multiplication:
Matrix-vector multiplication: If \(A\) is an \(m\times n\) matrix and \(\overrightarrow{x}\) is an \(n\)-dimensional vector, then their product \(A\overrightarrow{x}\) is an \(n\)-dimensional vector whose \((i,1)\)-entry is \(a_{i1}x_1+a_{i2}x_2+\cdots+a_{im}x_n\), the dot product of the row \(i\) of \(A\) and \(\overrightarrow{x}\). Note that \[A\overrightarrow{x}=\left[\begin{array}{c} a_{11}x_1+a_{12}x_2+\cdots+a_{1n}x_n\\ a_{21}x_1+a_{22}x_2+\cdots+a_{2n}x_n\\ \vdots\\ a_{m1}x_1+a_{m2}x_2+\cdots+a_{mn}x_n\end{array}\right] = x_1\left[\begin{array}{c} a_{11}\\ a_{21}\\ \vdots\\ a_{m1} \end{array}\right]+ x_2\left[\begin{array}{c} a_{12}\\ a_{22}\\ \vdots\\ a_{m2} \end{array}\right]+\cdots+ x_n\left[\begin{array}{c} a_{1n}\\ a_{2n}\\ \vdots\\ a_{mn} \end{array}\right].\] Example. If \(A=\left[\begin{array}{rrr}1&2&0\\-3&0&-1\end{array} \right]\) and \(\overrightarrow{x}=\left[\begin{array}{r}1\\-1\\0\end{array} \right]\), then \(A\overrightarrow{x}=\left[\begin{array}{r}-1\\-3\end{array} \right]\) which is a linear combination of first and second columns of \(A\) with weights \(1\) and \(-1\) respectively.

Matrix-matrix multiplication: If \(A\) is an \(m\times n\) matrix and \(B\) is an \(n\times p\) matrix, then their product \(AB\) is an \(m\times p\) matrix whose \((i,j)\)-entry is the dot product the row \(i\) of \(A\) and the column \(j\) of \(B\). \[(AB)_{ij}=a_{i1}b_{1j}+a_{i2}b_{2j}+\cdots+a_{im}b_{mj}\]
Example. For \(A=\left[\begin{array}{rrr}1&2&2\\0&0&2\end{array} \right]\) and \(B=\left[\begin{array}{rr}2&-2\\0&0\\1&1\end{array} \right]\), we have \(AB=\left[\begin{array}{rr}4&0\\2&2\end{array} \right].\)

Determinant: The determinant of an \(n\times n\) matrix \(A\) is denoted by \(\det A\) and \(|A|\). It is defined recursively. By hand we will only find determinant of order 2 and 3. \[\left\vert\begin{array}{rr}a_{11}&a_{12}\\a_{21}&a_{22}\end{array} \right\vert =a_{11}a_{22}-a_{12}a_{21}.\] \[\left\vert \begin{array}{rrr}a_{11}&a_{12}&a_{13}\\ a_{21}&a_{22}&a_{23}\\ a_{31}&a_{32}&a_{33}\end{array} \right\vert =a_{11}\;\begin{vmatrix}a_{22}&a_{23}\\a_{32}&a_{33}\end{vmatrix} -a_{12}\;\begin{vmatrix}a_{21}&a_{23}\\a_{31}&a_{33}\end{vmatrix} +a_{13}\;\begin{vmatrix}a_{21}&a_{22}\\a_{31}&a_{32}\end{vmatrix}\;.\]
Example. \(\left\vert\begin{array}{rrr} 2&1&7\\ -3&0&-8\\ 0&1&-3\end{array} \right\vert =2\;\begin{vmatrix}0&-8\\1&-3\end{vmatrix} -1\;\begin{vmatrix}-3&-8\\0&-3\end{vmatrix} +7\;\begin{vmatrix}-3&0\\0&1\end{vmatrix}=-14.\)

Inverse of a matrix: An \(n\times n\) matrix \(A\) is called invertible if there is an \(n\times n\) matrix \(B\) such that \(AB=BA=I_n.\) Here \(B\) is called the inverse of \(A\) which is denoted by \(A^{-1}\). So \[AA^{-1}=A^{-1}A=I_n.\]
Example. \(\left[ \begin{array}{rr}a&b\\c&d\end{array} \right]^{-1}=\displaystyle\frac{1}{ad-bc}\left[ \begin{array}{rr}d&-b\\-c&a\end{array} \right]\).

Theorem. An \(n\times n\) matrix \(A\) is invertible iff \(\det A\neq 0\).

\(A^{-1}\) can be found by the Gauss-Jordan row reductions and adjoint formula: \[A^{-1}=\displaystyle\frac{1}{\det A}\operatorname{adj} A\]

A system of linear equations with \(n\) variables \(x_1,\ldots,x_n\) and \(m\) equations can be written as follows: \[\begin{eqnarray*} \begin{array}{ccccccccc} a_{11}x_1&+&a_{12}x_2&+&\cdots &+&a_{1n}x_n&=&b_1\\ a_{21}x_1&+&a_{22}x_2&+&\cdots &+&a_{2n}x_n&=&b_2\\ \vdots&&\vdots&& &&\vdots&&\vdots\\ a_{m1}x_1&+&a_{m2}x_2&+&\cdots &+&a_{mn}x_n&=&b_m. \end{array} \end{eqnarray*}\] This linear system is equivalent to the matrix equation \(A\overrightarrow{x}=\overrightarrow{b}\), where \[A=\left[\begin{array}{cccc} a_{11}&a_{12}&\cdots &a_{1n}\\ a_{21}&a_{22}&\cdots &a_{2n}\\ \vdots&\vdots& &\vdots\\ a_{m1}&a_{m2}&\cdots &a_{mn} \end{array}\right], \overrightarrow{x}=\left[\begin{array}{c} x_1\\x_2\\ \vdots\\x_n\end{array} \right] \mbox{ and } \overrightarrow{b}=\left[\begin{array}{c} b_1\\b_2\\ \vdots\\b_m \end{array} \right].\] \(A\) is called the coefficient matrix. There are multiple ways to solve the matrix equation \(A\overrightarrow{x}=\overrightarrow{b}\).

Linear independent vectors: \(\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}\) are linearly independent if \[c_1\overrightarrow{v_1}+c_2\overrightarrow{v_2}+\ldots+c_k\overrightarrow{v_k}=\overrightarrow{0}\implies c_1=c_2=\cdots=c_k=0.\] In other words, \(\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}\) are linearly independent if \([\overrightarrow{v_1}\;\overrightarrow{v_2}\;\ldots\;\overrightarrow{v_k}]\overrightarrow{x}=\overrightarrow{0}\) has only the zero solution \(\overrightarrow{x}=\overrightarrow{0}\).

Theorem. Consider \(n\) vectors \(\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_n}\) in \(\mathbb R^n\). Then the following are equivalent.

1. \(\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_n}\) are linearly independent


2. \(\det[\overrightarrow{v_1}\; \overrightarrow{v_2}\;\ldots\;\overrightarrow{v_n}]\neq 0\)


3. any vector \(\overrightarrow{x}\) in \(\mathbb R^n\) is a linear combination of \(\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_n}\),
   i.e., \(\overrightarrow{x}=c_1\overrightarrow{v_1}+c_2\overrightarrow{v_2}+\cdots+c_n\overrightarrow{v_n}\) for some scalar \(c_1,c_2,\ldots,c_n\).

Example. Show that the following vectors are linearly independent: \(\left[\begin{array}{r}2\\-3\\0 \end{array} \right],\; \left[\begin{array}{r}1\\0\\1 \end{array} \right] ,\; \left[\begin{array}{r}7\\-8\\-3 \end{array} \right]\).
Solution. \(\det\left[\begin{array}{rrr} 2&1&7\\ -3&0&-8\\ 0&1&-3\end{array} \right]=-14\neq 0.\) So the vectors are linearly independent. Also note that any 3-dimensional vector can be written as a linear combination of these three vectors.

Span of vectors: The span of \(\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}\), denoted by \(\operatorname{Span}\{\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}\}\), is the set of all linear combinations of \(\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}\).

Example. \(\mathbb R^2=\operatorname{Span}\left\lbrace\left[\begin{array}{r}1\\0 \end{array} \right],\, \left[\begin{array}{r}0\\1 \end{array} \right] \right\rbrace\).

Eigenvalues and eigenvectors: Let \(A\) be an \(n\times n\) matrix. If \(A\overrightarrow{x}=\lambda \overrightarrow{x}\) for some nonzero vector \(\overrightarrow{x}\) and some scalar \(\lambda\), then \(\lambda\) is an eigenvalue of \(A\) and \(\overrightarrow{x}\) is an eigenvector of \(A\) corresponding to \(\lambda\).

Example. Consider \(A=\left[\begin{array}{rr}1&2\\0&3\end{array} \right],\;\lambda=3,\; \overrightarrow{v}=\left[\begin{array}{r}1\\1\end{array} \right],\; \overrightarrow{u}=\left[\begin{array}{r}-2\\1\end{array} \right]\).
Since \(A\overrightarrow{v} =\left[\begin{array}{rr}1&2\\0&3\end{array} \right]\left[\begin{array}{r}1\\1\end{array} \right] =\left[\begin{array}{r}3\\3\end{array} \right] =3\left[\begin{array}{r}1\\1\end{array} \right] =\lambda\overrightarrow{v}\), \(3\) is an eigenvalue of \(A\) and \(\overrightarrow{v}\) is an eigenvector of \(A\) corresponding to the eigenvalue \(3\).
Since \(A\overrightarrow{u} =\left[\begin{array}{rr}1&2\\0&3\end{array} \right]\left[\begin{array}{r}-2\\1\end{array} \right] =\left[\begin{array}{r}0\\3\end{array} \right] \neq \lambda\left[\begin{array}{r}-2\\1\end{array} \right] =\lambda\overrightarrow{u}\) for all scalars \(\lambda\), \(\overrightarrow{u}\) is not an eigenvector of \(A\).

Note that an eigenvalue can be a complex number and an eigenvector can be a complex vector.
Example. Consider \(A=\left[\begin{array}{rr}0&1\\-1&0\end{array} \right]\). Since \(\left[\begin{array}{rr}0&1\\-1&0\end{array} \right]\left[\begin{array}{r}1\\i\end{array} \right] =\left[\begin{array}{r}i\\-1\end{array} \right] =i\left[\begin{array}{r}1\\i\end{array} \right]\),
\(i\) is an eigenvalue of \(A\) and \(\left[\begin{array}{r}1\\i\end{array} \right]\) is an eigenvector of \(A\) corresponding to the eigenvalue \(I\).

The characteristic polynomial of \(A\) is \(\det(A-\lambda I)\), a polynomial of \(\lambda\).
The characteristic equation of \(A\) is \(\det(A-\lambda I)=0\).
Since the roots of the characteristic polynomial are the eigenvalues of the \(n\times n\) matrix \(A\), \(A\) has \(n\) eigenvalues, not necessarily distinct. The multiplicity of a root \(\lambda\) in \(\det(A-\lambda I)\) is the algebraic multiplicity of the eigenvalue \(\lambda\) of \(A\).

Suppose \(\lambda\) is an eigenvalue of the matrix \(A\). Then \[\operatorname{NS} (A-\lambda I)=\{\overrightarrow{x}\;|\;(A-\lambda I)\overrightarrow{x}=\overrightarrow{O}\}\] is the eigenspace of \(A\) corresponding to the eigenvalue \(\lambda\).

Example. Let \(A=\left[\begin{array}{rr}1&2\\0&3\end{array} \right]\).

1. Find the eigenvalues of \(A\) with their algebraic multiplicities.


2. Find the eigenvectors and eigenspaces of \(A\).

Solution. (a) The characteristic polynomial of \(A\) is \[\det(A-\lambda I)=\left|\begin{array}{cc}1-\lambda& 2\\0&3-\lambda \end{array} \right|=(1-\lambda)(3-\lambda).\] \(\det(\lambda I-A)=(1-\lambda)(3-\lambda)=0\implies \lambda=1,3\).\\ So \(1\) and \(3\) are eigenvalues of \(A\) with algebraic multiplicities \(1\) and \(1\) respectively.

(b) The eigenspace of \(A\) corresponding to the eigenvalue \(1\) is \[\operatorname{NS} (A-1I)=\{\overrightarrow{x}\;|\;(A-I)\overrightarrow{x}=\overrightarrow{O}\}.\] \[(A-1I)\overrightarrow{x}=\overrightarrow{O}\implies\left[\begin{array}{rr}0&2\\0&2\end{array}\right]\left[\begin{array}{r}x_1\\x_2\end{array}\right]=\left[\begin{array}{r}0\\0\end{array}\right]\implies\left\lbrace \begin{array}{rrl} &2x_2&=0\\ &2x_2&=0 \end{array}\right.\] So we get \(x_2=0\) where \(x_1\) is a free variable. Thus \[\overrightarrow{x}=\left[\begin{array}{r}x_1\\x_2 \end{array}\right]=\left[\begin{array}{c}x_1\\0 \end{array}\right] =x_1\left[\begin{array}{c}1\\0 \end{array}\right] \in \operatorname{Span} \left\{\left[\begin{array}{c}1\\0 \end{array}\right]\right\}.\] Thus the eigenvector of \(A\) corresponding to the eigenvalue \(1\) is \(\left[\begin{array}{r}1\\0 \end{array}\right]\) and the eigenspace of \(A\) corresponding to the eigenvalue \(1\) is \[\operatorname{NS} (A-1I)=\operatorname{Span} \left\{\left[\begin{array}{r}1\\0 \end{array}\right]\right\}.\] The eigenspace of \(A\) corresponding to the eigenvalue \(3\) is \[\operatorname{NS} (A-3I)=\{\overrightarrow{x}\;|\;(A-3I)\overrightarrow{x}=\overrightarrow{O}\}.\] \[(A-3I)\overrightarrow{x}=\overrightarrow{O}\implies\left[\begin{array}{rr}-2&2\\0&0\end{array}\right]\left[\begin{array}{r}x_1\\x_2\end{array}\right]=\left[\begin{array}{r}0\\0\end{array}\right]\implies\left\lbrace \begin{array}{rrl} -2x_1&+2x_2&=0 \end{array}\right.\] So we get \(x_1=x_2\) where \(x_2\) is a free variable. Thus \[\overrightarrow{x}=\left[\begin{array}{r}x_1\\x_2 \end{array}\right]=\left[\begin{array}{r}x_2\\x_2 \end{array}\right] =x_2\left[\begin{array}{c}1\\1 \end{array}\right] \in \operatorname{Span} \left\{\left[\begin{array}{c}1\\1 \end{array}\right]\right\}.\] Thus the eigenvector of \(A\) corresponding to the eigenvalue \(3\) is \(\left[\begin{array}{r}1\\1 \end{array}\right]\) and the eigenspace of \(A\) corresponding to the eigenvalue \(3\) is \[\operatorname{NS} (A-3I)=\operatorname{Span} \left\{\left[\begin{array}{r}1\\1 \end{array}\right]\right\}.\]

Matrix of functions: Entries of an \(m\times n\) matrix \(A\) may be functions of \(x\): \[A(x)=\left[\begin{array}{cccc} a_{11}(x)&a_{12}(x)&\cdots &a_{1n}(x)\\ a_{21}(x)&a_{22}(x)&\cdots &a_{2n}(x)\\ \vdots&\vdots& &\vdots\\ a_{m1}(x)&a_{m2}(x)&\cdots &a_{mn}(x) \end{array}\right].\] Calculus on matrix of functions: We take limit, derivative, and integration of a matrix \(A(x)\) entry-wise: \[\begin{align*} \lim_{x\to a}A(x)&=\left[\begin{array}{cccc} \displaystyle\lim_{x\to a}a_{11}(x)&\displaystyle\lim_{x\to a}a_{12}(x)&\cdots &\displaystyle\lim_{x\to a}a_{1n}(x)\\ \displaystyle\lim_{x\to a}a_{21}(x)&\displaystyle\lim_{x\to a}a_{22}(x)&\cdots &\displaystyle\lim_{x\to a}a_{2n}(x)\\ \vdots&\vdots& &\vdots\\ \displaystyle\lim_{x\to a}a_{m1}(x)&\displaystyle\lim_{x\to a}a_{m2}(x)&\cdots &\displaystyle\lim_{x\to a}a_{mn}(x) \end{array}\right],\\ A'(x)&=\left[\begin{array}{cccc} a_{11}'(x)&a_{12}'(x)&\cdots &a_{1n}'(x)\\ a_{21}'(x)&a_{22}'(x)&\cdots &a_{2n}'(x)\\ \vdots&\vdots& &\vdots\\ a_{m1}'(x)&a_{m2}'(x)&\cdots &a_{mn}'(x) \end{array}\right],\\ \text{and }\int A(x)\, dx&=\left[\begin{array}{cccc} \int a_{11}(x)\, dx &\int a_{12}(x)\, dx &\cdots &\int a_{1n}(x)\, dx\\ \int a_{21}(x)\, dx &\int a_{22}(x)\, dx&\cdots &\int a_{2n}(x)\, dx\\ \vdots&\vdots& &\vdots\\ \int a_{m1}(x)\, dx &\int a_{m2}(x)\, dx &\cdots &\int a_{mn}(x)\, dx \end{array}\right]. \end{align*}\]
Example. Consider \(A(x)=\left[\begin{array}{cc}e^x&\cos x\\\sin x&2e^{2x}\end{array} \right]\). Then \(\displaystyle\lim_{x\to 0}A(x)=\left[\begin{array}{cc}1&1\\0&2\end{array} \right]\), \(A'(x)=\left[\begin{array}{cc}e^x&-\sin x\\ \cos x& 4e^{2x}\end{array} \right]\), and \(\int A(x)\, dx=\left[\begin{array}{cc}e^x&\sin x\\-\cos x&e^{2x}\end{array} \right].\)

Last edited