Definition. A linear combination of vectors \(\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}\) of \(\mathbb{R}^n\) is a sum of their scalar multiples, i.e., \[c_1\overrightarrow{v_1}+c_2\overrightarrow{v_2}+\cdots+c_k\overrightarrow{v_k}\] for some scalars \(c_1,c_2,\ldots,c_k\) (real numbers throughout this chapter). The set of all linear combinations of a nonempty set \(S\) of vectors of \(\mathbb R^n\) is called the linear span or span of \(S\), denoted by \(\operatorname{Span}(S)\) or \(\operatorname{Span} S\), i.e., \[\operatorname{Span}\{\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}\} = \{c_1\overrightarrow{v_1}+c_2\overrightarrow{v_2}+\cdots+c_k\overrightarrow{v_k}\;|\; c_1,c_2,\ldots,c_k\in \mathbb R\}.\] We define \(\operatorname{Span} \varnothing=\{\overrightarrow{0}\}\). When \(\operatorname{Span}\{\overrightarrow{v_1},\ldots,\overrightarrow{v_k}\} =\mathbb R^n\), we say \(\{\overrightarrow{v_1},\ldots,\overrightarrow{v_k}\}\) spans \(\mathbb{R}^n\).

Example. For \(S=\left\lbrace \left[\begin{array}{r} 1\\ 1\\0 \end{array} \right],\;\left[\begin{array}{r} 1\\2\\0 \end{array} \right] \right\rbrace\), \[\operatorname{Span}(S) =\left\lbrace c_1 \left[\begin{array}{r}1\\ 1\\ 0 \end{array} \right] +c_2 \left[\begin{array}{r}1\\2\\0 \end{array} \right] \; |\; c_1,c_2 \in \mathbb R \right\rbrace. \] Note that \([0,0,1]^T\) is not in \(\operatorname{Span}(S)\) because there are no \(c_1,c_2\) for which \[\left[\begin{array}{r}0\\0\\1 \end{array}\right] =c_1 \left[\begin{array}{r}1\\ 1\\0 \end{array}\right] +c_2 \left[\begin{array}{r}1\\2\\0 \end{array}\right].\] Thus \(S\) does not span \(\mathbb R^3\). But any vector of the form \([a,b,0]^T\) is in \(\operatorname{Span}(S)\) because \[x_1 \left[\begin{array}{r}1\\ 1\\0 \end{array}\right] +x_2 \left[\begin{array}{r}1\\2\\0 \end{array}\right] =\left[\begin{array}{r}a\\b\\0 \end{array}\right] \implies x_1= 2a-b, x_2=-a+b .\] \[\text{i.e., } \left[\begin{array}{r}a\\b\\0 \end{array}\right] =(2a-b) \left[\begin{array}{r}1\\ 1\\0 \end{array}\right] +(-a+b) \left[\begin{array}{r}1\\2\\0 \end{array}\right]\in \operatorname{Span}(S).\] Thus \(S\) spans the following set \[\operatorname{Span}(S) =\left\lbrace \left[\begin{array}{r}a\\ b\\ 0 \end{array} \right] \; |\; a,b \in \mathbb R \right\rbrace,\] which is the \(xy\)-plane in \(\mathbb R^3\).

Definition. A subspace of \(\mathbb{R}^n\) is a nonempty subset \(S\) of \(\mathbb{R}^n\) that satisfies three properties:

1. \(\overrightarrow{0}\) is in \(S\).


2. \(\overrightarrow{u}+\overrightarrow{v}\) is in \(S\) for all \(\overrightarrow{u},\; \overrightarrow{v}\) in \(S\).


3. \(c \overrightarrow{u}\) is in \(S\) for all \(\overrightarrow{u}\) in \(S\) and all scalars \(c\).

In short, a subspace of \(\mathbb R^n\) is a nonempty subset \(S\) of \(\mathbb R^n\) that is closed under linear combination of vectors, i.e., \(c\overrightarrow{u}+d\overrightarrow{v}\) is in \(S\) for all \(\overrightarrow{u},\; \overrightarrow{v}\) in \(S\) and all scalars \(c,d\). When \(S\) is a subspace of \(\mathbb R^n\), we sometimes denote it by \(S\leq \mathbb R^n\).

Example.

1. \(\{\overrightarrow{0}\},\mathbb R^n \leq \mathbb R^n\), i.e., \(\{\overrightarrow{0}\}\) and \(\mathbb R^n\) are subspaces of \(\mathbb R^n\).


2. Show that \(S=\left\lbrace \left[\begin{array}{r} x\\y \end{array} \right] \;|\; x,y\in \mathbb R,\; 2x-y=0\right\rbrace\) is a subspace of \(\mathbb R^2\).
   Solution.
   1. \(\left[\begin{array}{r} 0\\0 \end{array} \right] \in S\) because \(2\cdot 0-0=0\).
   
   
   2. Let \(\overrightarrow{u},\overrightarrow{v} \in S\) and \(c\in \mathbb R\). Then \[\overrightarrow{u}=\left[\begin{array}{r} x_1\\y_1 \end{array} \right] \text{ and } \overrightarrow{v}= \left[\begin{array}{r} x_2\\y_2 \end{array} \right],\] for some \(x_1,x_2,y_1,y_2 \in \mathbb R\) such that \(2x_1-y_1=0\) and \(2x_2-y_2=0\). Then \[\overrightarrow{u}+\overrightarrow{v} =\left[\begin{array}{r} x_1\\y_1 \end{array} \right] +\left[\begin{array}{r} x_2\\y_2 \end{array} \right] = \left[\begin{array}{r} x_1+x_2\\y_1+y_2 \end{array} \right] \in S\] because \(2(x_1+x_2)-(y_1+y_2)=(2x_1-y_1)+(2x_2-y_2)=0\).
   
   
   3. \[c\overrightarrow{u}=c \left[\begin{array}{r} x_1\\y_1 \end{array} \right]=\left[\begin{array}{r} cx_1\\ cy_1 \end{array} \right] \in S\] because \(2(cx_1)-(cy_1)=c(2x_1-y_1)=0\).
   Thus \(S\) (which is the line \(y=2x\)) is a subspace of \(\mathbb R^2\).


3. Let \(S=\left\lbrace \left[\begin{array}{r} 1\\1\\0 \end{array} \right],\;\left[\begin{array}{r} 1\\2\\0 \end{array} \right] \right\rbrace\). Then \(\operatorname{Span}(S)\) is a subspace of \(\mathbb R^3\).
   First note that \(\left[\begin{array}{r} 0\\0\\0 \end{array} \right]=0\left[\begin{array}{r} 1\\1\\0 \end{array} \right] +0\left[\begin{array}{r} 1\\2\\0 \end{array} \right] \in \operatorname{Span}(S)\). Thus \(\operatorname{Span}(S)\neq \varnothing\). Let \(\overrightarrow{u},\overrightarrow{v} \in\operatorname{Span}(S)\) and \(c,d\in \mathbb R\). Then \[\overrightarrow{u}=c_1 \left[\begin{array}{r}1\\ 1\\0 \end{array}\right] +c_2 \left[\begin{array}{r}1\\2\\0 \end{array}\right] \text{ and } \overrightarrow{v}= d_1 \left[\begin{array}{r}1\\ 1\\0 \end{array}\right] +d_2 \left[\begin{array}{r}1\\2\\0 \end{array}\right],\] for some \(c_1,c_2,d_1,d_2 \in \mathbb R\). Then \[\begin{eqnarray*} c\overrightarrow{u}+d\overrightarrow{v} &=&c\left(c_1 \left[\begin{array}{r}1\\ 1\\0 \end{array}\right] +c_2 \left[\begin{array}{r}1\\2\\0 \end{array}\right] \right) +d\left(d_1 \left[\begin{array}{r}1\\ 1\\0 \end{array}\right] +d_2 \left[\begin{array}{r}1\\2\\0 \end{array}\right] \right)\\ &=& (cc_1+dd_1) \left[\begin{array}{r}1\\ 1\\0 \end{array}\right] +(cc_2+dd_2) \left[\begin{array}{r}1\\2\\0 \end{array}\right] \in \operatorname{Span}(S). \end{eqnarray*}\] Thus \(\operatorname{Span}(S)\) (which is the \(xy\)-plane) is a subspace of \(\mathbb R^3\).

Theorem. Let \(\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}\in \mathbb R^n\). Then \(\operatorname{Span}\{\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}\}\) is a subspace of \(\mathbb R^n\).

For a given matrix we have two important subspaces: the column space and the null space.

Definition. The column space of an \(m\times n\) matrix \(A=[\overrightarrow{a_1}\:\overrightarrow{a_2}\:\cdots\overrightarrow{a_n}]\), denoted by \(\operatorname{CS}\left(A\right)\) or \(\operatorname{Col}\left(A\right)\), is the span of its column vectors: \[\operatorname{CS}\left(A\right)=\operatorname{Span}\{\overrightarrow{a_1},\overrightarrow{a_2},\ldots,\overrightarrow{a_n}\}.\]
Remark. Since each column is an \(m\) dimensional vector, \(\operatorname{CS}\left(A\right)\) is a subspace of \(\mathbb R^m\).

Example. For \(A=\left[\begin{array}{rrr}1&2&3\\0&4&5\end{array} \right]\), \(\operatorname{CS}\left(A\right)=\operatorname{Span}\left\lbrace \left[\begin{array}{r}1\\0 \end{array} \right], \left[\begin{array}{r}2\\4 \end{array} \right], \left[\begin{array}{r}3\\5 \end{array} \right] \right\rbrace \leq \mathbb R^2\).

Example. Let \(A=\left[\begin{array}{rrrr}1&-3&-4\\-4&6&-2\\-3&7&6\end{array} \right]\) and \(\overrightarrow{b}= \left[\begin{array}{r}3\\3\\-4 \end{array} \right]\). Determine if \(\overrightarrow{b}\) is in \(\operatorname{CS}\left(A\right)\).
Solution. Note that \(\overrightarrow{b}\in \operatorname{CS}\left(A\right)\) if and only if \(\overrightarrow{b}\) is a linear combination of columns of \(A\) if and only if \(A\overrightarrow{x}=\overrightarrow{b}\) has a solution. \[\left[\begin{array}{rrr|r}1&-3&-4&3\\-4&6&-2&3\\-3&7&6&-4\end{array} \right] \xrightarrow[3R_1+R_3]{4R_1+R_2} \left[\begin{array}{rrr|r}1&-3&-4&3\\0&-6&-18&15\\0&-2&-6&5\end{array} \right] \xrightarrow{-\frac{1}{3}R_2+R_3} \left[\begin{array}{rrr|r}\boxed{1}&-3&-4&3\\0&\boxed{-6}&-18&15\\0&0&0&0\end{array} \right] (REF)\] Since the REF of \([A\:\overrightarrow{b}]\) has no row of the form \([0,0, 0, c], c\neq 0\), \(A\overrightarrow{x}=\overrightarrow{b}\) is consistent and consequently \(\overrightarrow{b}\) is in \(\operatorname{CS}\left(A\right)\).

Theorem. An \(m\times n\) matrix \(A\) has a pivot position in every row if and only if \(A\overrightarrow{x}=\overrightarrow{b}\) is consistent for any \(\overrightarrow{b}\in \mathbb R^m\) if and only if \(\operatorname{CS}\left(A\right)=\mathbb R^m\).

Example. Since \(A=\left[\begin{array}{rrr}\boxed{1}&2&3\\0&\boxed{4}&5\end{array} \right]\) has a pivot position in each row, \(\operatorname{CS}\left(A\right)=\mathbb R^2\).

Definition. The null space of an \(m\times n\) matrix \(A\), denoted by \(\operatorname{NS}\left(A\right)\) or \(\operatorname{Nul}\left(A\right)\), is the solution set of \(A\overrightarrow{x}=\overrightarrow{0}\): \[\operatorname{NS}\left(A\right)=\{ \overrightarrow{x}\in \mathbb R^n \;|\; A\overrightarrow{x}=\overrightarrow{0}\}.\]

Theorem. Let \(A\) be an \(m\times n\) matrix. Then \(\operatorname{NS}\left(A\right)\) is a subspace of \(\mathbb R^n\).

Example. Let \(A=\left[\begin{array}{rrrr}1&1&-1\\0&3&-2\end{array} \right]\). Find \(\operatorname{NS}\left(A\right)\).
Solution. We find the solution set of \(A\overrightarrow{x}=\overrightarrow{0}\). \[[A\;\overrightarrow{0}]=\left[\begin{array}{rrr|r}\boxed{1}&1&-1&0\\ 0&\boxed{3}&-2&0\end{array} \right] \xrightarrow{\frac{1}{3}R_2} \left[\begin{array}{rrr|r}\boxed{1}&1&-1&0\\ 0&\boxed{1}&-2/3&0\end{array} \right] \xrightarrow{-R_2+R_1} \left[\begin{array}{rrr|r}\boxed{1}&0&-1/3&0\\ 0&\boxed{1}&-2/3&0\end{array} \right] (RREF)\] Corresponding system is \[\begin{eqnarray*} \begin{array}{rcrcrcr} x_1&&&-&\frac{x_3}{3}&=&0\\ &&x_2 &-&\frac{2x_3}{3}&=&0 \end{array} \end{eqnarray*}\] where \(x_1\) and \(x_2\) are basic variables (for pivot columns) and \(x_3\) is a free variable (for non-pivot column). \[\begin{eqnarray*} \begin{array}{rcl} x_1&=&\frac{x_3}{3}\\ x_2&=&-\frac{2x_3}{3}\\ x_3&=&\text{free} \end{array} \end{eqnarray*}\] \[\operatorname{NS}\left(A\right)=\left\lbrace \left[\begin{array}{r} \frac{x_3}{3}\\\frac{2x_3}{3}\\ x_3 \end{array} \right] \; |\; x_3\in \mathbb R \right\rbrace =\left\lbrace \frac{x_3}{3} \left[\begin{array}{r} 1\\2\\ 3 \end{array} \right] \; |\; x_3\in \mathbb R \right\rbrace =\operatorname{Span}\left\lbrace \left[\begin{array}{r} 1\\ 2\\ 3 \end{array} \right] \right\rbrace\]

Remark. If an \(m\times n\) matrix \(A\) has \(k\) non-pivot columns (i.e., \(k\) free variables for \(A\overrightarrow{x}=\overrightarrow{0}\)), then \(\operatorname{NS}\left(A\right)\) is a span of \(k\) vectors in \(\mathbb R^n\).

Last edited