Linear Algebra Home

Linear Span and Subspaces

Definition. A linear combination of vectors $$\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}$$ of $$\mathbb{R}^n$$ is a sum of their scalar multiples, i.e., $c_1\overrightarrow{v_1}+c_2\overrightarrow{v_2}+\cdots+c_k\overrightarrow{v_k}$ for some scalars $$c_1,c_2,\ldots,c_k$$ (real numbers throughout this chapter). The set of all linear combinations of a nonempty set $$S$$ of vectors of $$\mathbb R^n$$ is called the linear span or span of $$S$$, denoted by $$\operatorname{Span}(S)$$ or $$\operatorname{Span} S$$, i.e., $\operatorname{Span}\{\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}\} = \{c_1\overrightarrow{v_1}+c_2\overrightarrow{v_2}+\cdots+c_k\overrightarrow{v_k}\;|\; c_1,c_2,\ldots,c_k\in \mathbb R\}.$ We define $$\operatorname{Span} \varnothing=\{\overrightarrow{0}\}$$. When $$\operatorname{Span}\{\overrightarrow{v_1},\ldots,\overrightarrow{v_k}\} =\mathbb R^n$$, we say $$\{\overrightarrow{v_1},\ldots,\overrightarrow{v_k}\}$$ spans $$\mathbb{R}^n$$.

Example. For $$S=\left\lbrace \left[\begin{array}{r} 1\\ 1\\0 \end{array} \right],\;\left[\begin{array}{r} 1\\2\\0 \end{array} \right] \right\rbrace$$, $\operatorname{Span}(S) =\left\lbrace c_1 \left[\begin{array}{r}1\\ 1\\ 0 \end{array} \right] +c_2 \left[\begin{array}{r}1\\2\\0 \end{array} \right] \; |\; c_1,c_2 \in \mathbb R \right\rbrace.$ Note that $$[0,0,1]^T$$ is not in $$\operatorname{Span}(S)$$ because there are no $$c_1,c_2$$ for which $\left[\begin{array}{r}0\\0\\1 \end{array}\right] =c_1 \left[\begin{array}{r}1\\ 1\\0 \end{array}\right] +c_2 \left[\begin{array}{r}1\\2\\0 \end{array}\right].$ Thus $$S$$ does not span $$\mathbb R^3$$. But any vector of the form $$[a,b,0]^T$$ is in $$\operatorname{Span}(S)$$ because $x_1 \left[\begin{array}{r}1\\ 1\\0 \end{array}\right] +x_2 \left[\begin{array}{r}1\\2\\0 \end{array}\right] =\left[\begin{array}{r}a\\b\\0 \end{array}\right] \implies x_1= 2a-b, x_2=-a+b .$ $\text{i.e., } \left[\begin{array}{r}a\\b\\0 \end{array}\right] =(2a-b) \left[\begin{array}{r}1\\ 1\\0 \end{array}\right] +(-a+b) \left[\begin{array}{r}1\\2\\0 \end{array}\right]\in \operatorname{Span}(S).$ Thus $$S$$ spans the following set $\operatorname{Span}(S) =\left\lbrace \left[\begin{array}{r}a\\ b\\ 0 \end{array} \right] \; |\; a,b \in \mathbb R \right\rbrace,$ which is the $$xy$$-plane in $$\mathbb R^3$$.

Definition. A subspace of $$\mathbb{R}^n$$ is a nonempty subset $$S$$ of $$\mathbb{R}^n$$ that satisfies three properties:

1. $$\overrightarrow{0}$$ is in $$S$$.

2. $$\overrightarrow{u}+\overrightarrow{v}$$ is in $$S$$ for all $$\overrightarrow{u},\; \overrightarrow{v}$$ in $$S$$.

3. $$c \overrightarrow{u}$$ is in $$S$$ for all $$\overrightarrow{u}$$ in $$S$$ and all scalars $$c$$.

In short, a subspace of $$\mathbb R^n$$ is a nonempty subset $$S$$ of $$\mathbb R^n$$ that is closed under linear combination of vectors, i.e., $$c\overrightarrow{u}+d\overrightarrow{v}$$ is in $$S$$ for all $$\overrightarrow{u},\; \overrightarrow{v}$$ in $$S$$ and all scalars $$c,d$$. When $$S$$ is a subspace of $$\mathbb R^n$$, we sometimes denote it by $$S\leq \mathbb R^n$$.

Example.

1. $$\{\overrightarrow{0}\},\mathbb R^n \leq \mathbb R^n$$, i.e., $$\{\overrightarrow{0}\}$$ and $$\mathbb R^n$$ are subspaces of $$\mathbb R^n$$.

2. Show that $$S=\left\lbrace \left[\begin{array}{r} x\\y \end{array} \right] \;|\; x,y\in \mathbb R,\; 2x-y=0\right\rbrace$$ is a subspace of $$\mathbb R^2$$.
Solution.
1. $$\left[\begin{array}{r} 0\\0 \end{array} \right] \in S$$ because $$2\cdot 0-0=0$$.

2. Let $$\overrightarrow{u},\overrightarrow{v} \in S$$ and $$c\in \mathbb R$$. Then $\overrightarrow{u}=\left[\begin{array}{r} x_1\\y_1 \end{array} \right] \text{ and } \overrightarrow{v}= \left[\begin{array}{r} x_2\\y_2 \end{array} \right],$ for some $$x_1,x_2,y_1,y_2 \in \mathbb R$$ such that $$2x_1-y_1=0$$ and $$2x_2-y_2=0$$. Then $\overrightarrow{u}+\overrightarrow{v} =\left[\begin{array}{r} x_1\\y_1 \end{array} \right] +\left[\begin{array}{r} x_2\\y_2 \end{array} \right] = \left[\begin{array}{r} x_1+x_2\\y_1+y_2 \end{array} \right] \in S$ because $$2(x_1+x_2)-(y_1+y_2)=(2x_1-y_1)+(2x_2-y_2)=0$$.

3. $c\overrightarrow{u}=c \left[\begin{array}{r} x_1\\y_1 \end{array} \right]=\left[\begin{array}{r} cx_1\\ cy_1 \end{array} \right] \in S$ because $$2(cx_1)-(cy_1)=c(2x_1-y_1)=0$$.
Thus $$S$$ (which is the line $$y=2x$$) is a subspace of $$\mathbb R^2$$.

3. Let $$S=\left\lbrace \left[\begin{array}{r} 1\\1\\0 \end{array} \right],\;\left[\begin{array}{r} 1\\2\\0 \end{array} \right] \right\rbrace$$. Then $$\operatorname{Span}(S)$$ is a subspace of $$\mathbb R^3$$.
First note that $$\left[\begin{array}{r} 0\\0\\0 \end{array} \right]=0\left[\begin{array}{r} 1\\1\\0 \end{array} \right] +0\left[\begin{array}{r} 1\\2\\0 \end{array} \right] \in \operatorname{Span}(S)$$. Thus $$\operatorname{Span}(S)\neq \varnothing$$. Let $$\overrightarrow{u},\overrightarrow{v} \in\operatorname{Span}(S)$$ and $$c,d\in \mathbb R$$. Then $\overrightarrow{u}=c_1 \left[\begin{array}{r}1\\ 1\\0 \end{array}\right] +c_2 \left[\begin{array}{r}1\\2\\0 \end{array}\right] \text{ and } \overrightarrow{v}= d_1 \left[\begin{array}{r}1\\ 1\\0 \end{array}\right] +d_2 \left[\begin{array}{r}1\\2\\0 \end{array}\right],$ for some $$c_1,c_2,d_1,d_2 \in \mathbb R$$. Then $\begin{eqnarray*} c\overrightarrow{u}+d\overrightarrow{v} &=&c\left(c_1 \left[\begin{array}{r}1\\ 1\\0 \end{array}\right] +c_2 \left[\begin{array}{r}1\\2\\0 \end{array}\right] \right) +d\left(d_1 \left[\begin{array}{r}1\\ 1\\0 \end{array}\right] +d_2 \left[\begin{array}{r}1\\2\\0 \end{array}\right] \right)\\ &=& (cc_1+dd_1) \left[\begin{array}{r}1\\ 1\\0 \end{array}\right] +(cc_2+dd_2) \left[\begin{array}{r}1\\2\\0 \end{array}\right] \in \operatorname{Span}(S). \end{eqnarray*}$ Thus $$\operatorname{Span}(S)$$ (which is the $$xy$$-plane) is a subspace of $$\mathbb R^3$$.

Theorem. Let $$\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}\in \mathbb R^n$$. Then $$\operatorname{Span}\{\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}\}$$ is a subspace of $$\mathbb R^n$$.

Since $$\overrightarrow{v_1} \in \operatorname{Span}\{\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}\}$$, $$\operatorname{Span}\{\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}\} \neq \varnothing$$. Let $$\overrightarrow{u},\overrightarrow{v} \in \operatorname{Span}\{\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}\}$$ and $$c,d\in \mathbb R$$. Then $$\overrightarrow{u}=c_1\overrightarrow{v_1}+c_2\overrightarrow{v_2}+\cdots+c_k\overrightarrow{v_k}$$ and $$\overrightarrow{v}=d_1\overrightarrow{v_1}+d_2\overrightarrow{v_2}+\cdots+d_k\overrightarrow{v_k}$$ for some $$c_1,\ldots,c_k,d_1,\ldots,d_k\in \mathbb R$$. Then $\begin{eqnarray*} c\overrightarrow{u}+d\overrightarrow{v} &=&c(c_1\overrightarrow{v_1}+c_2\overrightarrow{v_2}+\cdots+c_k\overrightarrow{v_k})+d(d_1\overrightarrow{v_1}+d_2\overrightarrow{v_2}+\cdots+d_k\overrightarrow{v_k})\\ &=&(cc_1+dd_1)\overrightarrow{v_1}+(cc_2+dd_2)\overrightarrow{v_2}+\cdots+(cc_k+dd_k)\overrightarrow{v_k} \in \operatorname{Span}\{\overrightarrow{v_1},\overrightarrow{v_2},\ldots,\overrightarrow{v_k}\}. \end{eqnarray*}$

For a given matrix we have two important subspaces: the column space and the null space.

Definition. The column space of an $$m\times n$$ matrix $$A=[\overrightarrow{a_1}\:\overrightarrow{a_2}\:\cdots\overrightarrow{a_n}]$$, denoted by $$\operatorname{CS}\left(A\right)$$ or $$\operatorname{Col}\left(A\right)$$, is the span of its column vectors: $\operatorname{CS}\left(A\right)=\operatorname{Span}\{\overrightarrow{a_1},\overrightarrow{a_2},\ldots,\overrightarrow{a_n}\}.$
Remark. Since each column is an $$m$$ dimensional vector, $$\operatorname{CS}\left(A\right)$$ is a subspace of $$\mathbb R^m$$.

Example. For $$A=\left[\begin{array}{rrr}1&2&3\\0&4&5\end{array} \right]$$, $$\operatorname{CS}\left(A\right)=\operatorname{Span}\left\lbrace \left[\begin{array}{r}1\\0 \end{array} \right], \left[\begin{array}{r}2\\4 \end{array} \right], \left[\begin{array}{r}3\\5 \end{array} \right] \right\rbrace \leq \mathbb R^2$$.

Example. Let $$A=\left[\begin{array}{rrrr}1&-3&-4\\-4&6&-2\\-3&7&6\end{array} \right]$$ and $$\overrightarrow{b}= \left[\begin{array}{r}3\\3\\-4 \end{array} \right]$$. Determine if $$\overrightarrow{b}$$ is in $$\operatorname{CS}\left(A\right)$$.
Solution. Note that $$\overrightarrow{b}\in \operatorname{CS}\left(A\right)$$ if and only if $$\overrightarrow{b}$$ is a linear combination of columns of $$A$$ if and only if $$A\overrightarrow{x}=\overrightarrow{b}$$ has a solution. $\left[\begin{array}{rrr|r}1&-3&-4&3\\-4&6&-2&3\\-3&7&6&-4\end{array} \right] \xrightarrow[3R_1+R_3]{4R_1+R_2} \left[\begin{array}{rrr|r}1&-3&-4&3\\0&-6&-18&15\\0&-2&-6&5\end{array} \right] \xrightarrow{-\frac{1}{3}R_2+R_3} \left[\begin{array}{rrr|r}\boxed{1}&-3&-4&3\\0&\boxed{-6}&-18&15\\0&0&0&0\end{array} \right] (REF)$ Since the REF of $$[A\:\overrightarrow{b}]$$ has no row of the form $$[0,0, 0, c], c\neq 0$$, $$A\overrightarrow{x}=\overrightarrow{b}$$ is consistent and consequently $$\overrightarrow{b}$$ is in $$\operatorname{CS}\left(A\right)$$.

Theorem. An $$m\times n$$ matrix $$A$$ has a pivot position in every row if and only if $$A\overrightarrow{x}=\overrightarrow{b}$$ is consistent for any $$\overrightarrow{b}\in \mathbb R^m$$ if and only if $$\operatorname{CS}\left(A\right)=\mathbb R^m$$.

Example. Since $$A=\left[\begin{array}{rrr}\boxed{1}&2&3\\0&\boxed{4}&5\end{array} \right]$$ has a pivot position in each row, $$\operatorname{CS}\left(A\right)=\mathbb R^2$$.

Definition. The null space of an $$m\times n$$ matrix $$A$$, denoted by $$\operatorname{NS}\left(A\right)$$ or $$\operatorname{Nul}\left(A\right)$$, is the solution set of $$A\overrightarrow{x}=\overrightarrow{0}$$: $\operatorname{NS}\left(A\right)=\{ \overrightarrow{x}\in \mathbb R^n \;|\; A\overrightarrow{x}=\overrightarrow{0}\}.$

Theorem. Let $$A$$ be an $$m\times n$$ matrix. Then $$\operatorname{NS}\left(A\right)$$ is a subspace of $$\mathbb R^n$$.

Since $$A\overrightarrow{0}=\overrightarrow{0}$$, $$\overrightarrow{0}\in \operatorname{NS}\left(A\right)$$. Thus $$\operatorname{NS}\left(A\right)\neq \varnothing$$. Let $$\overrightarrow{u},\overrightarrow{v} \in \operatorname{NS}\left(A\right)$$ and $$c,d\in \mathbb R$$. Then $$A\overrightarrow{u}=\overrightarrow{0}$$ and $$A\overrightarrow{v}=\overrightarrow{0}$$. Then $A(c\overrightarrow{u}+d\overrightarrow{v}) =c(A\overrightarrow{u})+d(A\overrightarrow{v}) =c\overrightarrow{0}+d\overrightarrow{0} =\overrightarrow{0}.$ Thus $$c\overrightarrow{u}+d\overrightarrow{v} \in \operatorname{NS}\left(A\right)$$.

Example. Let $$A=\left[\begin{array}{rrrr}1&1&-1\\0&3&-2\end{array} \right]$$. Find $$\operatorname{NS}\left(A\right)$$.
Solution. We find the solution set of $$A\overrightarrow{x}=\overrightarrow{0}$$. $[A\;\overrightarrow{0}]=\left[\begin{array}{rrr|r}\boxed{1}&1&-1&0\\ 0&\boxed{3}&-2&0\end{array} \right] \xrightarrow{\frac{1}{3}R_2} \left[\begin{array}{rrr|r}\boxed{1}&1&-1&0\\ 0&\boxed{1}&-2/3&0\end{array} \right] \xrightarrow{-R_2+R_1} \left[\begin{array}{rrr|r}\boxed{1}&0&-1/3&0\\ 0&\boxed{1}&-2/3&0\end{array} \right] (RREF)$ Corresponding system is $\begin{eqnarray*} \begin{array}{rcrcrcr} x_1&&&-&\frac{x_3}{3}&=&0\\ &&x_2 &-&\frac{2x_3}{3}&=&0 \end{array} \end{eqnarray*}$ where $$x_1$$ and $$x_2$$ are basic variables (for pivot columns) and $$x_3$$ is a free variable (for non-pivot column). $\begin{eqnarray*} \begin{array}{rcl} x_1&=&\frac{x_3}{3}\\ x_2&=&-\frac{2x_3}{3}\\ x_3&=&\text{free} \end{array} \end{eqnarray*}$ $\operatorname{NS}\left(A\right)=\left\lbrace \left[\begin{array}{r} \frac{x_3}{3}\\\frac{2x_3}{3}\\ x_3 \end{array} \right] \; |\; x_3\in \mathbb R \right\rbrace =\left\lbrace \frac{x_3}{3} \left[\begin{array}{r} 1\\2\\ 3 \end{array} \right] \; |\; x_3\in \mathbb R \right\rbrace =\operatorname{Span}\left\lbrace \left[\begin{array}{r} 1\\ 2\\ 3 \end{array} \right] \right\rbrace$

Remark. If an $$m\times n$$ matrix $$A$$ has $$k$$ non-pivot columns (i.e., $$k$$ free variables for $$A\overrightarrow{x}=\overrightarrow{0}$$), then $$\operatorname{NS}\left(A\right)$$ is a span of $$k$$ vectors in $$\mathbb R^n$$.

Last edited