Two additional vector spaces associated with a matrix

Let $$A \in \mathbb{F}^{m \times n}$$ where $$\mathbb{F}$$ is a field. We have seen $$N(A)$$, the nullspace of $$A$$, is given by $$\{ x \in \mathbb{F}^n : Ax = 0\}$$. There are two additional vector spaces associated with a matrix that we will now discuss.

The column space of $$A$$, denoted by $${\cal C}(A)$$, is the span of the columns of $$A$$. In other words, the we treat the columns of $$A$$ as vectors in $$\mathbb{F}^m$$ and take all possible linear combinations of these vectors to form the span. So $${\cal C}(A)$$ is a subspace of $$\mathbb{F}^m$$.

Example

If $$A$$ is the matrix $$\begin{bmatrix} 1 & 0 & 2 \\ 0 & -1 & 3\end{bmatrix}$$ defined over the real numbers, then $${\cal C}(A)$$ consists of the vectors $$\alpha \begin{bmatrix} 1 \\ 0\end{bmatrix} + \beta \begin{bmatrix} 0 \\ -1\end{bmatrix} + \gamma \begin{bmatrix} 2 \\ 3\end{bmatrix}$$ where $$\alpha, \beta, \gamma \in \mathbb{R}$$. Writing $$\alpha \begin{bmatrix} 1 \\ 0\end{bmatrix} + \beta \begin{bmatrix} 0 \\ -1\end{bmatrix} + \gamma \begin{bmatrix} 2 \\ 3\end{bmatrix}$$ as $$\begin{bmatrix} 1 & 0 & 2\\ 0 & -1 & 3\end{bmatrix} \begin{bmatrix} \alpha \\ \beta \\ \gamma\end{bmatrix},$$ we have ${\cal C}(A) = \left \{ \begin{bmatrix} 1 & 0 & 2\\ 0 & -1 & 3\end{bmatrix} \begin{bmatrix} \alpha \\ \beta \\ \gamma\end{bmatrix} : \alpha,\beta,\gamma \in \mathbb{R} \right \}.$ As a result, $$\begin{bmatrix} a \\b\end{bmatrix}$$ is in $${\cal C}(A)$$ if and only if the system $$\begin{bmatrix} 1 & 0 & 2\\ 0 & -1 & 3\end{bmatrix} \begin{bmatrix} \alpha \\ \beta \\ \gamma\end{bmatrix} = \begin{bmatrix} a \\b\end{bmatrix}$$ has a solution.

The row space of $$A$$, denoted by $${\cal R}(A)$$ is given by $${\cal C}(A^\mathsf{T})$$. So $${\cal R}(A)$$, is a subspace of $$\mathbb{F}^n$$.

Finding bases for column spaces and row spaces

We are going to see how to find a basis for the row space and a basis for the column space. We first consider the case when the matrix is in reduced row-echelon form.

Matrix in reduced row-echelon form

Let $$A = \begin{bmatrix} 1 & -2 & 0 & -1\\0 & 0 & 1 & 1\\ 0 & 0 & 0 &0\end{bmatrix}$$. Say we want to find a basis for $${\cal R}(A)$$.

Now, $$A^\mathsf{T} = \begin{bmatrix} 1 & 0 & 0\\-2 & 0 & 0\\ 0 & 1 &0 \\ 1 & 1 & 0\end{bmatrix}$$. Clearly, the span of the first two columns is the same as the column space of $$A^\mathsf{T}$$ since the third column, which has all entries equal to $$0$$'s, contributes nothing in a linear combination.

Note that each of the first two columns is needed to span $${\cal C}(A^\mathsf{T})$$ since the first column is the only column with a nonzero in the first entry and the second column is the only column with a nonzero in the third entry. Hence, the first two columns of $$A^\mathsf{T}$$ together form a basis for $${\cal R}(A)$$.

Now, let's find a basis for $${\cal C}(A)$$. Note that $$A_2$$, the second column of $$A$$, is a scalar multiple of $$A_1$$. As well, $$A_4 = -A_1 + A_3.$$ Thus, the span of $$\{A_1,A_3\}$$ is still $${\cal C}(A)$$. Clearly, $$\{A_1,A_3\}$$ is a linearly independent set. So it gives a basis for $${\cal C}(A)$$.

Given a matrix $$A$$ in reduced row-echelon form, the nonzero columns in $$A^\mathsf{T}$$ form a basis for $${\cal R}(A)$$, and the pivot columns in $$A$$ form a basis for $${\cal C}(A)$$.

General matrix

A useful fact concerning the nullspace and the row space of a matrix is the following:

Elementary row operations do not affect the nullspace or the row space of the matrix.

Hence, given a matrix $$A$$, first transform it to a matrix $$R$$ in reduced row-echelon form using elementary row operations. Then find a basis for the row space of $$R$$. It will then be a basis for the row space of $$A$$.

What about the column space? Clearly, elementary row operations do affect the column space. However, we can still make use of row reduction to help find a basis for the column space.

The key is the following: If $$A$$ has been row-reduced to a matrix $$R$$ in reduced row-echelon form, then the columns of $$A$$ that correspond to the pivot columns of $$R$$ form a basis for $${\cal C}(A)$$. In other words, if in $$R$$, columns 1,3,7 are the pivot columns, then $$A_1,A_3,A_7$$ form a basis for $${\cal C}(A)$$.

Example

Let $$A = \begin{bmatrix} 1 & -2 & 3 & 1\\ 0 & 1 & -1 & 0\\ -1 & 1 & -2 & -1 \end{bmatrix}$$ be defined over the real numbers. Transforming $$A$$ to a matrix in reduced row-echelon form gives $$\begin{bmatrix} 1 & 0 & 1 & 1\\ 0 & 1 & -1 & 0\\ 0 & 0 & 0 & 0 \end{bmatrix}$$. Since the pivot columns are the first two columns, $$A_1$$ and $$A_2$$ will give a basis for $${\cal C}(A)$$. Indeed, $$A_1$$ and $$A_2$$ are clearly linearly independent, $$A_3 = A_1 - A_2$$ and $$A_4 = A_1$$.

Rank of a matrix

It can be shown that $$\dim({\cal C}(A)) = \dim({\cal R}(A)).$$ The dimension of the row space or column space of $$A$$ is called the rank of $$A$$, denoted by $$\operatorname{rank}(A)$$. Hence, for the matrix $$A = \begin{bmatrix} 1 & -2 & 3 & 1\\ 0 & 1 & -1 & 0\\ -1 & 1 & -2 & -1 \end{bmatrix}$$ above, we have $$\operatorname{rank}(A) = 2$$.

Exercise

For each of the following matrices, find a basis for its nullspace, a basis for its row space, and a basis for its column space.

1. $$\begin{bmatrix} 1 & 2 & 3\\ \end{bmatrix}$$.

2. $$\begin{bmatrix} 1 & 0 & 2 & 0\\ 0 & 1 & -1 & 1\\ 1 & 1 & 1 & 1 \end{bmatrix}$$.