A vector space is simply a mathematical set on which we can perform addition and scalar multiplication. We already have some familiarity with vector spaces since \(\real^n\) is a good example. However, as mentioned in the introduction to this chapter, polynomials have similar operations so we would like to create a mathematical structure that allows us to study vectors and polynomials as equals. This is why the concept of a vector space is so useful.
The usual place to get started would be with a general definition of a vector space. However, this is one place in mathematics, among others, where a general definition can obscure the underlying idea. For that reason, let’s just start with some examples.
Notice that the entries in our matrices are real numbers \(\real\text{.}\) We could instead change the example so that we consider matrices whose entries are in the complex numbers \(\complex\text{.}\)
These examples show that vector spaces have an underlying field, which is the set of scalars by which we can multiply. You may or may not know about fields depending on whether you have studied abstract algebra. In either case, the underlying field of our vector spaces will always be either the real numbers or the complex numbers, which we will write as \(\field=\real\) or \(\field=\complex\text{.}\)
A vector space over a field \(\field\) is a set \(V\) with two operations, scalar multiplication by elements of \(\field\) and addition, under which \(V\) is closed. Moreover, these operations satisfy the following natural properties:
Addition is commutative; that is \(\vvec+\wvec=\wvec+\vvec\) for every pair of \(\vvec,\wvec\in V\text{.}\)
Every element \(\vvec\) has an additive inverse \(\wvec\) such that \(\vvec + \wvec = 0\text{.}\) We will usually write the additive identity as \(-\vvec\text{.}\)
That is a long list of properties. Technically speaking, if we want to check that some set with operations is a vector space, we need to check each one of those properties. In practice, however, we will know a vector space when we see one, and we will be fairly loose with these details. We will often think of a vector space as a set that is closed under operations that are called addition and scalar multiplication and that satisfy some familiar “compatibility properties.”
Rather than the set of all polynomials, we define the set \(\pbb_n\) to be the set of all polynomials whose degree is \(n\) or less. For example, \(\pbb_2\) contains all polynomials of degree two or less:
Of course, the set of all polynomials is larger than the set of quadratic polynomials, and we have \(\pbb_2\subset \pbb\text{.}\) We say that \(\pbb_2\) is a vector subspace of \(\pbb\text{.}\)
A subset \(W\) of a vector space \(V\) is called a subspace of \(V\) if \(W\) is closed under the operations of scalar multiplication and addition that it inherits from \(V\text{.}\)
Every vector space \(V\) has two subspaces that we will sometimes need to consider, namely, the subspace consisting of only the zero vector \(W=\{0\}\) and the entire vector space \(W=V\) itself.
Let \(\fcal\) be the set of functions whose domain is \(\real\) and whose codomain is \(\complex\text{;}\) that is, functions of the form \(f:\real\to\complex\text{.}\) It follows that \(\fcal\) is a complex vector space.
If we were to consider functions \(f:\real\to\real\text{,}\) we would obtain a real vector space. This is not a subspace of \(\fcal\text{,}\) however, since the underlying fields are different. Rather, here are some natural subspaces of \(\fcal\text{.}\)
The set of functions that satisfy \(f(17)=1\) is, however, not a subspace since it is not closed under scalar multiplication or vector addition. For instance, if \(f(17)
= 1\text{,}\) then \((2f)(17) = 2f(17) = 2\cdot1=2\text{.}\)
If \(V\) is a vector space and \(V_1\) and \(V_2\) are subspaces, then \(V_1\cap V_2\) is also a subspace of \(V\) as it can be seen that the interection is closed under scalar multiplication and addition. The “compatibility properties” are satisfied due to the fact that \(V\) is itself a vector space.
When working with a vector space \(V\text{,}\) we will frequently refer to the elements of \(V\) as vectors even though they may be polynomials, matrices, functions, or even something entirely different.
Our earlier study of linear algebra really began once we introduced linear combinations. Of course, linear combinations are defined purely in terms of scalar multiplication and addition so we can form linear combinations of elements in a vector space.
Suppose that \(\vvec_1,\ldots,\vvec_n\) is a set of vectors in a vector space \(V\) over a field \(\field\text{.}\) A linear combination of these vectors is a vector of the form
Consider the vector space \(\pbb_2\) consisting of polynomials having degree two or less and the polynomials \(p_1(x)=3x+4\) and \(p_2(x)=7x^2-2x+1\text{.}\) We can form the linear combination
It’s not hard to see that the span of a set of vectors \(\vvec_1,\vvec_2,\ldots,\vvec_m\) in \(V\) forms a subspace. We just have to check that the span is closed under scalar multiplication and addition. So we will consider vectors
The second statement is logically equivalent to the first so our proof will focus on the first statement. Suppose that the set \(\vvec_1,\ldots,\vvec_m\) is linearly dependent and that \(\vvec_k\) is the first vector that is a linear combination of vectors that occur previously in the list. This means that there are scalars \(c_1,c_2,\ldots,c_{k-1}\) such that
Suppose that \(\vvec_1,\vvec_2,\ldots,\vvec_m\) is a linear dependent set of vectors and that \(\vvec_k\) can be written as a linear combination of the other vectors. Removing \(\vvec_k\) from the set does not change the span; that is,
If \(\wvec = a_1\vvec_1+a_2\vvec_2+\ldots+a_m\vvec_m\text{,}\) then we can replace \(\vvec_k\) in this expression with a linear combination of the other vectors. This shows that \(\wvec\) can be written as a linear combination of the set of vectors with \(\vvec_k\) removed.
from which we conclude that \(a_1=0\text{,}\)\(a_2=0\text{,}\) and \(a_3=0\text{.}\) Therefore, \(p_1\text{,}\)\(p_2\text{,}\) and \(p_3\) are linearly indepedent by Proposition 1.1.17 and hence form a basis for \(\pbb_2\text{.}\)
To see this, suppose that \(p(x)=a_0 + a_1x + a_2x^2\) is a polynomial in \(\pbb_\text{.}\) We wish to see that \(p\) can be written as a linear combination of \(q_1\text{,}\)\(q_2\text{,}\) and \(q_3\text{.}\) This means that there are scalars \(c_1\text{,}\)\(c_2\text{,}\) and \(c_3\) such that
That is, \(c_kx^k\) is the only term involving \(x^k\text{.}\) Therefore, \(c_k=0\text{,}\) which contradicts our assumption that \(c_k\neq 0\text{.}\)
To see that these polynomials span \(\pbb_n\text{,}\) we offer a proof by induction. When \(n=0\text{,}\) we see that \(p_0 = 1\) spans \(\pbb_0\text{.}\) Now suppose that \(p_0,p_1,\ldots,p_{n-1}\) span \(\pbb_{n-1}\) and that \(p(x)=a_0+a_1x+a_2x^2 + \ldots + a_nx^n\) is a polynomial in \(\pbb_n\text{.}\) Notice that the polynomials \(p(x)\) and \(a_np_n(x)\) have the same coefficient of \(x^n\text{.}\) Therefore,
There is no finite set that forms a basis for \(\pbb\text{,}\) the set of all polynomials. Given any finite set, there is a polynomial having a highest degree \(m\text{.}\) Therefore, the polynomial \(x^{m+1}\) is not in the span of the set so it cannot be a basis.
We say that a vector space \(V\) is finite dimensional if there is a finite set whose span is \(V\text{.}\) Otherwise, we say that \(V\) is infinite dimensional.
If \(V\) is a finite dimensional vector space, there is a finite set of vectors whose span is \(V\text{.}\) If this set of vectors is linearly independent, then it forms a basis. It not, we can remove one vector that is a linear combination of the others. Proposition 1.1.18 says that the span of the remaining vectors is still \(V\) so we continue removing vectors one at a time until we obtain a linearly independent set, which must be a basis.
Notice that the two bases for \(\pbb_2\) in Example 1.1.20 and Example 1.1.21 both consist of three polynomials. That is, these two bases for \(\pbb_2\) have the same number of vectors. This is generally true as we now explain. First, we will prove a more technical, but still useful, result.
The number of vectors in a linearly independent set in the vector space \(V\) is no more than the number of vectors in any set whose span is \(V\text{.}\)
Suppose that \(\vvec_1,\vvec_2,\ldots,\vvec_m\) is a linear independent set in the vector space \(V\) and that \(\wvec_1,\wvec_2,\ldots,\wvec_n\) is a set whose span is \(V\text{.}\) We wish to show that \(m\leq n\text{.}\)
whose span is \(V\text{.}\) Because the span of the \(\wvec\) vectors is \(V\text{,}\)\(\vvec_m\) is a linear combination of the \(\wvec\) vectors, which means that this set of vectors is linearly dependent. We let \(\uvec\) be the first vector in the list that is a linear combination of vectors that occur previously in the list. Since the set of \(\vvec\) vectors is linearly independent, \(\vvec_m\) is nonzero, which means that \(\uvec\) must be one of the \(\wvec\) vectors. If we remove \(\uvec\text{,}\) we have a new list
which must be linearly dependent. Let \(\uvec\) be the first vector in the list that is a linear combination of vectors that occur previously in the list. Once again, since the \(\vvec\) vectors form a linearly independent set, we know that \(\uvec\) is one of the \(\wvec\) vectors. We can remove \(\uvec\) to obtain a new list of vectors whose span is \(V\text{.}\) Again, the cardinality of this new list is \(n\text{.}\)
We continue this process until all the \(\vvec\) vectors have been added to the beginning of the list. At each step, the vector we remove is one of the \(\wvec\) vectors since the \(\vvec\) vectors are linearly independent. Therefore, we have a list of \(n\) vectors that contains \(\vvec_1,\vvec_2,\ldots,\vvec_m\text{,}\) which says that \(m\leq n\text{.}\)
Suppose that \(\vvec_1,\vvec_2,\ldots,\vvec_m\) is one basis and that \(\wvec_1,\wvec_2,\ldots,\wvec_n\) is another. The set of \(\vvec\) vectors forms a linearly independent set and the set of \(\wvec\) vectors spans \(V\text{.}\) By Proposition 1.1.26, we know that \(m\leq n\text{.}\)
If \(V\) is a finite dimensional vector space, we define its dimension to be the number of vectors in a basis. In this case, the number of vectors in any basis is the same so this definition does not depend on which basis we choose.
We may informally think of the dimension of a vector space as a measure of its size. Therefore, it should follow that the dimension of a subspace cannot be larger than the dimension of the vector space in which it resides. We first call attention to a useful fact.
If \(\vvec_1,\vvec_2,\ldots,\vvec_m\) is a linearly independent subset of the vector space \(V\) whose span is not \(V\text{,}\) then there is a vector \(\uvec\) in \(V\) such that \(\vvec_1,\vvec_2,\ldots,\vvec_m,\uvec\) is a linearly independent subset of \(V\text{.}\)
Under the assumptions of this proposition, the span of \(\vvec_1,\vvec_2,\ldots,\vvec_m\) is not \(V\) so there is a vector \(\uvec\) that is not in the span of the \(\vvec\) vectors. This means that it is not a linear combination of the \(\vvec\) vectors and therefore
If \(V\) is a vector space whose dimension \(\dim V =
n\) and \(\vvec_1,\vvec_2,\ldots,\vvec_n\) is a linearly independent set in \(V\text{,}\) then \(\vvec_1,\vvec_2,\ldots,\vvec_n\) also spans \(V\) and is therefore a basis of \(V\text{.}\)
We will prove this fact by contradiction. Suppose that \(\basis{\vvec}{n}\) does not span \(V\text{.}\) Then by Proposition 1.1.31, we can add a vector \(\uvec\) so that
is a linearly independent subset of \(n+1\) vectors in \(V\text{.}\) However, any linear independent subset of \(V\) can have no more than \(n\) vectors by Proposition 1.1.26, which says that \(\vvec_1,\vvec_2,\ldots,\vvec_n\) must span \(V\text{.}\)
We will first explain why \(W\) is a finite dimensional vector space, which means we need to explain why there is a finite set \(W\) that spans \(W\text{.}\) We begin with any set of vectors \(\wvec_1,\wvec_2,\ldots,\wvec_m\) in \(W\text{.}\) By Proposition 1.1.18, we can remove vectors one at a time until we obtain a linearly independent set in \(W\text{.}\) If this set does not span \(W\text{,}\) then we can add vectors in \(W\) one at a time to obtain new linearly independent sets in \(W\text{.}\) This process must stop at some point since any linearly independent set in \(V\) can have no more than \(\dim V\) vectors. Therefore, we have obtained a finite set that spans \(W\text{,}\) which says that \(W\) is finite dimensional.
Since any basis for \(W\) is also a linearly independent subset of \(V\text{,}\) it can contain no more vectors than a basis of \(V\text{.}\) This tells us that
\begin{equation*}
\dim W \leq \dim V\text{.}
\end{equation*}
Suppose that \(\vvec_1,\vvec_2,\ldots,\vvec_m\) is a linearly independent set in \(V\) and that \(\wvec_1,\wvec_2,\ldots,\wvec_n\) is a basis for \(V\text{.}\) Join the two lists together to obtain
We are guaranteed that the span of this set is \(V\text{.}\) If it is not a linear independent set, then we remove the first vector that is a linear combination of the others. Since the \(\vvec\) vectors are linearly independent, the vector that is removed must be one of the \(\wvec\) vectors. Continuing in this way, we eventually obtain a basis that includes the vectors \(\vvec_1,\ldots,\vvec_m\text{.}\)
If \(V\) is a finite dimensional vector space of dimension \(n\) and \(\vvec_1,\vvec_2,\ldots,\vvec_n\) is a set of vectors in \(V\text{,}\) it follows that
if the set of vectors is linearly independent, then it is a basis.
If we have two subspaces \(V_1\) and \(V_2\) in a vector space \(V\text{,}\)Example 1.1.9 says that the intersection \(V_1\cap V_2\) is also a subspace of \(V\text{.}\) The same is not true, however, for the union \(V_1\cup
V_2\) as demonstrated by Example 1.1.37.
Suppose that \(V=\real^2\) with \(\vvec_1=\twovec11\) and \(\vvec_2=\twovec{-1}1\text{.}\) If \(V_1=\laspan{\vvec_1}\) and \(V_2=\laspan{\vvec_2}\text{,}\) then \(V_1\cup V_2\) consists of two lines as seen in Figure 1.1.38. Notice that \(\vvec_1 + \vvec_2 =
\twovec02\) is not in \(V_1\cup V_2\text{,}\) which shows that \(V_1\cup V_2\) is not closed under addition.
Instead, we construct a vector subspace that contains the union \(V_1\cup V_2\text{.}\) This construction will be useful in the future as it helps us to break vector spaces into simpler pieces. Suppose that we have a vector space \(V\) and that \(V_1\) and \(V_2\) are subspaces. We can define a new subspace \(V_1+V_2\text{.}\)
Given subspaces \(V_1\) and \(V_2\) of \(V\text{,}\) we form the vector space sum \(V_1+V_2\) as the subset of \(V\) whose elements can be written in the form \(\vvec_1+\vvec_2\) where \(\vvec_1\in
V_1\) and \(\vvec_2\in V_2\text{.}\)
Suppose that \(V=\real^3\text{,}\)\(V_1\) is the 1-dimensional subspace whose vectors are \(\threevec x00\text{,}\) and \(V_2\) is the 1-dimensional subspace whose vectors are \(\threevec 0y0\text{.}\) Then \(V_1+V_2\) is the 2-dimensional subspace whose vectors have the form \(\threevec xy0\text{.}\)
Suppose that \(V=\real^4\text{,}\)\(V_1\) is the 2-dimensional subspace having vectors \(\fourvec xy00\text{,}\) and \(V_2\) is the 2-dimensional subspace having vectors \(\fourvec 0yz0\text{.}\) Then \(V_1+V_2\) is the three-dimensional subspace consisting of vectors \(\fourvec
xyz0\text{.}\)
Suppose that \(V_1\cap V_2\) has a basis \(\uvec_1,\ldots,\uvec_m\text{.}\) Since \(V_1\cap V_2\) is a subspace of \(V_1\text{,}\) this basis can be extended to a basis for \(V_1\) by adding vectors \(\vvec_1,\ldots,\vvec_j\text{.}\) Similarly, the basis for \(V_1\cap V_2\) can be extended to a basis for \(V_2\) by adding vectors \(\wvec_1,\ldots,\wvec_k\text{.}\) Putting all these vectors together gives the set
Any vector in \(V_1+V_2\) can be written as the sum of a vector in \(V_1\) and a vector in \(V_2\text{.}\) Therefore, the span of the vectors in \(\bcal\) is \(V_1+V_2\) since contained within \(\bcal\) is a basis for \(V_1\)and a basis for \(V_2\text{.}\)
The vector on the left is in \(V_2\) but not \(V_1\cap
V_2\text{,}\) while the vector on the right is in \(V_1\text{.}\) The only way this can happen is for both vectors to be zero, which means that all the coefficients must be zero.
