Linear Algebra Basis & Dimension

Welcome back to Educator.com and welcome back to linear algebra.0000

Last couple of lessons, we talked about linear independence, and we talked about the span.0004

Today we are going to talk about something called basis and dimension, and we are going to use linear independence and span to define those things.0013

So, let us get started. Okay. Let us start with a definition here.0021

Again, math usually always starts with some definition. Okay.0031

Vectors v1, v2, and so on... all the way to vk are said to form a basis for vector space v.0037

If 1... v1, v2, all the way to v_k, with a span b.0065

And 2... v1, v2... v_k are independent, linearly independent, but I will just write independent.0086

So, again, in the case of a set of vectors that is both independent and happens to span a vector space or some subspace, span something that we are, that we happen to be interested in dealing with.0103

We actually give it a special name. It is called a basis. Now, you can have vectors that are independent, but do not necessarily span a space.0119

So, for example, if I had 3-space, I could take the i vector and the j vector.0130

Well, they certainly are independent, they are orthogonal, they have nothing to do with each other... and yet they do not span the entire space. They only span a part of the space... in other words, the xy plane.0135

Or, you can have vectors that span the entire space, but are not necessarily independent.0145

So, again, let us take 3-space. I can have i,j,k, and let us say I decided to take also the vector 5k, another vector in the direction of k, but 5 times its length.0152

Well, it is a different vector. So, there are four vectors but... and they span the space, you know, every single vector can be written as a linear combination of i,j,k, and 5k, but they are not linearly independent.0165

They are dependent, because 5k can be written as a, well, constant × k. It is just they are parallel, they are the same thing,0179

So, again, you can have something that spans a space but is not independent, and you can have vectors that are independent but do not necessarily span the space.0190

What we want is something that does both. When it does both, it is called a basis for that space... profoundly important.0199

Okay. Let us see what we can do. Let us just throw out some basic examples.0207

Okay. So, the one we just discussed, e1, e2, and e3, they form a basis for R3.0218

Just like e1, e2, e3, e4, e5 would form a basis for R5.0235

Let us do a computational example here.0243

We are going to take a list of four vectors.0249

v1 = (1,0,1,0), v2 = (0,1,-1,2), these are vectors by the way, I better notate them as such.0254

v3 = (0,2,2,1) and v4 = (1,0,0,1), again I just wrote them in a list, you can write them as vectors, anything you want.0273

Let me see. We want to show that these four vectors are... so, show that these form a basis for R4.0290

Well, what do we need to show that they form a basis. Two things, we need to show that the vectors span R4, in this case, and we need to show that they are linearly independent.0309

So, let us get started, and see which one we want to do first.0321

Let us go ahead and do independence first. So, again, we form the following.0326

So, equation 1. Remember, c1v1 + c2v2, I am not going to write out everything, but it is good to write out the equation which is the definition of dependence and independence... c3v3 + c4v4 = the 0 vector.0336

When I put the vectors, v1, v2, v3, v4 in here, multiply the c's, get a linear system, convert that to a matrix, I get the following (1,0,0,1,0).0354

Again, that final 0 is there... (0,1,2,0,0), (1,-1,2,0,0), again the columns of the matrix are just the vectors through v4 (0,2,1,1,0). Okay.0368

When I subject this to reduced row echelon, I get the following. c1 = 0, c2 = 0, c3 = 0, c4 = 0.0392

Again, you can confirm this with your mathematical software. This is the non-trivial solution. It implies independence. Good.0403

So, part of it is set, now let us see about the span. Well, for the span, we need to pick an arbitrary vector in R4, since we are dealing with R4.0415

We can just call it -- I do not know -- (a,b,c,d), and, we need to find to set up the following equation.0425

I will not use c because we used them before, I will use k... k1v1, constant, k2v2 + k3v3 + k4v4... symbolism in mathematics just gets crazy. Very tedious sometimes.0432

And... I will just call it v arbitrary... just some vector v.0452

Although, again, we can set up the solution, we can go (1,0,1,0), (0,1,-1,2), (0,2,2,1), (1,0,0,1), and we can do (a,b,c,d).0458

You know what, let me go ahead and just write it out, so you see it.0470

We have (1,0,0,1), (0,1,2,0), (1,-1,2,0), (0,2,2,1), and of course our vector, this time it is not a (0,0,0,0), it is going to be (a,b,c,d).0476

Again, the nice thing about mathematical software is that it actually solves this symbolically. Not necessarily numerically.0497

So, it will give you a solution for k1, k2, k3, k4 in terms of a, b, c, and d. Well, there does exist a solution.0503

Okay. There does exist a solution. That means that any arbitrary vector can be represented by these 4 vectors.0513

So, let us see, so v1, v2, v3, and v4, which are just v1, v2, v3, v4, span R4.0527

We found something that spans R4, and we also found that they are linearly independent, so yes, these vectors are a good basis.0543

Are they the best basis? Maybe, maybe not, it depends on the problem at hand... but they are a basis and it is a good basis for R4.0560

Okay. Let us list a theorem here... theorem... if s, the set of vectors, v1 so on and so forth onto vk, is a basis for v.0575

So, if the set is a basis for v, then every vector in v can be written in 1 and only 1 way, as a linear combination of the vectors in s.0603

That is not s, we should write the vectors in s.0655

So, in other words, if I know that s is a basis for the vector space, any vector in that vector space can only be represented 1 way.0665

That means the particular representation, the constants that are chosen is unique. Not multiple ways, it is unique.0675

Another theorem... actually, let me write this one in blue because we are possibly going to do something with this one.0684

Let s be v1... vk be a set of non-zero vectors in v, and we will let w equal the span of s.0702

So, we have this set s, there is a span of it, we will call that w, because it may not span the entire vector space, that is why we are giving it different, but obviously it is going to... I mean it is in v, so it is going to be some subset of it.0736

Then, some subset of s is a basis for w. Okay, let us stop and think about what this means.0752

I have a vector space v, I have some arbitrary collection of vectors that I have taken from v and I call that set s, just a list of vectors.0771

I know that these vectors span some part of v.0779

I call that w, if I need to give it a name, or I can just refer to it as the span of s.0785

Well, if I take some subset of this, maybe all of it, but so... either k vectors or less than k vectors, some subset of it, it actually forms a basis for the span.0791

That makes sense. Again, you have some set of vectors that spans an entire space, well, either all of the vectors together are independent, in which case that is your basis.0805

Or, they might be dependent, which means that you should be able to throw out a couple of them and reduce the number.0820

But to get something, some set of vectors from here, some collection that actually forms a basis for the span.0825

let us see how this works in terms of... a real life example. Okay.0834

We are going to list a procedure for finding the subset of s, of any s that is a basis for the span of s.0841

Let me actually move forward. Let me write down what this is.0854

Procedure for finding a subset of s, that is a basis for this span of s.0866

Okay. First thing we are going to do. We want to form c1v1 + c2v2 + so on and so forth... ckvk = 0.0890

We want to set up the homogeneous system, okay?0907

Now, we want to solve the system by taking it to reduced row echelon form.0911

Now, here is the best part. The vectors corresponding to the leading entries form a basis for span s.0930

This is actually kind of extraordinary. I love this, and I do not know why, but it is amazing.0963

I have this collection of vectors that spans a particular space.0969

I set up the homogeneous system and I subject it to Gauss Jordan elimination, bring it down to reduced row echelon form, and as you know, not every column needs to have a leading entry.0974

Well, the columns that do have a leading entry, that means I throw out all of the others.0986

The original vectors that correspond to those columns that have leading entries, they actually form a basis for my span of s.0992

So, let us just do an example and see what happens. Let us take the following vectors.0998

let me do this in red, actually... so v1 = 1... this is not going to work... (1,2,-2,1).1007

v2 = (-3,0,-4,3). v3, and of course these are vectors, so let me notate them as such... (2,1... this is also 1,-1).1029

v4 = (-3,3,-9,6).1054

v5 = (9,3,7,-6).1065

So, we have 5 vectors... we want to find a subset of these vectors, it might be all 5, it might be 2, it might be 3, it might be 4... that form a basis for the span of s.1073

Okay? Okay. So, we form for step... we do this thing right here.1091

So, we set up this equation and we put these vectors in for this equation, and we end up with the following system.1105

Columns... these vectors just going down... (1,2,-2,1)... or you can do them across... either way.1115

(1,-3,2,-3,9), 1, 2, 3, 4, 5 because we have 5 vectors... 5 columns, and of course the augmented is going to be 0.1126

(2,0,1,3,3,0), (-2,-4,1,-9,7,0), (3,-1,6,-6,0)... good.1140

We are going to subject that to reduced row echelon form.1161

When we do that, let me just put that there and write that there. Let me see...1165

Let me move on to the next page, that is not a problem.1172

So, we have subjected that matrix to reduced row echelon and we end up with the following... (1,0,0,0), (0,1,0,0), (1/2,3/2,3/2,0), (-1/2,3/2,5/2,0), and 0's everywhere else.1176

So our reduced row echelon looks like this.1206

Well, leading entry, leading entry, no leading entries anywhere else.1210

So, vector number 1, vector number 2, v1 and v2 form a basis.1217

So, it is not these, it is not (1,0,0,0), (0,1,0,0).1231

This is the reduced row echelon from the matrix, the columns of which are the vectors that we are talking about.1236

So, those vectors, the actual columns from the original matrix, those 2 vectors, so we started off with 5 vectors, and we found two of them that actually span the entire space.1242

We threw out the other three, they are not that important. We can describe the entire span with just these 2 vectors.1254

Form a basis for the span of s.1260

Again, this is really, really extraordinary.1266

Okay. Let us... another theorem... if s, v1... so on and so forth all the way to vk and t, which is, let us say w1 all the way to wk... okay?1270

wN, so if s is the set of vectors v1 to vk, k could be 13 so we might have 13 vectors in this one... and t is equal to w1 all the way to wN.1308

So, k and n do not have to necessarily be the same, but here is what the theorem says.1320

If these 2 sets are bases for v, then k = n.1328

In other words, if I have a given vector space, and if I have the bases for them, the bases have the same number of vectors.1340

So, the basis set has the same number of vectors. In other words, I cannot have a vector space that has one basis that is 3 vectors and another that is 5 vectors.1349

That is not what basis is. Basis expands the set, and it is linearly independent.1358

Therefore, if I have 2 bases, they have to have the same number of elements in them.1364

It makes sense. Okay. Now, because of this, once again, every basis of a given vector space has the same number of vectors in it.1368

There are an infinite number of bases for a vector space... but of that infinite number, they all have the same number of vectors in them.1382

Therefore, we define... again, very, very important definition... the dimension of a non-zero vector space dimension -- fancy word -- is the number of vectors in a basis for the vector space.1393

So, read this again, the dimension of a non-zero vector, of a non-zero vector space is the number of vectors in the basis for that space.1440

So, dimension is kind of a fancy word that a lot of people throw around.1452

So, we talk about 3-dimensional space, the space that we live in. Well, 3-dimensional space, there are a couple of ways to think about it.1458

Yes, it means 3-dimensional space because it will require 3 numbers to actually describe a point, (x,y,z)... three coordinates.1467

However, the actual mathematical definition is 3 space is 3 dimensional because any basis for 3-space has to be made up of 3 vectors... 5 dimensional space.1475

Any basis for 5-dimensional space has to be made up of 5 vectors. I cannot have 4 vectors describing it for 5 dimensional space. It is not going to happen.1489

Can I have 6 vectors that actually describe it? Yes, I can have 6 vectors that span the 5-dimensional space, but that span is not linearly independent.1501

So, because... and that is the whole idea. The dimension of a space is the number of vectors that form the basis and a basis is expansive and it is linearly independent.1513

Okay. Let us see here. p2, which we have used a lot.1526

That is the vector space of all polynomials of degree < or = 2.1534

The dimension is 3, and here is why. The basis, we will list a basis, and that should tell you.1548

This is the best part, if you just want to list a basis you can just count the number of vectors... that is how many dimensions that space is.1555

t², t, and 1. Any linear combination of t², t, and 1 will give you every single possible polynomial of degree < or = 2.1562

For example, 3t² + 6t - 10. Well, it is 3 × t², 6 × t, -10 × 1.1580

3t + 2... 3 × t... 2... it is of degree 2, this is degree 1. degree less than or equal to 2.1592

So, this one has to be in there. So, p2 has a dimension 3.1602

pn has dimension n + 1.1615

Okay. Now, here is where it gets really, really interesting and sort of just a sideline discussion, something sort of think about a little bit of mathematical culture. A little bit of abstraction...1621

Notice that this p2 has a dimension of 3. Well, our 3-space, our normal 3-space that we live in also has a dimension of 3.1636

As it turns out, all vector spaces of a given dimension, the only different between the vector spaces is the identity of their elements.1649

In one vector space, R3, we are talking about points, or vectors, arrows.1658

In this vector space, where this is a basis, it is a dimension of 3... the elements are actual polynomials.1664

As it turns out, the identity of the elements is the only thing that is different about those 2 spaces. These two spaces have the exact same algebraic properties.1675

They behave exactly the same way. In fact, I do not even need to think about it... if I can find myself 15 other vector spaces that have a dimension of 3, the identity of those elements completely does not matter.1683

In fact, it does not even matter, I can treat it completely symbolically. I can call them whatever I want. I can label them whatever I want.1697

What is important is the underlying algebraic property, and it is the same for every single vector space of a given dimension. That is what is extraordinary, that is what gives mathematics its power.1704

Once I understand, let us say R3, and we understand R3 pretty well... we live in this space, we enjoy the world around us, look at what we have done with the world around us.1715

If I find any other vector space with strange objects in it, if it has a dimension of 3, I know everything about it. I know absolutely everything about it because it behaves the same way that R3 does.1724

Again, that is really, really extraordinary... the last thing that I want to leave you with in this particular lesson is that what we have dealt with are finite dimensional vector spaces.1737

In other words, we know that R3 has an infinite number of vectors in them, but the basis, the dimension is finite... 3.1748

That means I only need 3 vectors in order to describe the entire space.1757

Now, that is not always true. There are infinite dimensional vector spaces that require an infinite number of vectors to actually describe them.1761

Those of you that go on into higher mathematics, or not even that, those of you who are engineering and physics majors, at some point you will be discussing something called the Fourier series, which is an infinite series of trigonometric polynomials.1772

Sin(x), cos(x), sin2(x), cos2(x), sin3(x), cos3(x), and so on. That is an infinite dimensional vector space.1785

Okay. So, I will list... let us see... 2 infinite dimensional vector spaces, we of course are not going to deal with it.1796

Linear algebra, mostly we stick with finite dimensional vector spaces, but I do want you to be aware of them.1804

p, the space of all polynomials... all polynomials, that is an infinite dimensional vector space.1811

It requires... it has an infinite number of vectors in its basis. Not like p2 or R3, that only have 3.1817

The other one is the space of continuous functions on the real line.1824

So, the space of continuous functions, you will see it represented like this... from negative infinity to infinity... that is defined on the entire real line.1838

That space has an infinite number of dimensions. I need an infinite number of functions in order to be able to describe all of the other functions, if I need to do so.1845

I just wanted to throw that out there. Really, what I wanted you to take away from this is that the identity for a vector space of any given dimension, the identity of the elements is completely irrelevant.1859

The underlying behavior is what we are concerned with, and the underlying behavior is exactly the same.1870

Thank you for joining us here at Educator.com, linear algebra, we will see you next time.1876