Linear Algebra Change of Basis & Transition Matrices

Welcome back to Educator.com and welcome back to linear algebra.0000

In the previous lesson, we talked about the coordinates of a particular vector and we realized that if we had two different bases that the coordinate vector with respect to each of those bases is going to be different.0004

So, as it turns out, it is not all together... it has to be this or that.0018

One basis is as good as another. We are going to continue that discussion today, deal with coordinates some more and we are going to talk about something called a transition matrix.0022

Where, if we are given the coordinates... if we are given both bases and if we are given the coordinates with respect to one basis, can we actually transform that and is there a matrix that actually does that.0032

The answer is yes, there is a matrix. It is called the transition matrix from one basis to another, and it ends up being a profoundly important matrix.0045

So, let us just dive right in.0053

The first thing I want to talk about it just two brief properties of the coordinates that we mentioned.0058

Their properties are exactly the same as that of vectors, so, it is going to be nothing new.0066

It is just the notation is, of course, slightly different because we have that little subscript s and t underneath the coordinate vector.0070

So, let us just write it out and start with that.0076

So, we have v + w, so if I add two vectors and take the coordinate with respect to a certain basis, well, I can treat that, I can just sort of separate them.0080

That is just going to be the coordinate vector with respect to s for v, + the coordinate vector with respect to s for w.0098

Again, it is something that you already know. The sum of two vectors is nothing new here.0106

If I have a vector v and I multiply by a constant, and if I have the coordinate vector with respect to a certain basis s, well, I can just go ahead and pull that constant out and multiply by the coordinate vector for that vector first, and then multiply by the constant.0112

So, just to have these properties to describe what it is that we are going to do in a minute.0130

Okay. I am actually going to go through something that I would not normally go through.0137

It is the derivation of where this thing called a transition matrix comes from, simply because I want you to see it.0141

It is not going to be particularly notationally intensive, but there are going to be indices, you know, there are some numbers and letters, things floating around.0148

So, it is really, really important to pay attention to where things are and what each number is doing.0155

Okay. Let us say we have two bases for a vector space that we call v, of course.0161

The first basis is going to be s, it is going to consist of vectors v1, v2, and so on all the way to vN.0180

And... we have t, which is another basis.0195

We will call these w... w1, w2, all the way to wN, and again they are bases for the same vector space, so they have the same number of vectors in them. That is the dimension of the vector space.0200

Now, choose some v in v, some random in the vector space.0215

Well, we can write this particular v with respect to this basis, let us choose this basis, t.0228

So, we can write... we can say that v = c1 × w1, c2 × w2, we have done this a thousand times... + cN × wN.0235

Okay, just a linear combination of the vectors in this basis t. Well, once I actually solve for these constants, c1, c2 through cN... what I end up with is the coordinate vector v with respect to the basis t.0252

That is what this t down here means, which is c1, c2, all the way to cN. Okay.0268

Now. Here is where it gets kind of interesting. So, let us watch very, very carefully. Let me put a little arrow right here to show what we are doing.0279

If I take v, and if I want to find the coordinate vector with respect to the basis s, I am just going to take this thing that I wrote, which is the ... so the left side, I put it in this notation... sub s.0290

Well, the left side is equal to the right side. I just happen to have written the right side with respect to this basis.0310

I am just going to write... I am basically just copying it.0317

c1w1, c2w2 + ... + cNwN, with respect to s. All of them is take this thing, and subjected it to this notation. Everything should be okay.0324

Well, now I am going to use these properties. So, that is equal to c1 × w1, with respect to s + c2 × w2, with respect to s + so on + cN × wN, with respect to s.0340

Okay. Take a look at what we have done. A random vector v with respect to a basis t, and then, we want to find the coordinate vectors for... with respect to the basis s.0368

So, I have just taken this definition, and subjected it to the notation for the coordinate vector for s.0384

Then I use these properties up here, which you might call the linearity properties of these coordinate vectors, and just rewrite it.0390

Well, let us just see what this actually says... c1 × w1 with respect to s, w2 with respect to s... wN with respect to s.0400

Let us move forward, that is just this.0409

It says that coordinate vector v, with respect to s is equal to 1s, w2 respect to s, and so on and so forth.0414

I just set up, well let me actually finish up writing it and then I will tell you what it is we are doing here.0434

wN with respect to s, × c1, c2, all the way to cN.0445

So, the equation that I wrote on the previous slide is just this equation in matrix form.0457

What I am doing is I am taking the columns, I am taking each w1 in the basis t, I am expressing that vector with respect to the basis s, and whatever I get I am putting in as columns of my matrix.0461

Well, what this ends up being... this matrix that I get by doing that is precisely... well let me rewrite it.0483

So, we have the equation in front of us... p s... notice the arrow is going from right to left, not usually from left to right... × this thing, which is just coordinates of v with respect to t.0496

Okay, so what we have done, this is our ultimate goal. It is okay if you do not completely understand what it is that we did.0520

We will go through the procedure for how to find this matrix. This matrix right here, which is called the transition matrix from t to s.0525

There is a reason why I wrote it this way with the arrow going backwards from right to left. I will tell you what it is in a second.0537

It says that if I have a vector, and if I can find the coordinate vector with respect to a basis t, but I want to convert that to a coordinate vector with respect to the other basis that I have, s, I can multiply the coordinate vector with the one basis on the left by some matrix.0543

The transition matrix that takes it from t to s. This is why it is written this way. Notice this vector space -- I am sorry -- this coordinate vector with respect to s is on the left.0566

Here, the notation for the transition matrix has the s on the left, has the t on the right, because you are multiplying it by the coordinate vector for the basis t on the right.0576

It is just a way of... again, it is a notational device to remind us that we are going from the t basis to the s basis.0588

It is written this way simply because of how we wrote the equation. We wrote the coordinate vector with respect to s on the left side of the equality sign. That is why it is written this way.0597

Now, here is how we did it. We have the basis t which consists of vector w1, w2, and w3 and so on.0609

We take each of those vectors, we express them as coordinate vectors with respect to s, and we do that by solving this system, just like we did for the previous lesson.0618

Then, the coordinate vectors that we get, we just put them in columns and the final matrix that we get when we just put in all of the columns, that is our transition matrix.0630

Okay. Let us just do an example and I think it will all make sense.0645

Let us move forward here. This is going to be a bit of a long example notationally, but it should be reasonably straight forward.0650

Okay, now, let s = the set (2,0,1), (1,2,0), (1,1,1).0659

Okay. That is one basis for R3. Three vectors, three entries, it is R3.0680

T, let it be another set, let it be (6,3,3), (4,-1,3), (5,5,2).0695

So, we have two different bases. Okay, what we want to do is a, we want to compute the transition matrix.0714

What matrix will allow us to convert from t basis to s basis? The transition matrix from t to s, that is the first thing we want to do.0725

The second thing we want to do is we want to verify the equation that we just wrote.0735

That the coordinate with respect to basis s is equal to this transition matrix, multiplied by the coordinate for v with respect to t.0740

Okay. So, let us see what we have got here. Alright, so let us do the first thing first.0755

Let us go ahead and compute this transition matrix. So, we said that in order to compute the transition matrix, we have to take... so we are going from t to s.0768

That means we take the vectors in the basis t, (6,3,3) (4,-1,3), (5,5,2), and we express each of these vectors with respect to the basis s.0779

Again, these are just vectors in R3. They are random vectors, but they do form a basis and that forms a basis.0791

So, we want to change, we want to be able to write these vectors as a linear combination, each of these as a linear combination of these 3. That is what we are doing.0796

So, let us write that down. So, we have... we will take this one first, right?0808

Let us actually label these. No, that is okay, we do not need to label them.0815

So, we want (6,3,3) to equal some constant a1 × (2,0,1) + a2 × (1,2,0).0824

Actually, let us not, let us choose a different letter here. Let us choose b1, and we will make this c1 × (1,1,1).0845

So, this is one of the things that we want. We can solve this system. We can just solve this column, this column, this... let me write it on the other side.0857

We are accustomed to seeing it on the right, let us go ahead and be consistent... (6,3,3). That is one thing.0865

Okay. The other thing we want to do, is we want to express this one. The second vector in the basis t.0876

Again, t to s, so we want to take the vectors in t, the second vector expressed as a linear combination of these two.0882

So, this time we have a different set of constants, we will call them a2 × (2,0,1) + b2 × (1,2,0) + c2 × (1,1,1) = (4,-1,3).0890

Now we want to express the third vector in t as a linear combination of these.0908

So, we will take a3 ×, well, (2,0,1) + b3 × (1,2,0) + c3 × (1,1,1).0914

That is going to equal (5,5,2). So, we solve this system, we solve this system, we solve this system.0932

Well, this system, for each of these, the left hand side, these columns are the same, (2,0,1), (1,2,0), (1,1,1).0941

So, we can take all three of these and do them simultaneously. Here is what it looks like.0948

All we are doing is taking (2,0,1), (1,2,0), and (1,1,1) and then we are augmenting the (6,3,3).0959

(2,0,1), (1,2,0), (1,1,1), augmenting it with (4,-1,3).0965

And this one... (2,0,1), (1,2,0), (1,1,1)... augmenting it with (5,5,2).0970

Well, we can do all of the augmentations simultaneously. We can just add three columns and then do our matrix in reduced row echelon form. Here is what it looks like.0974

So, we get (2,0,1), (1,2,0), (1,1,1), and then we have our augmented... we have (6,3,3), (4,-1,3), and we have (5,5,2). Okay.0985

When we subject this to reduced row echelon form, let me go horizontally actually, we end up with the following.1007

We end up with (1,0,0), (0,1,0), (0,0,1), and we end up with (2,1,1), (2,-1,1), (1,2,1).1016

So, this, right here, in the red -- oh, I did not get red, oops -- right here, that is our transition matrix.1035

It is the columns of the vectors in t, expressed as... these are the coordinates of those vectors with respect to the s basis.1046

That is what we did, just like the previous lesson, so our transition matrix from t to s = (2,1,1), (2,-1,1), (1,2,1).1058

There you go. That is the first part. Okay.1074

Now, we want to confirm that that equation is actually true. In other words, we want to confirm this equation.1077

That the coordinate vector of some random vector v with respect to s is equal to this transition matrix that we just found × the coordinate vector with respect to the basis t.1095

Okay. Well, let us let v... well, let us choose a random vector. We will let v - (4,-9,5).1105

Okay, so now the first thing that we want to do is... again we are verifying so we are doing a left hand side, we are going to do a right-hand side.1120

We are verifying this. We need to check to see if this is actually equal. So, we need to do this side, and we need to do this side.1128

Okay. First of all, let us find the coordinate of this vector with respect to t, that is this right here. Okay.1136

Well, we need to set up the following. c1w1 + c2w2 + c3w3 = our (4,-9,5).1148

Well, let us take our columns, which are our basis t, so we get the following. We get (6,3,3), (4,-1,3), (5,5,2).1171

It is going to be (4,-9,5).1187

Convert to reduced row echelon form, we get (1,0,0), (0,1,0), (0,0,1), and we get (1,2,-2). Okay.1194

So, the coordinates of v with respect to the basis t is equal to (1,2,-2). That is part of the right hand side.1209

Well, we have the transition matrix, that is this, so let us circle what we have. We have that. that is our coordinate with respect to t.1223

We have our transition matrix, so we have the right-hand side. Now, we need to find the left hand side, do a multiplication, and see if they are actually equal to confirm that equation.1232

Okay. Now, let us move to the next page, so we want to find with respect to s. Well, with respect to s we set up the columns from the vectors in the basis s.1241

So, we get (2,0,1), (1,2,0), (1,1,1), and we are solving for (4,-9,5).1261

Reduced row echelon, when you do that, you end up with... I will actually write that out... and I put (4,-5,1)... so now we have the left hand side.1276

Now, what we want to do is we want to check, is (4,-5,1)... does it equal that transition matrix × the coordinate vector with respect to... yes, as it turns out, when I do the multiplication on the right hand side, I end up with (4,-5,1).1293

So, yes. It is verified.1329

So again, our equation is this. If I have some coordinate vector with respect to a basis t, and I want to find the coordinates with respect to another basis s, I multiply on the left with something called the transition matrix.1333

That will give me the coordinates with respect to s, and the columns of that transition matrix are the individual basis vectors for the basis t expressed as coordinate vectors with respect to the basis, s.1353

That is what this notation tells me. Okay.1372

Now, that is exactly what we did. We were given two bases, t and s, we took the basis, the vectors in the basis from t, we expressed them as coordinate vectors with respect to the basis s, and that which we got, we set up as columns in a matrix.1379

That matrix that we get is the transition matrix. That allows us to go from 1 basis to another, given one coordinate, or another.1405

Okay. Let us see. Let us continue with a theorem here.1417

s = v1, v2... vN. And, t = w1, w2... wN. Okay.1431

Let s and t be two bases for an n-dimensional vector space. Okay.1456

If p, from t to s, is the transition matrix... transition matrix from t to s... then, the inverse of that transition matrix from t to s is the transition matrix from s to t.1471

So, if I have 2 bases, and if I calculate a transition matrix from t to s, I can take that matrix, I can take the inverse of that matrix, and that is going to be the transition matrix from s to t.1526

So, I do not have to calculate it separately. I can if I want to, but really all I have to do is take the inverse of the matrix that I found.1539

That is the relationship between these two. Okay.1546

Also, the transition matrix which we found is non-singular.1552

Of course, invertible. Non-singular means invertible. Okay.1566

Okay. Let us see what we have got here.1576

So, let us continue with our previous example. Let us recall the conditions... we said that s is equal to (2,0,1), (1,2,0), (1,1,1).1580

And... t is equal to (6,3,3), (4,-1,3), and (5,5,2).1605

Okay. What we want to do is we want to compute the transition matrix from s to t, directly.1618

So, we can do it directly, and the other way we can do it is to take the inverse of the transition matrix from t to s that we already found. That is going to be the second part of this.1632

So, the first part a will be computed directly, and the second part, we want to show that this thing from s to t is actually equal to the inverse of the matrix from t to s.1647

Make sure you look at these very, very carefully to make sure you actually know which direction we are going in.1661

Well, in order to calculate it directly, we take the... so this, we are going from s to t, alright?1666

So, let us go with vectors in s... so we are going to write (6,3,3), so in other words, we are going to express... so this is from s to t.1680

So, we want to take the vectors in s and express them as a linear combination of these vectors.1700

These vectors are the ones that actually form the matrix over here... (6,3,3), (4,-1,3), (5,5,2), and we augment with the (2,0,1), (1,2,0), (1,1,1).1706

Again, we are going from s to t. We want to express the vectors in s as linear combinations of these. That is why s is on the augmented side, and t is over on this side. Okay?1725

These are three linear equations. This, this augment, that, that augment, this, this augment. Okay?1738

When we subject to reduce row echelon form, we end up with (1,0,0), (0,1,0), (0,0,1).1749

We end up with -- nope, we do not end up with stray lines -- we have (3/2,-1/2,-1,1/2,-1/2,0,-5/2,3/2,2) make these as clear as possible.1760

Therefore, q transition matrix from s to t is equal to (3/2,-1/2,-1,1/1,-1/2,0,-5/2,3/2,2).1793

Okay. So, that is the direct computation of u(s(t).1819

Now, let us go back to blue. Now, let us calculate the inverse to show that that equals the inverse of that. Yes.1827

Okay. So, now, let us see. We want to take, in order to find... so we have... let us recall what the transition matrix from t to s was.1844

We had (2,1,1), (2,-1,1), (1,2,1), okay? That was our transition matrix.1859

Now, you recall, when you actually, in order to find the inverse of the matrix, you set up this system... (2,1,1), (2,-1,-1), (1,2,1).1881

Then, you put the identity matrix here, (0,1,0), (1,0,0)... yes.1898

Then, of course, if you do reduced row echelon form, this right side ends up being the inverse of that.1910

In this particular case, we do not need to do that. If we say that one thing is the inverse of another, all that I have to really do is multiply them and see if I end up with the identity matrix.1915

So, part b, in order to confirm that that is the case, all I have to do is I have to take the transition matrix of t from s to t, and I multiply by the matrix from t to s, to see if I get the identity matrix.1926

While I do that, I have of course my (3/2,-1/2,-1,1/2,-1/2,0,-5/2,3/2,2).1949

Multiply that by our transition matrix (2,1,1), (2,-1,-1), (1,2,1), and I can only hope I have not messed up my minus signs or anything like that.1970

As it turns out, when I do this, I get (1,0,0), (0,1,0), (0,0,1), which is the identity matrix - n-dimensional, the 3 by 3 identity matrix, so this is not n, this is 3.1982

That confirms that q, s to t = inverse of t to s.2000

When I find a transition matrix from t to s, if I want the transition matrix from s to t, all I do is take the inverse. That is what we have done here.2014

Thank you for joining us Educator.com to discuss transition matrices, we will see you next time.2023