Linear Algebra Similar Matrices & Diagonalization

Welcome back to Educator.com and welcome back to linear algebra.0000

In our previous lesson, we introduced the notion of Eigenvector and Eigenvalue.0004

Again, very, very profoundly profoundly important concepts throughout mathematics and science.0009

Today, we are going to dig a little bit deeper and we are going to introduce the notion of similar matrices and the idea of diagonalization.0015

So, let us jump right on in. Let us start with a definition... Let me go to blue ink here.0025

Okay. A matrix b is said to be similar to matrix a if there is a non-singular matrix, p, such that... let us see what this definition says.0036

If I have some matrix a, and I find some other matrix p, and if I multiply on the left of a by p inverse, and on the right by p, so if I take p inverse a × p, the matrix that I get, b, I say that b is similar to a.0083

So, there is a relationship that exists between, if I can actually sandwich this matrix a between some matrix p and the inverse of p.0102

In the course of this lesson, we are going to talk to you actually about how to find this matrix p, and about what this matrix b looks like. Really, really quite beautiful.0109

Okay. A quick example just so you see what this looks like in real life. So, if I let the matrix a equal (1,1,-2,4), just a little 2 by 2... and if I say I had p which is (1,1,1,2), well, if I calculate the inverse of p, that is going to equal (2,-1,-1,1)... okay?0120

Now, as it turns out, if I take b, if I actually do p inverse × a × p so I multiply that by that, then by that, I end up with the following... I end up with the matrix (2,0,0,3).0149

Now, what you might want to do is take a look at the previous lesson. This matrix a, you remember, and I will discuss this in a minute... just to let you know what is coming up, what is coming ahead, this matrix a, we have dealt with this matrix before in the previous lesson.0170

We found the Eigenvalues for it. The Eigenvalues were 2 and 3. Well, notice what we did.0183

We found this matrix p... and we took the inverse of that, we multiplied p inverse a p, and we end up with a diagonal matrix, where the entries on the diagonal are exactly the Eigenvalues of a. That is what is going to end up being so beautiful.0189

Here is what is even better. If you remember the two Eigenvectors that we found for the two Eigenvalues 2 and 3 were exactly of the form (1,1,1,2).0203

So, this matrix p is going to end up being made up of the actual Eigenvectors for the Eigenvalues. We will just... a little preview of what is coming.0212

Okay. Just some quick properties of similarity. So, the first property is a is similar to a, of course... intuitively clear.0223

If b is similar to a, then a is similar to b. That just means that... we can always multiply on the left by p, here, and p × p inverse, this goes away, and we multiply on the right by p inverse so this is just the same.0244

And three... If a is similar to b, and b is similar to c, then, by transitivity, c is similar to a... I am sorry, a is similar to c, which means that c is similar to a by property too... then a is similar to c.0262

So, standard properties... we will be using those in a second. Another definition. We say matrix a is diagonalizable... did I spell that correct?... diagonalizable... if it is similar to a diagonal matrix.0293

In this case, we say a can be diagonalized -- put a comma there, in this case we say... a can be diagonalized.0344

Okay. Now, let us see what we have got. Alright. Profoundly, profoundly, profoundly important theorem.0367

An n by n matrix is diagonalizable if, and only if, it has n linearly independent Eigenvectors.0387

In this case, a is similar to a diagonal matrix d, where d is equal to p inverse a... p... and the diagonal elements of p, whose diagonal elements are the Eigenvalues of a.0423

It is like we did before. It is similar to the diagonal matrix b, and the entries on that diagonal are precisely the Eigenvalues of a.0468

Well, p is the matrix whose columns respectively... it means respective to the individual Eigenvalues... so Eigenvalue Λ1 gives column 1, Λ2 gives column 2, Λ3 gives column 3... so on and so forth, respectively.0481

Each of these columns, respectively, are the n linearly independent Eigenvectors of a.0512

So, again, this is what we did in our example. So now we state it as a theorem... an n by n matrix is diagonalizable if and only if it has n linearly independent Eigenvectors.0529

So, if I have a 3 by 3, I need 3 Eigenvectors. If I have a 5 by 5, I need 5 linearly independent Eigenvectors.0542

In this case, the a is similar to a diagonal matrix d, whose diagonal elements are precisely the Eigenvalues of a, and the matrix p in this relation here, is the matrix made up of the columns that are the n independent, linearly independent Eigenvectors of a.0548

So, from a, I can derive the matrix d, I can derive the matrix p, and the relationship is precisely this.0570

Let us just do an example here. Okay. We will let a equal the matrix (1,2,3)... (0,1,0)... (2,1,2).0577

I am not going to go through the entire process, again I use mathematical software to do this... to find the Eigenvalues and to find the Eigenvectors.0594

Here is how it works out. As it turns out, one of the Eigenvalues is 4. 4 generates the following Eigenvector, when I do solve the homogenous system, I get (1,0,1).0602

A second Eigenvalue is -1. Also real. It generates the Eigenvalue... the Eigenvector (3,0,2). The third Eigenvalue is equal to 1, all distinct, all real, and it generates 1 - 6 and 4, when I solve the homogeneous system.0619

Therefore, p = (1,0,1), (-3,0,2), (1,-6,4).0651

If I want to define p inverse, which I can, it is not a problem -- you know what I will go ahead and write it out here... it is not going to be too big of an issue -- I have (2/5, -1/5, 0), (7/15, 1/10, -1/6), (3/5, -1/5, 0), and of course my diagonal matrix d is going to end up being (4,0,0), (0,-1,0), (0,0,1).0667

If I were to confirm... yes, I would find out that d does in fact equal p inverse × a × p. Excuse me... found the Eigenvalues... found the associated Eigenvector...put those Eigenvectors... these are linearly independent, put them as columns in p... if I take p, multiply and find p inverse, if I multiply a by p inverse on the left, p on the right, I end up with a diagonal matrix where the entries on the main diagonal are precisely the Eigenvalues (4,-1,1), (4,-1,1).0714

So, 4 is the first, that is why its Eigenvector is the first column, that is what we meant by respectively.0753

Okay. Now, if all the roots of the characteristic polynomial, which is what we solve to find the Eigenvalues, of a are real and distinct... if the roots of the characteristic polynomial are real and distinct... in other words if the Eigenvalues of the matrix are real and distinct, then a is diagonalizable. Always.0761

Note how we wrote this. If all of the roots of the characteristic polynomial are real and distinct, then a is diagonalizable. There is no if and only if here.0812

That does not mean that if a is diagonalizable, that the roots of the characteristic polynomial, the Eigenvalues are real and distinct.0822

It is possible for a to be diagonalizable and have roots that are not distinct. You might have an Eigenvalue, you might have a 4 by 4, and you might have an Eigenvalue 4, and then (1,1,1). That 1 might be an Eigenvalue 3 times over, but it will still be diagonalizable.0829

So, again. If then does not mean that it works the same backwards. It is not the same. It is not the same as if and only... if and only if means it goes both ways.0846

So, if the roots of the polynomial are real and distinct. If it is diagonalizable, the Eigenvalues may or may not be real and distinct.0855

Okay. Now, the characteristic polynomial... so the characteristic poly for non-distinct Eigenvalues looks like this.0866

Well, we know we are dealing with some characteristic polynomial... Λ³ + something Λ², lambda;ⁿ.0890

Well, every time we find a value, we can factor... that is what the fundamental theorem of algebra says... every polynomial can be factored into linear factors.0899

For the non-distinct, we end up with something like this... Λ - Λ_i to the k_i power.0910

So, if some root ends up showing up 5 times, that means that I have 5 factors for that root. Λ - 1, Λ - 1, Λ - 1, Λ - 1, Λ - 1, well, that factor is Λ - Λ_i... you know, to this power. Okay.0917

This is Λ1, this is Λ 2, in the case of 2, in the case of i... Okay.0939

Well, this k_i is called multiplicity of the Eigenvalue... Λ_i0974

It can be shown that if the Eigenvalues of a are all real and distinct, in this case we already dealt with the fact they are distinct, we know they are diagonalizable if they are all real and distinct.0977

So, we can also show, now that we have introduced this idea of multiplicity... so if our characteristic polynomial has multiple roots, so if I have a fourth degree equation, one of the roots is 4, the other root is 3, and the other roots are 2 and 2.1019

Well, 4 has a multiplicity 1, 3 has a multiplicity 1, 2, because it shows up twice, has a multiplicity 2.1032

It can be shown that if the Eigenvalues of a are all real, then a can be diagonalized, it can still be diagonalized if and only if for each Λ_i of multiplicity k_i, we can find... if we can find k_i linearly independent Eigenvectors.1038

This means that the null space for that thing that we solved, L_i, i_n - a, × x = 0, this means that the null space of that equation that we solved to find the Eigenvectors has dimension k_i.1088

In other words, if I have a root of the characteristic polynomial, an Eigenvalue that has a multiplicity 3... let us say its 1 shows up 3 times.1118

Well, when I put it into the equation, this homogeneous equation, if I can actually find 3 linearly independent vectors... if I can find that the dimension of that null space is 3, I can diagonalize that matrix.1128

If not, I cannot diagonalize that matrix. Okay.1145

So, let us see what we have got. You will let a = (0,0,1), (0,1,2), (0,0,1).1150

That is interesting... okay.1169

When we take the characteristic polynomial of this, we end up with the following. Λ × (Λ - 1)²... so we have λ1 = 0, Λ2 = 1, so one has a multiplicity of 2, and Λ3 is also equal to 1.1173

Well, let us just deal with this multiplicity of 1 Eigenvalue. When we solved the homogeneous system, we end up with the following. We end up finding that the Eigenvector x is this... (0,r,0).1195

Well, this is only 1 vector. Here we have 2 Eigenvalues, multiplicity is 2. Well, in order for this matrix to be diagonalizable, I have to... when I solve that homogeneous system to find the actual Eigenvectors, I need 2 vectors, not just the 1. This is only one vector so this is not diagonalizable.1214

Now, mind you, it still has Eigenvalues, and it still has an associated Eigenvector, but it is not diagonalizable. I cannot find some matrix that satisfies that other property.1240

Okay. So, now, let us try this one. Let us let a equal to (0,0,0), (0,1,0), (1,0,1).1255

Well, as it turns out, this characteristic polynomial is also Λ × (Λ - 1)².1272

So, again we have Λ = 0, Λ2 = 1, Λ3 = 1, so our Eigenvalue 1 has a multiplicity of 2, so we want to find 2 Eigenvectors.1280

We want to find the dimension of this null space associated with this Eigenvalue if it has a dimension 2, we are good. We can actually diagonalize this.1294

Let us see what we have. Let us see. When we solve the associated homogeneous system, we end up with (1,0,0,0), (0,0,0,0), (-1,0,0,0).1302

When we subject this to reduced row echelon form, it becomes (1,0,0,0), (0,0,0), (0,0,0,0)... so we have... there we go... we have x1 = 0, we have x2 = let us see, this is x2, x3, 3 parameters equals r... x3 = s, and we can rewrite this as equal to r × (0,1,0) + s × (0,0,1), so there you go.1326

We have two Eigenvectors, a basis... our basis for that null space has 2 Eigenvectors. It is of dimension 2. It matches the multiplicity of the Eigenvalue, therefore this can be diagonalized.1375

So, let us go ahead and actually finish the diagonalization process... when I go back and solve for the Eigenvalue Λ1 = 0, for that Eigenvalue, I get the following vector... r0 - r, as a general, and its specific would be -- let us see -- (1,0,-1).1392

Therefore our matrix p would be (1,0,-1), (0,1,0), (0,0,1). This is our matrix p.1417

It is of course diagonalizable. Matrix d is going to end up being (0,0,0), (0,1,0), (0,0,1). These along the main diagonal are the Eigenvalues of our matrix. This is our matrix p. We can find the inverse for it, and when we multiply we will find that d, in fact, equals p inverse, a p.1428

So, now, procedure for diagonalizing matrix a, this is going to be our recap. First thing we want to do is form the characteristic polynomial, which is symbolized also with f(Λ) equals the determinant of Λ × i_n... Λ is our variable... Λ times the identity matrix minus a.1458

That is our polynomial. Then, we find the roots of the characteristic polynomial... okay?1502

If not all real, if they are not all real, then you can stop... it cannot be diagonalized.1517

Okay. Three. For each Eigenvalue, Λ_i of multiplicity k_i... find a basis for the null space of Λ_i, i_n - a × x = 0.1537

So, for each Eigenvalue Λ_i of multiplicity k_i, find a basis for the null space... also called the Eigenspace.1581

If the dimension... the dimension that we just found... the dimension of this null space that we just found is less than the k_i, then you can stop. a cannot be diagonalized.1594

If so, well, then, let p be the matrix whose columns are the n linearly independent Eigenvectors... found above... then, the inverse a × p = d, where d has Λ_i along the diagonals. The diagonal entries are the Eigenvalues.1622

Okay. So, let us recap. We want to form the characteristic polynomial once we are given a matrix a, n by n.1697

Once we find the characteristic polynomial, we want to find its roots. If the roots are not all real, if there is... they have to all be real.1704

If they are not all real, then you can stop. You cannot diagonalize the matrix.1713

If they are real, well, for each Eigenvalue of multiplicity... you know k_i, for each one that has a multiplicity, you want to find a basis for the null space of that equation... we solve the homogeneous system.1718

Well, if the dimension of that null space is equal to k_i, we can continue. If not, we can stop, we cannot diagonalize the matrix.1735

But, if we can, and for each distinct Eigenvalue, we are going to have 1 Eigenvector. We will let... take those n linearly independent Eigenvectors that we just found, we will arrange them as columns respectively.1742

So Λ1, column 1, Λ2, column 2... that is going to be our matrix p. When we take p inverse × a × p, that actually is equal to our diagonal matrix d, the entries of which are the respective Eigenvalues of our original matrix a.1757

So, we will be dealing more with Eigenvalues and Eigenvectors in our next lesson, so we are not quite finished with this, it gets a little bit deeper. We will have a little more exposure with it.1780

So, until then, thank you for joining us at Educator.com, and we will see you next time.1791