Linear Algebra Inverse of a Matrix

Welcome back to educator.com, this is linear algebra, and today we are going to be talking about finding the inverse of a matrix.0000

The inverse is kind of analogous to a reciprocal as far as the real numbers are concerned, but again like matrices with respect to real numbers.0009

A lot of the things carry over as you remember from some of the properties like of commutativity of addition, distribution, things like that.0020

But certain properties don't carry over, so we can certainly think about it, you know analogously, if we want to, but we definitely want to understand that matrices and real numbers are very different mathematical objects, even though they do have certain things in common.0026

Okay, let's go ahead and get started, so finding the inverse of a matrix, let's start off with a definition so...0039

...An N by N matrix...0052

... A is called non-singular...0057

... Or invertible...0069

... If there exists -- and remember, this reverse E, it means there exists...0076

... An N by N...0083

.... Matrix, B such that A times B = B times A = the N by N identity matrix, which remember the identity matrix is that N by N matrix, where everything on the main diagonal is a 1.0087

The analogy as far as real numbers are concerned, it is kind of like taking the number 2 and multiplying it by 1 half.0107

You get 1, or 1 half times 2, you get 1, because the 2's cancel.0113

Well 2 and 1 half are in some sense inverses of each other, so when you multiply them, you get the identity for the numbers, the real numbers which is 1.0117

Well the analogous identity for matrices is the one along the main diagonal, so this is the definition in N by N matrix, A is called non-singular or invertible.0126

If there exist an N by N matrix B such that this holds, A times B = B times A, gives you the identity matrix.0136

Okay, so B is called the inverse...0144

... A, oh, we will quickly, both are these terms are used interchangeably, sometimes I am going to use non-singular, sometimes I am going to use invertible.0156

To be perfectly honest with you to this day for me, non-singular, it always takes me a couple of seconds to remember what that actually means, so when we say non-singular, we mean that it's invertible, that means inverse actually exist.0165

You are going to run across in a minute.0179

Matrices that are singular, which means that they are non-invertible, which means that inverse doesn't exist, so again you are welcome to use, each one, we will be using both interchangeably and eventually, I think you will just be comfortable with the little one, okay...0181

... And as we just said, if no such matrix exists...0200

... Oops...0210

... Then A is singular...0214

... Or, non-invertible, I think some of the confusion comes from the fact that sometimes we use non-singular, and then invertible, and singular, non-invertible.0221

Okay, so let's just take an example, a nice little 2 by 2, we have (2, 3, 2, 2).0233

This matrix right here, and again use mathematical software, it gives you the inverse, just like this, so B happens to be (-1, 3 halves, 1, -1), so there are two matrices A and B.0240

Well, when we actually multiply A times B and when we multiply B times A, and remember matrix multiplication does not commute, so they are not necessarily equal, but in this case AB does equal Ba, and they both happen to equal the identity matrix, which is equivalent to this thing (0, 1).0258

Again a matrix with 1's along the main diagonal, okay if...0280

... A matrix has an inverse...0291

... The inverse is unique, again you can't have two or three or four different inverses, you only have one.0302

We won't prove this, but it is a very, actually it's a rather quick proof, but we won't worry about that we are concerned with the using this idea as supposed to proving it.0312

Okay, let's talk a little bit about notation...0321

... We want to denote...0328

... The inverse of A as A with the little -1 as a superscript. NB, which means nota bene, which means notice this very carefully.0336

This is symbolic, okay...0353

What that means, this A^-1, it's a symbol, this A^-1 does not mean 1 over A, this doesn't work for matrices, it's not defined, this is strictly a symbol that we use.0360

Sure, you are used to seeing numbers like 2^-1, which is equivalent to 1 half, you just flip it.0374

That's not the same here, we use the same symbolism, but it is only symbolic, it doesn't mean take 1 an divide by a matrix.0381

Division by a matrix is not defined, it's not even something that we can deal with, but so bear that in mind....0386

... Excuse me, now let's just take a couple of properties of non-singular matrices, and again non-singular means invertible, once that actually have an inverse...0398

... And we call again, we are talking about square matrices, N by N, 2 by 2, 3 by 3, 4 by 4 and so on, we don't speak of inverses of other matrices.0416

Okay, property A, if I have the inverse and if I take the inverse of the inverse, I recover A, which makes sense, you take the inverse, you take the inverse again, you are back where you started, which is actually the definition of inverse.0426

It works in a circle, if you remember dealing with inverse functions, it works the same way.0440

B, if I take two matrices A and B and multiply then and then take the inverse, I can actually get the same thing if I take the inverse of B first, multiply by the inverse of A, and notice the order here, this is very important.0448

Just like with the transpose, when we did A times B transpose, that's equal to B transpose times A transpose, the same thing here.0469

Those of you that are actually working from a book are interested in actually seeing the proof of this, I would encourage you to take a look at it, again the proof is not complicated, it's just a little tedious in the sense that you are dealing with every individual, little detail.0479

It's easy to follow, it's just arithmetic, but it's sort of interesting to see how something which is not very intuitive, would actually end up looking like this, so make sure that the order is correct.0493

We also have a B prime, which is just the same thing for multiple entries, so if I have for example, A times B times, C times D, so on.0505

Inverse, well I just reverse, I'll just do it backwards, that's equal to D inverse times C inverse, B inverse, A inverse, just work your way backwards, just like the transpose.0518

And see our final property, if we take a matrix an take the transpose of it, and then take the inverse of it, well what we can do is just take the inverse first and then take the transpose.0532

In other words, the transpose and the inverse are switchable, okay.0545

Let's see what we can do, we want to find a practical procedure for finding the inverse of any given matrix.0553

And here it is, it's actually very simple, it's something that we have already done, we are going to be using Gauss Jordan elimination again, we are going to be doing reduced row echelon form, except now, we are going to put two matrices next to each other.0561

We are going to be doing it simultaneously, and then the one on the right that we end up getting will actually be our inverse, it's really quite beautiful.0572

Step 1, form and don't worry if this procedure as I write it out doesn't really make much sense, when you see the example, it will be perfectly clear.0583

Form the N by 2N matrix...0595

... A augmented by the identity matrix..0603

... Step 2...0614

... Transform...0620

... The augmented matrix...0625

... To reduced row echelon form, this entire matrix here., transform the entire thing to reduced row echelon again using mathematical software.0631

I can tell you how wonderful mathematical software is it, it has made life so wonderful, it's amazing...0641

... Final 3, now you have couple of possibilities, suppose after converting it to reduced row echelon form, you have produced...0650

... The following matrix, C, D, so basically after conversion, A has been converted to C, this identity matrix has been converted to D, here are the possibilities.0664

If C turns out to be the identity matrix itself, then D is your inverse.0679

Really all we have done, we have taken the original matrix, put the identity matrix next to it, and then we have reduced to, we have done a reduced row echelon form.0691

Well it converts, if the, if the inverse actually exists, a, this thing becomes the identity and the identity matrix becomes the inverse.0699

And B...0711

... If C doesn't equal the identity matrix...0714

... The C has a row of 0's...0721

... And this implies that A inverse does not exist...0731

... Okay so we have formed the N by 2N matrix, we take the matrix, put the identity matrix next to it, we transform it to reduced row echelon form.0744

If this happens to be the identity matrix, then our matrix D is our inverse, we are done, if it's not the identity matrix, one of the rows will actually be all 0's that means the matrix, that means the inverse doesn't exist, let's do some example...0752

... Okay, we want to find the inverse of A, so let's do A = 1, 1, 1, 0, 2, 3, 5, 5, 1), okay, so step 1, we want to go ahead and form the augmented matrix.0771

We take the, we just do al the whole line here, so we go (1, 1, 1, 0, 2, 3, 5, 5, 5), and we are going to augment it with the 3 by 3 identity matrix...0789

... (0, 0, 1), which is just 1's along the main diagonal, let me go ahead and ut brackets around this, and then we convert to reduced row echelon form, let me, let me go down here.0807

We run our math’s software, and when we end up with is, and again reduced row echelon is unique, you get (1, 0, 0, 0, 1, 0, 0, 0, 1) and over here you will get some fractions.0821

13 eights -1 half - 1 eighth -15 eighths, 1 half, eighths, 5 fourths, you get a 0 and you get -1 fourth.0837

Sure enough, now we ask ourselves, if this, the identity matrix it is 1 along the main diagonals, everything else is 0, it is 3 by 3, so the inverse exist.0855

Not only does the inverse exist, there is your inverse, so we have done the existence and the process itself gives us our inverse, so A inverse = well I am not going to write it out, but.0869

Again, that's your matrix, 13 eighths - 1 half - 1 eighths - 15 eighths, 1 half, 3 eights, 5 fourths, 0 and -1 fourth.0881

That means that A, the original matrix times this gives me the identity matrix, an this times that gives me the identity matrix, these are inverses of each other, okay.0890

Lets do another example...0902

... This time we will take A = (1, 2, -3), (1, -2, 1), (5, -2, -3) okay, I am just going to go ahead and augment it already to the right again this is 3 by , so we have (1, 0, 0) , (0, 1, 0) , (0, 0, 1).0906

We will subject it to reduced row echelon, when we do that, what we get is the following matrix, (1, 2, -3, -0, -4, 4, 0, 0,) and we get (1 0, 0, -1, 1, 0, -2, -, 0) this...0930

... Is not I3, that is not the identity matrix, therefore A inverse does not exist...0961

... This is actually kind of amazing to think that you can just sort of pick a collection of numbers and arrange them in a square, sometimes an inverse exist for it and sometimes it doesn't, by virtue of the actual identity of the numbers.0976

Just as an aside, there is some really strange and beautiful mathematics going on here, so every once in a while that's nice and sort of pull back away from the computation, away from the practicality of what you are doing and think about some.0992

This is illicitly leading to very deep fundamental truths about nature and how nature operates, and about the things which exist and the things which don't, so that's what ultimately makes mathematics beautiful, in addition to, of course, it's practical value, okay.1003

Let's talk about linear systems and inverses, so we dealt with matrices, inverses, now let's associate it with linear systems, because again ultimately we are going to deal with linear systems.1022

Okay, so let's write a few things down here...1032

... If A is N by N, then Ax = B...1038

.... Is a system of...1055

... Any equations in N unknowns...1062

Let's take just an example of N =3 as supposed to doing it in its most general case, so we have, so you remember this is the matrix in vector representation of a linear system.1069

We can take a matrix, multiply by the vector X, the vector variables, which is just A, and in this particular case maybe a # by 1, and it's equal to a 3 By 1 vector and a vector is just that thing what the, its just a 1 by N matrix in other words.1086

We have something that actually looks like this, A₁₁, A₁₂, A₁₃, A₂₁, A₂₂, A₂₃, A₃₁, A₃₂, A₃₃, and of ‘course these are just the entries, the first number is the row, second number is a column.1104

Just as a quick review, times X₁, X₂, X₃, here variable vector equals B₁, B₂, B₃.1123

This is a symbolic representation, short-hand notations will of the entire system, that's what it actually looks like when you spread it out, this is our quick way of talking about it.1135

Now let's do something with this to see what happens, let's rewrite it again, we have, let's write it over here, AX = B, okay.1144

Now we just talked about inverses, so presuming that A actually has an inverse, well then inverse is just another N by N matrix, so we can multiply by A, so let's go ahead and multiply by A inverse on the left hand side.1156

And of course in a equality, anything I do to the left hand side, I have to do to the right hand side to retain the equality, so let's multiply both sides by the inverse, so i end up with something like this.1172

A inverse, times AX = well, A inverse times B, well properties of matrices, this times this times that, associative, so why don't i just associate these two.1185

I can write this as A inverse times A, put those together times X = A inverse times B, well A inverse times A, it's just the identity matrix, so it's identity matrix = A inverse times B.1199

And the identity matrix times something, it just is the identity matrix, it gives you that thing back, so identity matrix times X, just gives you X = A inverse times B.1221

Stop and take a look at that one for a second, okay, so if A is non-singular, we have discovered a way of actually finding the unique solution for this, for the variables.1238

It's equal to, well if I take this and if I just multiply on the left by the inverse of the original matrix, the coefficient matrix, I actually find the solution, XS = A inverse times B.1253

So just by using the inverse in standard mathematical manipulation that we are all familiar with, we have actually come up with a way of finding an unique solution for this.1262

Let's actually list this as a theorem, linear systems and inverses, so...1276

... If A is N by N, actually this theorem that I am going to list is four homogeneous systems, and will, and a little bit will actually talk about non-homogeneous systems where the right hand side actually does have a vector B, not just all 0's.1288

Then the homogeneous system X = 0, and remember this is the 0 matrix, all 0's, 0 vector I mean is that al the entries are 0's, has a...1313

... Non- trivial solution...1335

... If...1341

... And only if A is singular, and we remember singular means non-invertible...1348

... For a homogeneous system, notice what we did before for the AX = B, we notice that if we have a system AX = B, if A is non-singular, meaning if it is invertible, if the inverse exist, we can use the inverse to actually find the solution X by just multiplying B on the left hand side to the left of B by that inverse matrix.1363

For the homogeneous system, it's actually different, for the homogeneous when the solution set is all 0's, then it has a non-trivial solution if and only if A is singular.1384

In other words for the homogeneous system, if A, if the inverse doesn't exist, then I can conclude that the system has a solution, this, if and only if, we will actually see it a lot in mathematics, and all this means is that you will see it symbolized like this as a little aside.1397

It just means its equivalent to, what this actually says is that it, that this solution has a non-trivial solution means that A is non-singular, or it means that A is not, if A is singular or if A is singular, then it has a non-trivial solution, so in other words it goes both ways1416

Lets do an example, let me draw a little line here, okay...1439

... Let's consider the linear system, do it in matrix form, (1, 2, -3, 0) and we will do the augment here, to show that we are talking specifically about a homogeneous system, (0, 5, -2, -3, 0), okay.1448

This says X + 2Y - 3Z = 0, 1 - X - 2I + 3Z = 0, 5Z - 2I - 3Z = 0, okay let’s see if we can find the inverse.1469

When we form the augmented matrix, we have, let's do (1, 2, -3, 1, 0, 0, 1, -2, 1, 5, -2, -3, 0, 1, 0, 0, 0, 1) and then we subject it to...1484

... Reduced row echelon form, let me move it over here, we end up with the following (1, 0, -1, 0, 1, -1, 0, 0, 0), we don't even have to worry about the other entries, it actually is not relevant.1509

Simply because we notice here we don't have the identity matrix, therefore we don't have the identity matrix, that means that A inverse does not exist...1528

...Which means that it is singular, and according to our theorem, if it's singular, that means that this solution, this system does have a solution, so again...1541

... Singular, non- invertible, singular, it's non-invertible, that means that this has a solution, okay different than the other way, so this is very unusual when all of this, when everything on the right hand side is equal to a 0, the matrix has to be non-invertible for there to be a solution.1557

Where else if there are a series of numbers here on this side, we need it to be non-singular; we need to be able to find an inverse in order to be able to find a solution for it, okay...1576

Okay, so...1593

Let...1599

Okay, so now let's talk about some theorems and something which is going to, something called a list of non-singular equivalences of this list is going to be very important for us, to grow up a list of linear algebra, we are going to be adding to the list, then it's actually going to get quite long.1604

And again this list of non-singular equivalences allows us to move back and forth between things that are equivalent, when I know something about a system, I look through this list, I can tell you something else about that system.1621

Let's start with a theorem first though, so we have, if A is an N by N matrix, then A is non-singular or invertible if and only if the linear system AX = B has a unique solution.1634

That was the thing that we did when we multiplied on the left by the inverse, so this if and only if, again, it just means that it goes in both directions.1647

If A is non-singular, that means that the system AX = B has a unique solution, the other way it means if AX = B has a unique solution, that means that A is non-singular.1658

Now you might think that it's over a quill to actually state it as equivalence, to actually explicitly say it and it has to work forward as well as backward.1671

As it turns out, if you remember from your geometric course where you studied logic, where you did P, then Q, it doesn't always work the other way around.1680

For example if I said if it's raining today, then it's cloudy, that's true, but if I reversed it, and if I said if it's cloudy and then it's raining today.1689

That isn't necessarily true, we can have a cloudy day, but without it being, without it raining, so it's very important especially in mathematics to make sure that things go both ways, or if not both ways.1697

We have to specify that it's only one way, that's why we list things the way that we do, that's why we write things the way we do, math’s is very precise.1708

Okay, so list of non-singular equivalences, now this is not like, these are equivalence, which means that 1 is the same as 2 is the same as 3 is the same as 4.1717

What that means is any one of these can replace any one, other one of these, this doesn't mean that if this is so, then this is so, they are all equivalent, there are just different ways of representing the same thing.1730

If I know that A is non-singular, I also know that AX = 0, the homogeneous has only the trivial solution, because our theorem said that if it's non, if its singular meaning non-invertible, then it has a trivial solution.1740

But here we are saying that A is non-singular, it's invertible, that means the homogeneous system only has the trivial solution, meaning all of the X's are equal to 0, that A is well equivalent to the identity matrix, which is what we did before, we set up the augmented matrix.1759

We converted one to the other, the normal matrix became the identity, and the identity became the inverse, and the system AX = B has a unique solution, so these are the first four equivalences, and each section that we actually move forward to.1777

We are going to add to these equivalences, by the end of the course, you will have a whole series of equivalences for non-singularity or invertibility, so obviously invertibility, non-singularity is a profoundly important concept in linear algebra, absolutely central, okay.1793

Let's see what we can do here, let's do an example.1811

Okay, we want to know, let's go back to black...1817

... Does the following system...1829

Have a non-trivial solution? Our system is 2X - Y + 5Z = 0.1835

3X + 2Y - 3Z = 0, X - Y + 4Z = 0, okay so...1854

... Non-trivial solution means...1873

... Singular matrix, in other words the matrix formed by (2, -1, 5, 3, 2, -3, 1, -1, 4)...1882

... Should be singular, well let's check.1897

We go ahead and we form the augmented matrix, so we will take (2, -1, 5) we will do (1, 0, 0, 0, 1, 0, 0, 0, 1), then we will finish this one off, (3, 2, -3, 1, -1 and 4).1900

We will subject this to our mathematical software and we end up with the following (1, 0, 1, 0, 1, -3, 0, 0, 0) okay, doesn't matter what these are, there are entries of course that they don't matter because we notice this row of 0's.1921

We are definitely talking about something which is, doesn't have an inverse, which means that it is singular, which implies that yes there exists a non-trivial solution...1941

... Those of you to go on and to working a science is particularly in engineering, often times it's true that you are going to be interested in finding the solution to the particular equations that you are dealing with, but as it turns out a lot of times, you are going to be looking for the quality of the solution.1962

Sometimes, the quality of the solution may need not necessarily the solution about what you can say about it, as much as you can say about it without actually finding it.1981

Will believe it or not, give you more information that the actual solution itself, sometimes you guess, sometimes its possible to find a solution to a problem, sometimes it isn't, but you can infer different properties of the solution without finding the solution itself.1988

And again often times the qualitative value is going to be more important than the solution itself, and of ‘course a lot of this will make sense as you go on in your engineering studies.2005

But just to let you know sometimes it's nice to know whether something exists or not before we actually decide to find it, and in fact the history of mathematics is replete with hundreds of years going by with people looking for a solution to a particular problem, only to discover several hundred years later that the solution actually doesn't exist.2015

They were looking for something that didn't exist, very curious...2033

Okay, so now let's go ahead and solve this, so let me see what we have here...2040

... Let's go ahead when we decide to actually find the solution itself, we will do row reduction on the augmented matrix, so we do (2, -1, 5) and we just take the matrix and the absolute linear system and subject that to reduced row echelon.2054

(3, 2, -3, 1, -1, 4), okay, reduced row echelon form we get the following, we get (1, 0, 1, 0, 0, 1, -3, 0) get (0, 0, 0, 0, ).2076

Again when we are reducing just the system itself to reduced row echelon, this row of 0's is not a problem, here we have a leading entry, here we have a leading entry, here we have a parameter.2101

We can just read this off, if we say that this is X, this is Y, this is Z, what we end up with is X + Z = 0.2115

And we get Y - 3Z = 0, so Z is our free parameter, this is the implicit solution, we can leave it like that, or we can say Z = S, we can say, then put Z back in here, we can say Y = 3S, and we can put X = -S.2127

This is the explicit solution, s, you just put in any value you want, solve for X, Y and Z, so in this case not only a non-trivial solution, an infinite number of non-trivial solutions.2154

Okay...2169

let's do a slightly more complex example, find all values of A...2177

... Such that...2189

... The inverse of...2193

... A, which we will take as (1, 1, 0, 1, 0, 0, 1, 2) A...2200

... Exists, so find al values of A, that's A right here, such that the inverse of this thing actually exists, meaning it is non-singular, it is invertible.2213

We will say...2226

Inverse exists, that implies non-singular and going back to our list of non-singular equivalences, that means that its row equivalent to, in this case this is a 3 by 3, row equivalent to I3.2230

In other words I should be able to buy a series of, those things that we do gets converted to our reduced row echelon form, I should be able to convert this to the identity matrix, the 3 by 3 identity matrix are 1's all along the main diagonal.2256

Let's go ahead and actually do that and see why happens by, you know using A, and here is where mathematical software absolutely comes in, it will do this entirely symbolically for you and here is what we get.2273

We get (1, 1, 0, 1, 0, 0, 1, 2, a) (1, 0, 0, 0, 1, 0, 0, 0, 1) that's our augmented matrix.2286

What we want to convert it to when we do reduced row echelon form, it ends up being the following, (1, 0, 0, 0, 1, 0, 0, 1), you end up with (0, 1, 0) here, (1, -1, 0) and you get -2 over A.2306

1 over A and 1 over A, this process gives us the inverse, this is A inverse right here, we have done it, converted it, it's row equivalent to this.2330

It's row equivalent to I3, in the process of doing that we have actually created the inverse right here, and the only thing we have to look at now is what this A have to be to make this defined.2345

Well as it turns out, A can be absolutely anything, but A cannot be 0, so A not equal to 0...2360

It’s kind of curious to think you have this system, any number here will work, but the minute you put a 0 here, you have all of a sudden created a matrix where the inverse doesn't exist.2377

We have used the same process up here, we have variable math software, reduced row echelon form, we have come down, we have created this.2388

We take a look here to see what is it that makes this defined, well as long as A is not 0, these numbers are perfectly well defined.2396

Okay, so that was dealing with inverses, thank you for joining us here at educator.com, let’s see you next time for linear algebra.2405