Linear Algebra Dot Product & Matrix Multiplication

Welcome back to linear algebra, so we have talked about linear systems, we have talked about matrix addition, we have talked about scalar multiplication, things like transpose, diagonal matrices.0000

Now we are going to talk about dot product and matrix multiplication, so matrix multiplication is not numerical multiplication, yes it does involve not just standard multiplying of numbers, but it's handled differently.0011

And one of the things that you are going to notice about this is that matrix multiplication does not commute.0028

In other words, I know that 5 times 6 = 6 times 5, I can do the multiplication in any order and it ends up being 30.0035

However if I take the matrix A and multiply by a matrix B, it's actually not the same as the matrix B multiplied by A.0041

It might be, but there is no guarantee that it will be, and in fact most of the time it won't be, so that's the one thing that's actually different about matrices and then numbers.0049

Le's just jump in, get started and see what we can do, okay.0058

Let's go ahead and start with a definition and this is going to be our definition of dot product, which is going to be very important and it shows up in all areas of science., so we will let...0065

... equal, A1, A2, A3, let me erase this A3 and just use ... all the way to AN.0083

And B = B1, B2..... BN, okay so let me just explain what these notations means here.0102

Whenever we see a normally a lowercase letters a, b, c, d, x we will often use x with an arrow on top, that means it's a vector, and a vector is just a list of numbers.0116

A1, A2 all the way to AN in this particular case we are talking about an N vector, which means it has N entries, so 5 vector would have 5 entries.0131

An example might be, let's say the vector V might be 1, 3, 7, 6, that's all this means, this is the vector in these the components of that vector.0140

It's composed of (1, 3, 7, 6), it's a four vector, because it has four entries in it, that's all this notation means, this is just a generalized version of it.0153

Okay, so let A, the vector A = A1 to An, let the vector B = B1 through BN, now we defines something called the dot product as the following, A.B.0163

The product of two vectors is equal to A1 times B1 + A2 times B2 + ... +A_nB_n and I am going to write this in σ notation.0180

σ notation, I'll explain in just a minute, if you guys haven't seen it, I am sure you have, but you just, I know that you don't deal with it all too often.0196

Okay, so if the vector A is composed of A₁ through AN, B is the list, B₁ through BN, the dot product A.B = the product of the corresponding entries added together.0205

When I add these together, I end up with a number, so the dot product of two vectors gives me a scalar; it gives me a number, so I just add them all up.0222

This σ notation is the capital Greek letter S, and stands for sum, and it says take the sum of the Ith entry of A, the Ith entry of B.0233

Multiply them together and add them, so A1B1, 1I = 1, and then go to the next one, I = 2 + A2B2 abd then go to I = 3 + A3B3.0246

This is just a short hand notation for this, we won't deal with σ notation all that much, what end our definitions, whatever we do I'll usually write this explicitly.0260

I just want you to be aware that in your book, you'll probably see this; you'll definitely see it in the future.0270

That's all this means, it's a short hand notation for a very long sum, so don't let the symbolism intimidate you, scare you, confuse you, anything like that, it's very simple.0275

Okay, let's just do an example of a dot product and everything should make sense, so example; we will let...0285

... Vector A = (1, 2, -3 and 4), so this is a four vector, and will let B = (-2, 3, 2, 1), notice I wrote one of them in row form, one of them in column form.0298

This is also a four vector because we have four entries, I wrote it this way because in a minute when we talk about matrix multiplication, it's going to make sense, it will make sense why it is that I wrote it this way, but just for now understand that there is no real difference between these two.0319

I could have written this as a column, I could have written this as a row, it's just a question of corresponding entries.0335

But I did like this because in a minute when we do matrix multiplication, symbolically, its going to help make sense when you move your fingers across a row and down a column, just sort of keep things straight, because matrix multiplication, there is lot, a of lot of arithmetic involved.0340

Okay, so our dot product A.B here, A.B, we just go back to our definition, it says take corresponding entries and just multiply it together, that's you got to do,0356

I take A1 times B1, so which is 1 times -2, which is -2 + 2 times 3, which is 6 + -3 times 2, which is -6...0370

... + 4 times 1, which is 4.0392

Well, that equals, so the 6 is cancelled, -2 + 4 gives me a 2, so I have a vector of 4 vector, times of 4 vector and I end up with a number 2.0396

The dot product of two vectors is a scalar, and all I am doing is multiplying corresponding entries and adding them all up, that's it simple arithmetic, nice and easy, no worries.0408

Let's go ahead and move forward now on to matrix multiplication, okay.0423

Let me go ahead and write down the definition of matrix multiplication and then we wi do some examples.0429

We will let A = that matrix A _IJ, B...0439

... M by P, so this is an M by P, be 3 by 2.0448

Let B be the IJth P...0458

P by N, so A is M by P and B is P by N, notice that the number of columns of the matrix A equal to the number of rows of the matrix B, that's going to be very important.0468

Then AB is the...0482

... M by N matrix.0493

C equals C _IJ, such that the IJth entry of C is equal to Row _I of A.0498

In other words, the Ith row of A dotted with the Jth column of B.0517

let's take a look at the definition again, A is a matrix, A _IJ, it is M by P, B is a matrix, B _IJ is P by N.0526

when I multiply those two matrices the, essentially what happens is that the column of the first matrix, the one on the left cancels the column accounts with the row of the matrix on the right an what you end up with is a matrix which is M by N.0536

And that matrix is such that the IJth entry = Ith of A dot end with the Jth column of B, that's why this P and this P have to be the same.0554

In order to multiply two matrices, let's write this one out specifically, okay.0573

In order to multiply two matrices....0581

... The number of columns of the first...0596

... Must equal...0603

... The number of rows of the second and that's what this says M by P, P by N, the number of columns of the first has to equal the number of rows.0611

That's the only way that matrix multiplication is defined and what we mean when we say is defined, means if they are not the same, you can't do the multiplication.0623

That's what defined means, it's the only way you can do it if that's the case, okay.0632

let's see what we have got, so for example if I have a 2 by 3 matrix and I want to multiply it by a 3 by 2 matrix, yes I can do that because the number of columns of the first one is equal to the number of columns of the second one, and essentially they go away.0639

What I am left with is the final matrix which is 2 by 2, this is kind of interesting.0661

Now notice if I reverse them and if I did a 3 by 2 matrix, and if I multiply that by a 3 by 2 matrix, I am sorry 2 by 3...0666

... Now, it is defined, number of columns of the first equals the number of rows of the second, so now I end up with a 3 by 3 matrix, okay.0686

These are all defined, that will work.0698

Let's see 2 by 3, 3 by 2, 3 y 2, 2 by 3, so notice what's happened here, take a quick look, I have a 2 by 3 times the 3 by 2 gives me a 2 by 2.0703

If I switch these, a 3 by 2 by a 2 by 3 I'll let them switch them, I get a 3 by 3, a 3 by 3 and a 2 by 2 are not the same.0714

In general not only do the dimensions not match, it won’t work, AB is not equal to BA, that’s the take on lesson for this, matrix multiplication does not commute.0725

AB does not equal BA, and we will actually do an example later on where we can actually do AB and BA, but they end up being completely different matrices.0737

Okay, let's do some examples, let's let A = (1, 2, -1, 3, 1, 4) when doing much matrix multiplication goes very slowly and go systematically.0747

Lot of arithmetic, lots of room for mistake, (-2, 5, 4, -3, 2, 1) okay.0767

We said that the IJth entry = Ith row of A times the Jth column of B, well we are looking at here, A, let me use black, this is a 2 by 3, 2 by 3 matrix and this is a 3 by 2.0779

Yes, it is defined because this 3 and this 3 are the same, so we should end up with a 2 by 2 matrix, okay.0799

Lets go ahead and put little thing here for our 2 by 2 matrix, now for our...0810

.. This, the first row first column, this entry it's going to equal the first row of A dotted with the first column of B, so it's going to be that row and that column, so I take 1 times -2.0820

Let me actually write over here or let's call this the, so now we are doing the A11 entry, this one right up here, A11 entry equals 1 times -2, which is -2, 2 times 4 which is 8, -1 times 2, -2.0838

-2 - 2 + 8, answer should be 4, so 4 goes there.0861

Now let's do this entry which is the first row, second column, well the first row, second column means I take the dot product of the first row of A and the second column.0870

A12 Entry = 1 times 5, which is 5, 2 times -3, which is -6, -1 times 1, which is -1.0882

That means, let's try this again without these little extra lines, 1 times 5 is 5, 2 times -3 is -6, -1 times 1, -1.0895

5 - 6 is -1, -2, so this becomes -2, now we are going to go to the second row, first column, which means we do second row first column.0909

This is A21, 3 times -2 is -6, 1 times 4 is 4, 4 times 2 is 8, so 8 + 4 is 12 - 6 is 6, so this entry is 6.0922

And now we have our last entry which is the 2,2, so the 2,2 entry, second row, second column, which means we dot product the second row with the second column, second row of A, second column of B.0941

3 times 5 is 15, 1 times -3 is -3, 4 times, oops, that's nice.0955

4 times, is that a -1, I don’t even know, no that's 1...0972

4 times 1, is 4, so we get 15 + 4,., which is 19, 19 - 2 is 16, so this entry is 6.0983

The product so, AB = 4 - 2, 6, 16, 2 by 3 matrix multiplied by a 3 by 2 matrix gives us a 2 by 2 matrix, and we get that by this row this column, this row this column, and then this row this column, this row this column.0996

That's all you are doing, rows and columns, now you know why I arranged it, remember a little bit back when we did dot product, I arranged it, the first one horizontally and the other one vertically.1021

This is the reason why, because when we multiply, we are doing this times that, this times that, this times that, we can move one this way, one this way, it seems sort of, it's a way to keep things separate, as one hand, one finger moves across a row.1031

The other finger should move down a column as used to going this way or this way., okay.1047

Lets do another example here, we will...1058

... Okay, let A equal, this is going to be a 3 by 3, so it's 1(1, -2, 3, 4, 2, 1, 0, 1, -2)1066

And B is equal to, let's make it a 3 by 2, so this is a 3 by 3, and then we have (1, 4, 3, -1, -2, and 2) , so this is a 3 by 2, so AB.1084

A times be is defined, so AB is defined and it's going to be well, 3 goes away, the 2 inside 1's, so we are left with a 3 by 2, so AB is a 3 by 2 matrix.1105

Okay, well let's just multiply it out, this time we are not going to write everything out we are just going to do the multiplication and keep it straight, they are final numbers.1121

We know that we are looking for a 3 by 2, so let's just start putting in entries, well the first entry; first row first column is going to be first row first column.1131

1 times 1 is 1, -2 times so 1 times, that's 1, -6 is going to be a -5, and then -6 is going to be -11.1142

This is going to be -11, there and now we are going to do the second entry, okay, first row second column, which means first row of A, second column.1161

One times 4 is 4, -2 times -1 is 2, 4 + 2 is 6, and then 3 times 2 is 6, that becomes 12.1172

When we continue ion this way, we end up with 8, we end up with 16, we end up with 7, we end up with -5, that's our AB.1185

Okay, now let's try something.1196

Let's let A =(1, 2, -1, 3) and we will let B = (12, 1, 0, 1) in this case, because this is 2 by 2, and because this is 2 by 2, both AB and BA, they are both defined.1203

I can do the multiplication, well let's do the multiplication and see if AB = BA.1224

There are two ways that you can, there are certain demonstrate non-commutivity, is if the dimensions don't match when you switch them or if it's defined, multiplication is defined and doable this way and that way.1229

Then you might end up with different matrices, again proving that it doesn't commute, alright.1246

Let's see what we have got, when we do AB, okay we end with the following, we end up with (2, 3, -2, 2) and when we do BA, we said it is defined.1252

We end up with (1, 7, -1, 3).1266

AB and BA are not the same, AB is not equal to BA, matrix multiplication does not commute.1271

Okay, so now let's talk about matrices and linear systems, so we introduced linear systems in our first lesson, we talked about matrices in our second, and we have just introduced matrix multiplication.1283

Now let's combine them together to see if we can take a matrix and represent it as a linear system, or a linear system and represent it in matrix form.1297

Let's let me go back to blue here, we will let, excuse me, A = A₁₁, A₁₂, A₁₃, A₂₁, A₂₂, A₂₃, A₃₁, A₃₂, A₃₃.1307

And we will let X with the little line, the vector be our vector, let's call it X₁, X₂, X₃, this is the vector formulation, this is the component form, it's just a 3 vector.1332

Okay, so this is a, we can do this in red, this is a 3 by 3 matrix, and this is a 3 by 1, right, so if I multiply this matrix by that vector X, well it's just a 3 by 3 times a 3 by 1.1346

Well those are the same, so I end up with a 3 by 1, it is defined and it's going to equal some vector b, which is going to be a 3 by 1 vector, just something with 3 entries in it.1366

And let's let B therefore equal, we will call it B₁, B₂, B₃, so again we have a matrix.1383

We have this 3 vector, I can multiply them because matrix multiplication is defined, their answer is going to be a 3 vector, so we will call that 3 vector B, and will call it's components B₁, B₂, B₃.1394

Okay, well let's actually do the multiplication here, so A₁ X, I am sorry, AX.1407

When i do this multiplication, this row, this column, this row, this column, this row, this column.1415

Here is what I get, A₁₁ times X₁ + A₁₂ times X₂ + A₁₃ times X ₃, that's what I get, that's the multiplication.1423

A₁₁ X₁ + A₁₂ X₂ + A₁₃ X₃, and then I do this second row, that column again, I get A₂₁ X₁ + A₂₂ X₂ + A₂ X₃.1435

And then I will do the third row, which is A₃₃ X₁ + A₃₂ X₂ + A₃₃ X₃, that's going to be my matrix.1455

That's my actual matrix multiplication; well I know that equals this B, so I write B₁, B₂, and B₃.1469

Well, this thing = this thing, this thing = equals this thing, this thing = this thing, that's what this says, this is just a 3 by 1 in its most expanded form.1481

That's the A times the X, this thing is the B, that are equals, and so now I am just going to set corresponding entries equal to each other, this whole thing is equal to that.1491

I write A₁₁ X₁ + A₁₂ X₂ + A₁₃ X₃ = B₁.1500

A₂₁ X₁ + A₂₂ X₂ + A₂₃ X₃ = B₂, and I am sorry that I have got extra little lines here that are showing up.1514

Try to spread a little bit slower, A₃₁ X₁ + A₃₂ X₂ + A₃₃ X₃ = B₃.1528

Well take a look at this, this is just a linear system, that's it, it's just a linear system, you have seen this before.1544

This is three equations in three variables...1554

...X₁, X₂, X₃, X₁, X₂, X₃, X₁, X₂, X₃, these A₁₁ 's A₂, all of these are coefficients and these are the actual solutions.1561

You can actually write a linear system as a matrix, so it looks like A₁₁, A₁₂, A₁₃, this is the coefficient, the matrix of coefficients for the linear system.1581

A₂₁, A₂₂, A₂₃, A₃₁, A₃₂, A₃₃, and then you multiply it by the variables, which are X₁, X₂, X₃, and it equals B₁, B₂, B₃.1602

We can take a linear system and represent it in matrix form; we take the matrix of coefficients, so this is the coefficient matrix.1626

M by N in this case is 3 by 3m, but it can be anything, this is the matrix of variables, it's the variable matrix, and it's always going to be some N vector.1640

And this is just the you might call the solution matrix, it's not really the solution matrix, the solution matrix is once you find X1, X2, X3, those are going to be your solutions, so you know what let's not even give this a name, let's just say this happens to be the, whatever it is.1657

It's the B that makes up linear system on the right side of the equality, okay now given this; we can actually form as it turns out.1678

We can form a special matrix...1692

... If we attach...1701

... B₁, B₂, B₃ to the coefficient matrix...1709

... Okay, now we are going to write this out one more time, so I am going to take this thing and I am going to add up another column to this matrix.1723

I end up with, (A₁₁, A₁₂, A₁₃), B₁, (A₂₁, A₂₂, A₂₃), B₂, (A₃₁, A₃₂, A₃₃) B₃.1732

And sometimes we put like a little dotted line there, just to let you know that this is, and this is called an augmented matrix.1752

All I have done is I have augmented my coefficient matrix with my solutions on the right for the linear system, and we do separate it.1759

Sometimes you can see a solid, I tend to put a solid line, that's just my personal preference, some people don't put anything at all, again it's, it's completely up to you.1770

Therefore any linear system can be represented in matrix form and vice versa, any matrix with more than one column can be thought of as forming a linear system.1780

Let's see what we have here, example -2X as a system -2X + 0Y + Z = 5, and then we have 2X + 3Y -4Z =7.1796

We have 3X + 2Y + 2Z = 3, so this is our linear system and now let's break it up and in matrix form, so we want to write it this way, AX = B, this is the matrix representation.1821

A matrix A times a vector B gives us a vector B, so A the matrix A is going to be the coefficients, it' going to be , excuse me, the 2, the 0, the 1, the 2, the 3, the -4, the 3, the 2, the 2.1842

(-2, 0, 1, 2, 3, -4, 3, 2, 2) that’s our coefficient matrix, X let me do this in red, oops.1861

X is going to be our variables, our variables happen to be X, Y and Z, that's going to be X, Y, and Z, and B, let me go back to red, B vector is going to be (5, 7, 3).1881

That's it, you can represent, now we will do the augmented matrix, which means take the coefficient matrix and add this to it, so we end up with A augment with B, symbolized like that.1903

It is equal to (-2, 0, 1, 5) and I'll go ahead and do a solid line, because I like solid lines.1918

(2, 3, -4, 3, 2, 2), you have your coefficient matrix and you have your matrix that represents the linear system, that was originally given to you like that.1928

Now, let's see, now let's go the other way, let's say we have a matrix, (2, -1, 3, 4, 3, 0, 2, 5) let's say you are given this particular matrix, this particular matrix actually can represent a linear system.1946

We could take a linear system, represent it in matrix form, which we just did, we can take a matrix and represent it as a linear system, if we need to.1977

This ends up being, so let's say that this is the augmented matrix, so that means this is (1, 2, 3), that means we have 3 variables, that's what the column represent are the variables, and these are the equations.1986

We have 2X - 1Y + 3Z = 4, and then 3X + 0Y + 2Z = 5.2001

Linear system, matrix form, matrix represent a linear system.2023

Excuse me,, there you go, okay now let's talk about what we did today, recap our lesson.2030

We talked about the dot product of two vectors and a vector is just an N by 1 matrix, either as a column or row, it doesn't really matter.2042

What you do is you multiply the corresponding entries in the two vectors and you add up the total.2051

The dot product gives you a single number, a scalar, it's also called the scalar product, so dot product, scalar product, as you go on in mathematics you will actually refer to it as a scalar product not necessarily an dot product.2056

After that we talked about matrix multiplication where we actually invoke the dot product, so with matrix multiplication you can only multiply two matrices if the number of columns in the first matches the number of rows in the second.2070

Matrix multiplication does not commute, in other words A times B does not equal B times A in general.2084

It might happen accidentally, but it's not true in general.2091

The IJth entry in the product is the dot product of the Ith row of the first and the Jth column of the next.2095

Okay, now matrix representations of linear systems, any linear systems of equations can be represented as an augmented matrix, you take the matrix of coefficients and you add the column of solutions.2107

Any matrix with more than one column can represent a linear system of equations, that last column is going to be your solutions, that's the augment.2122

Okay, so let's do one more example here, so we will let A = (3, 5, 2, 4, 9, 2, excuse me.2134

And B = (1, 0, 1, 6), oh she knows, (2, 1, 3, 7) so here we have a 3 by 2 matrix and here we have a 2 by 4 matrix.2153

Yes, the 2's on the inside are the same, they end p cancelling and it's going to end up giving up a 3 by 4 matrix, so we are left with 2 outside.2172

We are going to be looking for a matrix which is 3 by 4, this is kind of interesting if you think about it, 3 by 2, 2by 4, now you get a 3 by 4, you get something that's bigger than both in some sense, okay.2182

AB equal to, well we take the first row and first column, 3 times 1 + 5 times 2, 3 times 1 is 3, 5 times 2 is 10, you end up with 13, first row, second column.2195

Well you take the first row, dot product of the second column, 3 times 0 is 0, 5 times 1 is 5, so you end you with 5., then you keep going.2214

you end up (18, 53, 8, 4, 14, 40, 13, 2, 15 and 69), so our product AB = this matrix.2224

Notice 8 times B is defined if I did B times A, well B times is equal to a 2 by 4 times the 3 by 2.2243

This 4 and this 3 aren't equal, BA is not even defined, we can't even do the multiplication, leave alone and find out whether it equals or not, which in general it doesn't, so in this case it's not even defined.2256

It only works when A and B are such that A is on the left of B, B is on the right of A, and they will often say that, we will often say in linear algebra, multiply by this on the left, multiply by this on the right.2267

We don't do that with numbers, we just say multiply the numbers, okay now let's let the variable that's the X, the vector = X, Y and Z and let's let the vector Z = (4, 2, 9).2279

Now we want to express...2300

.. Well actually we don't want it, let's go ahead...2308

... We want to express AX = Z as a linear system and as an augmenting matrix, both, so we have a matrix A, that's this one; we have a matrix X, we have a matrix Z.2314

We want to express AX = Z as a linear system in augmented matrix, okay, we’ll wait a minute, let's try it to bring it, it works.2332

We can't even do, this is, this has 3 and 2, so this can be XYZ, this is going to have to be XY, my apologies.2346

XY, there we go, because this is 3 by 2, this is 2 by 1, yes we want them at, the multiplication to be defined, so we end up with (3, 5, 2, 4, 9, 2)...2356

... Times XY = (4, 2, 9) so you end up with, well 3X + 5Y = 4.2375

3X + 5Y = 4, because you are doing the first row, first entry, first row first column, 2X + 4Y, 2X + 4Y = 2.2392

And 9X + 2Y, 9X + 2Y = (, all we have done is go this times that, equals that, this times that equals that, this times that equals that.2409

And we end up with our linear system, now we want to convert that to an augmented matrix; well we take 3, 2, 9, (3, 2, 9, 5, 4, 2).2422

I'll start coefficient matrix, right, and we have augmented with our solution matrix 4, 2, 9, (4, 2, 9).2438

That's all we have done AB, A times B, it is defined, we can find the multiplication.2451

We can take given X and given Z, this is a two vector, this is a three vector, we can take AXC represented as a linear system.2459

We express it this way, we do the matrix multiplication, we said corresponding things equal to each other, and we have actually converted this to a linear system.2469

This and this are equivalent, we can take this linear system and express it completely just as a matrix, an augmented matrix by adding the solutions as the augment on the right, and we end up with that.2477

Okay, thank you for joining us today for linear algebra, and our discussion of dot products and matrix multiplication on linear systems.2493

Thank you for joining us at educator.com, we will see you next time, bye, bye.2499