Statistics Type I and Type II Errors

Hi and welcome to www.educator.com.0000

Today we are going to talk more in-depth about type 1 and type 2 errors.0001

If you want to know more about power and effect size it is good to go through this lesson0006

because it is going to help you understand some of the pictures that we are going to draw in the future.0013

Here is the roadmap for today.0017

We need to know about these type 1 and type 2 errors, but we also need to know when we make those errors in relationship to hypothesis testing.0021

So far we only used t test as our hypothesis test.0033

We have shown these errors and their relationship to hypothesis testing before as a box, but frequently in hypothesis testing we draw distributions.0037

The SDOM to be more specific.0048

What I want to show you how the errors fit on this distribution picture.0051

We are going to show you how the box and the distributions fit together because these two things actually relationship to each other.0058

They refer to the same concept.0065

There are just 2 different ways of showing you that same concept.0067

We go through hypothesis testing, but in the real world there is some reality that either the null hypotheses is just true or the null hypothesis is false.0071

Although we do not know this reality, all we know is the result of our hypothesis testing.0086

There are two kinds of ways we can make errors.0092

We can make an incorrect decision by false alarming.0095

We reject the null, but we should not have rejected the null.0099

That is called the false alarm or a type 1 error.0106

I used to get confused between which one is type 1 and type 2, these are arbitrate.0110

I like to think of this as the more serious error when you successfully reject the null hypothesis that is a more extreme thing that you do.0116

This is actually more dangerous than this miss.0127

That is not much of an error but actually false alarming.0131

That is how I remember the number 1 error you should look out for.0136

The type 1 error is often also called the likelihood of false alarming.0142

The probability of false alarming and that is referred to as alpha.0151

If the reality that we do not know is that this null hypothesis is true we have a probability of false alarming with the rate of alpha.0157

We have the probability of failing to reject when we should have rejected, a correct failure your probability is 1-alpha.0171

These two things add up to 1.0186

The probability of false alarming + the probability of making a correct failure =1.0189

On the flipside, let us say that null hypothesis is false that is not a true picture or model of the world.0199

Then we really should have reject it.0210

It is not true, we should reject it, that would be a correct decision and that is called the hit where we are rejecting the null when we should have rejected it.0213

That gives us the probability of hits.0226

We could be incorrect and fail to reject when we should have rejected that is also another incorrect decision.0230

That is the type 2 error.0242

It is a miss and the probability of miss is given as beta.0244

Beta + 1 –beta = 1.0248

The probability of misses + probability of hits =1.0254

In which of these boxes is the sample statistic statistically significant?0262

In which of these boxes is our p value less than .05 or whatever our alpha level is.0274

Let us think about that.0280

When we reject the null hypothesis that means our test statistic in this case t is extreme.0282

Our p value is significant and remember we mean significant as it stands out.0294

It is very weird.0304

In this case, these two quadrants up here is what we should worry about.0307

This is the decision we need to worry about when we reject the null hypothesis.0315

The other possibility is that when we reject the null hypotheses and our p is significant we made a correct decision.0322

These are our two choices if we know that p is less than alpha or if our test statistic is extreme.0334

Here p is not significant.0343

It is not too weird and because of that we will fail to reject and we can be correct in failing to reject0349

or when we fail to reject we could be wrong by making a type 2 error.0358

Here is what I want you to know.0363

Let us say we carry out hypothesis testing and I think I have a really low p value.0365

I am going to reject my null hypotheses.0372

Which error am I likely to make, a false alarm or a missed?0375

Since I rejected my null, the only error I can possibly make is this one where I reject the null and get wrong.0381

Let us say I go through my hypothesis testing and I get p=.4.0397

Let us say I do not reject my null.0405

What mistake or what error could I have possibly made?0408

The only error I can make is the missed error.0411

Here I fail to reject and I could be wrong in doing it.0414

Let us talk about distributions and how errors fit in here.0418

We have a one sample t test we set up some null population.0426

This is our null hypotheses population and our hypothesized mu might be 230.0431

We do not know whether our sample is part of this or it is part of some other population, not the null population.0443

We can hypothesize maybe it comes from some other population like this one.0454

When we set our alpha levels and create critical t and zones of rejection and all of that stuff what we are doing is recreating the line.0459

If our sample t is outside here then we are going to reject the null.0476

So far we have only colored in this part, but we really mean this part as well as all of this part.0492

That is our reject the null zone, this entire area.0508

In order to find out whether we should reject the null or not we also need to look past the raw score.0514

We need to look past the raw score and we need to look at it in terms of the critical t.0528

The critical t might be whatever like -2. Something .0536

We need to find out this t value and so I am just going to make one.0544

Let us say this t value is 5.5 and if our t value is sufficiently extreme then we reject are null hypothesis.0549

This would be our critical t and this is our sample x bar, but this is our sample t.0560

And that is how it looks out here.0574

Our possibility of making an error is this little gray spot that I have colored in red.0577

Just in case my sample really does come from these areas, I should not have rejected the null.0587

If it happens by chance rule 50 heads in a row it is very unlikely but it is still possible.0596

It is still possible that I got this x bar even though this is the true population distribution.0613

This is my possibility of making a type 1 error.0621

We actually have to add this side up to this side type 1 error.0628

We know that this is alpha=.05.0640

This part is 1 – alpha which is .95 and that is our possibility of not rejecting given that the null hypothesis is true.0646

That is the example of one sample hypothesis testing.0660

This is the same picture as before.0666

I just written it more neatly for you by typing it out and you can think of this test statistic as just t.0669

I have just written the generic word test statistic to think of this as critical t and sample t.0676

Here is the important thing to realize.0682

This gray distribution here represents an SDOM and that is why this is mu sub x bar and there is also an x bar here as a sample.0685

This SDOM actually represents the probability where the null hypothesis is true and that probability equal to 1.0696

Remember we talked about that before when we said the area underneath the normal distribution equal to 1.0706

This represents the possibility that this may not be true and that there exists some other population that our sample really came from.0713

We do not just know what that population is.0727

That is the probability that the null hypothesis is false.0731

That normal distribution also has an area =1.0736

What we can additionally find out is when we create the zones of rejection and we say anything outside of this critical t reject it.0744

We color in this area here.0759

What we are saying is this is the probability of rejecting given that the null hypothesis is true.0761

This is the area where we fail to reject.0777

This probability right here represents the conditional probability of failing to reject given that h knot or null hypothesis is true.0784

And that equals 1 – alpha because this one equal alpha.0807

Those are the important things to remember.0815