Further Statistics: Hypothesis Testing, Type I and Type II Errors
Within A-level Further Statistics, a hypothesis test uses sample data to decide between a null hypothesis H0 (a specific claimed value of a parameter, such as a Poisson mean lambda) and an alternative hypothesis H1 (that the parameter has increased, decreased or changed). The critical region is the set of outcomes that lead to rejecting H0; because a Poisson or geometric variable is discrete, the actual significance level (the true P(Type I error) for the critical region actually chosen) is usually not exactly equal to the nominal significance level, and the critical region is chosen so the actual level is as close as possible without exceeding the nominal level. A Type I error occurs when H0 is rejected but is actually true, with probability equal to the actual significance level; a Type II error occurs when H0 is not rejected but H1 is actually true, and its probability depends on the true value of the parameter. The power of a test is 1 - P(Type II error).
Before you start
Make sure you're comfortable with these topics first:
Method
- Write down H0 and H1 in terms of the parameter (e.g. H0: lambda=5, H1: lambda<5 for a one-tailed test), and identify the significance level given.
- Find the critical region by working outwards from the extreme values of the test statistic's distribution under H0, adding one value at a time and checking the cumulative probability, stopping just before it would exceed the significance level.
- State the actual significance level of the test: the exact probability, under H0, of the test statistic falling in the critical region chosen - usually slightly less than the nominal level because the distribution is discrete.
- To find P(Type I error), use the actual significance level found above (the probability of rejecting H0 when H0 is true).
- To find P(Type II error) for a specific alternative value of the parameter, find the probability, using that alternative value, that the test statistic falls OUTSIDE the critical region (so H0 is not rejected even though H1 is true).
- State conclusions in context: compare the observed value of the test statistic with the critical region, and word the conclusion in terms of the original scenario, not just symbols.
Worked example
A machine is thought to produce faulty components at a mean rate of 4 per day, and Josh believes a new process has reduced this rate. Let X be the number of faulty components found on a randomly chosen day after the new process is introduced. Josh tests H0: lambda=4 against H1: lambda<4 at the 5% significance level. (a) Find the critical region for the test. (b) State the actual significance level of the test. (c) On the day tested, 0 faulty components were found. State the conclusion of the test.
- Under H0, X ~ Po(4). To find the critical region for H1: lambda<4 at the 5% level, find the largest c such that P(X<=c) <= 0.05.
- Calculate P(X=0) = e^-4 = 0.0183 (3 s.f.) and P(X<=1) = P(X=0)+P(X=1) = e^-4 + 4e^-4 = 5e^-4 = 0.0916 (3 s.f.).
- Since P(X<=0) = 0.0183 <= 0.05 but P(X<=1) = 0.0916 > 0.05, the critical region is X<=0, i.e. the single outcome {0}.
- The actual significance level of the test is P(X<=0) = 0.0183 (3 s.f.), which is less than the nominal 5% because X is discrete.
- The observed value X=0 lies in the critical region {0}.
- Final answer: critical region is {0}; actual significance level = 0.0183 (3 s.f.); since 0 is in the critical region, reject H0 - there is evidence at the 5% level that the mean rate of faulty components has decreased.
Practice questions
Try each question, then tap to reveal the answer.
Q1Explain what is meant by a Type I error in a hypothesis test.Show answer
Answer: A Type I error occurs when the null hypothesis H0 is rejected, but H0 is actually true
Q2Explain what is meant by a Type II error in a hypothesis test.Show answer
Answer: A Type II error occurs when the null hypothesis H0 is not rejected, but H0 is actually false (H1 is true)
Q3Explain why the actual significance level of a hypothesis test on a Poisson variable is often not exactly equal to the nominal significance level (e.g. 5%).Show answer
Answer: Because the test statistic is discrete, its cumulative probabilities take only specific values, so it is usually impossible to choose a critical region with probability exactly equal to the nominal significance level; the critical region is chosen so the actual level is as close as possible without exceeding the nominal level
Q4Define the power of a hypothesis test.Show answer
Answer: The power of a test is the probability of correctly rejecting H0 when H1 is actually true, i.e. power = 1 - P(Type II error)
Q5A test of H0: lambda=6 against H1: lambda>6 uses the critical region X>=10, where X ~ Po(6) under H0. Find the probability of a Type I error for this test.Show answer
Answer: P(Type I error) = P(X>=10 | lambda=6) = 1 - P(X<=9) = 0.0839 (3 s.f.)
Q6A test of H0: lambda=5 against H1: lambda<5 uses the critical region X<=2, where X ~ Po(5) under H0. Find the probability of a Type I error for this test.Show answer
Answer: P(Type I error) = P(X<=2 | lambda=5) = 0.125 (3 s.f.)
Q7A test of H0: p=0.4 against H1: p<0.4 for the parameter of a geometric distribution uses critical region X>=6. Given that H0 is actually true and X=8 is observed, state which type of error, if any, has been made.Show answer
Answer: X=8 lies in the critical region {6,7,8,...}, so H0 is rejected; since H0 is actually true, this is a Type I error
Exam-style questions
Written in the style of a A Level Further Maths exam paper, with a full mark scheme.
An online retailer receives customer complaints about a particular product at a mean rate of 4.5 per week. Following a packaging redesign, the retailer tests H0: lambda=4.5 against H1: lambda<4.5 at the 5% significance level, using X, the number of complaints received in a randomly chosen week, with critical region X<=0. Given that the true mean rate after the redesign is actually 2 complaints per week, find the probability of a Type II error for this test.
Show mark scheme
Tick each line you got. Your score builds from the marks on the scheme.
Nothing ticked yet - 5 available
A website receives spam messages at a mean rate of 8 per day. The administrator believes a new filter has increased the rate at which spam is caught and logged, and tests H0: lambda=8 against H1: lambda>8 at the 5% significance level, using X, the number of spam messages logged on a randomly chosen day. (a) Find the critical region for the test, in the form X>=c. (b) State the actual significance level of the test.
Show mark scheme
Tick each line you got. Your score builds from the marks on the scheme.
Nothing ticked yet - 6 available
A rare genetic condition is diagnosed in newborns at a mean rate of 3.5 per month at a large hospital. A new screening programme is introduced, and the hospital believes the true rate has fallen. Let X be the number of diagnoses in a randomly chosen month after the programme starts. A test of H0: lambda=3.5 against H1: lambda<3.5 is carried out at the 5% significance level. (a) Find the critical region for the test. (b) State the actual significance level of the test. (c) Given that the true mean rate after the programme is actually 1 diagnosis per month, find the probability of a Type II error.
Show mark scheme
Tick each line you got. Your score builds from the marks on the scheme.
Nothing ticked yet - 7 available
See real A Level Further Maths past-paper questions, with official mark schemes →
Free printable worksheet
Want more practice on paper? Download the further statistics: hypothesis testing, type i and type ii errors worksheet pack - 11 pages of exam-style questions with a full mark scheme. One email opens every download in this browser for 14 days - no account, no card. Print it for personal and classroom use.
Next topics
Not quite what you needed?
Tell us what is missing on further statistics: hypothesis testing, type i and type ii errors, or which topic to write up next. Every request is read, and we reply to every one.
Build a full practice pack.
This topic is one of hundreds in the library - pick the ones a student needs and generate a printable PDF in minutes.