Further Statistics: Hypothesis Testing, Type I and Type II Errors - Worksheets, Questions and Revision
22 original exam-style questions - 8 pages of questions with a full mark scheme - free printable PDF.
Further Statistics: Hypothesis Testing, Type I and Type II Errors revision guideWrite-in editionMore A Level Further Maths practiceMock exams
This topic is chapter 4 of A Level Further Maths Further Statistics Topic Workbook and chapter 14 of A Level Further Maths 10-Minute Tests 2.
2.4 Further Statistics: Hypothesis Testing, Type I and Type II Errors
Hypothesis Testing for a Poisson or Binomial Parameter
Revision Library
For a hypothesis test on a Poisson mean lambda or a binomial probability p, a null hypothesis H0 states a fixed value of the parameter, and an alternative hypothesis H1 states that the parameter has increased, has decreased (a one-tailed test), or is simply different (a two-tailed test). A test statistic X, assumed to follow the distribution stated in H0, is compared against a critical region: the set of values of X for which H0 is rejected in favour of H1.
Because X is discrete, the probability of X falling in a proposed critical region can rarely be made to equal a chosen significance level exactly. The usual convention is to choose the largest critical region whose probability does not exceed the significance level requested. This probability is called the actual significance level of the test, and it is also equal to P(Type I error), the probability of wrongly rejecting a true H0.
A Type II error occurs when H0 is not rejected even though H1 is actually true. Its probability depends on the true value of the parameter, and is found by calculating the probability that X falls outside the critical region, using the true (alternative) value of the parameter in place of the value stated in H0. The power of a test is defined as 1 - P(Type II error): the probability of correctly rejecting a false H0. Power increases as the true parameter value moves further from the value stated in H0, and is also affected by the significance level chosen and the sample size used.
Question 1
- (a) attempts P(X≥9) using X ~ B(12, 0.5)M1
- (a) P(X≥9) = 299/4096 = awrt 0.0730A1
- (a) doubles this probability for a two-tailed testM1
- (a) p-value = awrt 0.146A1
- (a) Answer: p-value = awrt 0.146
- (b) 0.146 > 0.05, so do not reject H0; there is insufficient evidence, at the 5% level, that the spinner is biased towards even numbers (ft, in context)B1
- (b) Answer: Since 0.146 > 0.05, H0 is not rejected; there is insufficient evidence that the spinner is biased towards even numbers.
Question 2
- (a) states H0: p = 0.3, H1: p < 0.3, with X ~ B(20, 0.3) under H0, and attempts a lower-tail search comparing cumulative probabilities with 0.05M1
- (a) compares P(X≤2) and P(X≤3) with 0.05M1
- (a) critical region is X ≤ 2 (i.e. {0, 1, 2}), caoA1
- (a) actual significance level = awrt 0.0355A1
- (a) Answer: Critical region: X ≤ 2 (i.e. {0, 1, 2}). Actual significance level = awrt 0.0355.
- (b) 1 lies in the critical region {0, 1, 2} found in part (a) (ft)B1
- (b) reject H0; there is evidence, at the 5% level, that the fungicide has reduced the proportion of seedlings affected by blight (ft, in context)B1
- (b) Answer: 1 is in the critical region {0, 1, 2}, so H0 is rejected; there is evidence at the 5% level that the fungicide has reduced the proportion affected.
- (c) recognises the Type II error region is the complement of the critical region, i.e. X ≥ 3, using the true distribution B(20, 0.15)M1
- (c) uses P(X≥3 | p=0.15) = 1 - P(X≤2 | p=0.15)M1
- (c) awrt 0.595A1
- (c) Answer: P(Type II error) = awrt 0.595
- (d) power = 1 - 0.595 = awrt 0.405 (ft from part (c))B1
- (d) Answer: power = awrt 0.405
- (e) the probability of a Type II error would decrease (oe)B1
- (e) a larger significance level allows a larger critical region to be used; more of the alternative distribution's probability then falls inside this larger critical region (i.e. leads to rejecting H0), leaving less probability in the acceptance region, so P(Type II error) falls (oe)B1
- (e) Answer: The probability of a Type II error would decrease, because a 10% test uses a larger critical region, so a greater proportion of the B(20, 0.15) distribution falls inside it, correctly leading to rejection of H0.
Question 3
- (a) attempts P(X≤3) with λ = 10M1
- (a) P(X≤3) = awrt 0.0103A1
- (a) attempts P(X≥18) = 1 - P(X≤17) with λ = 10M1
- (a) P(X≥18) = awrt 0.0143A1
- (a) P(Type I error) = 0.0103 + 0.0143 = awrt 0.0246A1
- (a) Answer: P(Type I error) = awrt 0.0246
Question 4
- (a) compares P(X≤0), P(X≤1) and P(X≤2) with 0.01M1
- (a) critical region is X ≤ 1 (i.e. {0, 1}), caoA1
- (a) actual significance level = awrt 0.00764A1
- (a) Answer: Critical region: X ≤ 1 (i.e. {0, 1}). Actual significance level = awrt 0.00764.
- (b) the critical region {0, 1} used here is smaller than the critical region {0, 1, 2} used in Question 2(a), so the acceptance region is correspondingly larger (oe)B1
- (b) for the same true alternative value of p, a larger proportion of the alternative distribution now falls in the acceptance region, so P(Type II error) is larger (power is lower) than in Question 2 (oe)B1
- (b) Answer: The critical region {0, 1} is smaller than {0, 1, 2}, so more of the alternative (p = 0.15) distribution falls in the acceptance region here than in Question 2; P(Type II error) is therefore larger, and the power lower, than for the 5% test.
Question 5
- (a) states H0: λ = 8, H1: λ < 8, and attempts to compare P(X≤3) and P(X≤4) with 0.05M1
- (a) correctly evaluates both cumulative probabilitiesM1
- (a) critical region is X ≤ 3 (i.e. {0, 1, 2, 3}), caoA1
- (a) actual significance level = awrt 0.0424A1
- (a) Answer: Critical region: X ≤ 3 (i.e. {0, 1, 2, 3}). Actual significance level = awrt 0.0424.
- (b) identifies the acceptance region as X ≥ 4, using the true distribution Po(5)M1
- (b) uses P(X≥4 | λ=5) = 1 - P(X≤3 | λ=5)M1
- (b) awrt 0.735A1
- (b) Answer: P(Type II error) = awrt 0.735
- (c) power = 1 - 0.735 = awrt 0.265 (ft from part (b))B1
- (c) Answer: power = awrt 0.265
- (d) valid suggestion, e.g. increase the significance level (widen the critical region, for example to a 20% test, giving the critical region X ≤ 5 and power awrt 0.616), so that more of the Po(5) distribution falls inside the (larger) critical regionB1
- (d) valid corresponding disadvantage, e.g. this increases the probability of a Type I error (wrongly concluding the mean has decreased when it has not) (oe)B1
- (d) Answer: Suggestion: increase the significance level (e.g. to 20%, giving the critical region X ≤ 5), widening the critical region so more of the true Po(5) distribution falls inside it, raising the power above 0.5. Disadvantage: this also increases the probability of a Type I error.
Question 6
- (a) uses Po(5) probabilities in the appropriate tailM1
- (a) P(X ≥ 10) = 0.0318 ≤ 0.05A1
- (a) critical region X ≥ 10A1
- (a) Answer: critical region X ≥ 10
- (b) the probability of a Type I error = P(X in the critical region | H0 true)M1
- (b) 0.0318A1
- (b) Answer: 0.0318
- (c) P(X not in the critical region | λ = 6)M1
- (c) 0.916A1
- (c) Answer: 0.916
Question 7
- (a) uses B(25, 0.25) cumulative probabilitiesM1
- (a) critical region X ≤ 3A1
- (a) Answer: critical region X ≤ 3
- (b) P(X in critical region | p = 0.25)M1
- (b) 0.0962A1
- (b) Answer: 0.0962
- (c) P(X not in critical region | p = 0.1)M1
- (c) 0.236A1
- (c) Answer: 0.236
Question 8
- (a) H0: p = 0.15B1
- (a) H1: p < 0.15B1
- (a) Answer: H0: p = 0.15, H1: p < 0.15
- (b) P(X ≥ 16) = (1 - 0.15)16 - 1M1
- (b) 0.0874A1
- (b) reject H0A1
- (b) Answer: 0.0874; reject H0
- (c) finds the smallest d with (1 - 0.15)d - 1 ≤ 0.1M1
- (c) critical region X ≥ 16A1
- (c) P(X ≤ 15 | p = 0.08) = 1 - (1 - 0.08)15M1
- (c) 0.714A1
- (c) Answer: X ≥ 16; Type II error 0.714
Question 9
- (a) uses Po(8) probabilities in the appropriate tailM1
- (a) P(X ≤ 4) = 0.0996 ≤ 0.1A1
- (a) critical region X ≤ 4A1
- (a) Answer: critical region X ≤ 4
- (b) the probability of a Type I error = P(X in the critical region | H0 true)M1
- (b) 0.0996A1
- (b) Answer: 0.0996
- (c) P(X not in the critical region | λ = 3)M1
- (c) 0.185A1
- (c) Answer: 0.185
Question 10
- (a) uses B(12, 0.2) cumulative probabilitiesM1
- (a) critical region X ≥ 5A1
- (a) Answer: critical region X ≥ 5
- (b) P(X in critical region | p = 0.2)M1
- (b) 0.0726A1
- (b) Answer: 0.0726
- (c) P(X not in critical region | p = 0.7)M1
- (c) 0.00949A1
- (c) Answer: 0.00949
Question 11
- (a) uses Po(4) probabilities in the appropriate tailM1
- (a) P(X ≥ 8) = 0.0511 ≤ 0.1A1
- (a) critical region X ≥ 8A1
- (a) Answer: critical region X ≥ 8
- (b) the probability of a Type I error = P(X in the critical region | H0 true)M1
- (b) 0.0511A1
- (b) Answer: 0.0511
- (c) P(X not in the critical region | λ = 10)M1
- (c) 0.220A1
- (c) Answer: 0.220
Question 12
- (a) H0: p = 0.25B1
- (a) H1: p < 0.25B1
- (a) Answer: H0: p = 0.25, H1: p < 0.25
- (b) P(X ≥ 11) = (1 - 0.25)11 - 1M1
- (b) 0.0563A1
- (b) do not reject H0A1
- (b) Answer: 0.0563; do not reject H0
- (c) finds the smallest d with (1 - 0.25)d - 1 ≤ 0.05M1
- (c) critical region X ≥ 12A1
- (c) P(X ≤ 11 | p = 0.08) = 1 - (1 - 0.08)11M1
- (c) 0.600A1
- (c) Answer: X ≥ 12; Type II error 0.600
Question 13
- (a) uses Po(2) probabilities in the appropriate tailM1
- (a) P(X ≥ 6) = 0.0166 ≤ 0.05A1
- (a) critical region X ≥ 6A1
- (a) Answer: critical region X ≥ 6
- (b) the probability of a Type I error = P(X in the critical region | H0 true)M1
- (b) 0.0166A1
- (b) Answer: 0.0166
- (c) P(X not in the critical region | λ = 6)M1
- (c) 0.446A1
- (c) Answer: 0.446
Question 14
- (a) uses B(10, 0.4) cumulative probabilitiesM1
- (a) critical region X ≤ 1A1
- (a) Answer: critical region X ≤ 1
- (b) P(X in critical region | p = 0.4)M1
- (b) 0.0464A1
- (b) Answer: 0.0464
- (c) P(X not in critical region | p = 0.15)M1
- (c) 0.456A1
- (c) Answer: 0.456
Question 15
- (a) H0: p = 0.1B1
- (a) H1: p < 0.1B1
- (a) Answer: H0: p = 0.1, H1: p < 0.1
- (b) P(X ≥ 12) = (1 - 0.1)12 - 1M1
- (b) 0.314A1
- (b) do not reject H0A1
- (b) Answer: 0.314; do not reject H0
- (c) finds the smallest d with (1 - 0.1)d - 1 ≤ 0.1M1
- (c) critical region X ≥ 23A1
- (c) P(X ≤ 22 | p = 0.05) = 1 - (1 - 0.05)22M1
- (c) 0.676A1
- (c) Answer: X ≥ 23; Type II error 0.676
Question 16
- (a) uses B(12, 0.2) cumulative probabilitiesM1
- (a) critical region X ≥ 6A1
- (a) Answer: critical region X ≥ 6
- (b) P(X in critical region | p = 0.2)M1
- (b) 0.0194A1
- (b) Answer: 0.0194
- (c) P(X not in critical region | p = 0.35)M1
- (c) 0.787A1
- (c) Answer: 0.787
Question 17
- (a) H0: p = 0.15B1
- (a) H1: p < 0.15B1
- (a) Answer: H0: p = 0.15, H1: p < 0.15
- (b) P(X ≥ 23) = (1 - 0.15)23 - 1M1
- (b) 0.0280A1
- (b) reject H0A1
- (b) Answer: 0.0280; reject H0
- (c) finds the smallest d with (1 - 0.15)d - 1 ≤ 0.05M1
- (c) critical region X ≥ 20A1
- (c) P(X ≤ 19 | p = 0.05) = 1 - (1 - 0.05)19M1
- (c) 0.623A1
- (c) Answer: X ≥ 20; Type II error 0.623
Question 18
- (a) H0: p = 0.15B1
- (a) H1: p < 0.15B1
- (a) Answer: H0: p = 0.15, H1: p < 0.15
- (b) P(X ≥ 19) = (1 - 0.15)19 - 1M1
- (b) 0.0536A1
- (b) do not reject H0A1
- (b) Answer: 0.0536; do not reject H0
- (c) finds the smallest d with (1 - 0.15)d - 1 ≤ 0.05M1
- (c) critical region X ≥ 20A1
- (c) P(X ≤ 19 | p = 0.05) = 1 - (1 - 0.05)19M1
- (c) 0.623A1
- (c) Answer: X ≥ 20; Type II error 0.623
Question 19
- (a) H0: p = 0.3B1
- (a) H1: p < 0.3B1
- (a) Answer: H0: p = 0.3, H1: p < 0.3
- (b) P(X ≥ 19) = (1 - 0.3)19 - 1M1
- (b) 0.00163A1
- (b) reject H0A1
- (b) Answer: 0.00163; reject H0
- (c) finds the smallest d with (1 - 0.3)d - 1 ≤ 0.1M1
- (c) critical region X ≥ 8A1
- (c) P(X ≤ 7 | p = 0.1) = 1 - (1 - 0.1)7M1
- (c) 0.522A1
- (c) Answer: X ≥ 8; Type II error 0.522
Question 20
- (a) states H0: p = 0.4, H1: p ≠ 0.4, with X ~ B(15, 0.4) under H0, and searches the lower tail comparing P(X≤2) and P(X≤3) with 0.05M1
- (a) lower critical region X ≤ 2 (i.e. {0, 1, 2}), actual lower-tail level = awrt 0.0271A1
- (a) searches the upper tail comparing P(X≥9) and P(X≥10) with 0.05M1
- (a) upper critical region X ≥ 10 (i.e. {10, 11, ..., 15}), actual upper-tail level = awrt 0.0338A1
- (a) combined critical region stated as {0, 1, 2} union {10, 11, ..., 15}A1
- (a) total actual significance level = 0.0271 + 0.0338 = awrt 0.0609A1
- (a) Answer: Critical region: {0, 1, 2} union {10, 11, ..., 15}. Total actual significance level = awrt 0.0609.
- (b) 3 does not lie in the critical region {0, 1, 2} union {10, ..., 15} found in part (a) (ft)B1
- (b) do not reject H0; there is insufficient evidence, at the 10% level, that the true win probability differs from 0.4 (ft, in context)B1
- (b) Answer: 3 is not in the critical region, so H0 is not rejected; there is insufficient evidence at the 10% level that the win probability differs from 0.4.
- (c) any one correctly matched advantage and disadvantage, e.g. advantage: a larger critical region gives higher power, so a genuine change in win probability is more likely to be detected; disadvantage: this also gives a higher probability of a Type I error, wrongly concluding the win probability has changed when it has not (oe)B1
- (c) Answer: Advantage: a 10% test has a larger critical region and higher power, so it is more likely to detect a genuine change in win probability. Disadvantage: it also has a higher probability of a Type I error (wrongly rejecting a true H0) than a 5% test would.
Question 21
- (a) recognises that, since the critical region is X ≤ 3, the power here equals P(X≤3 | λ=3), and the Type II error probability is its complementM1
- (a) attempts to sum P(X=0) to P(X=3) with λ = 3M1
- (a) power = awrt 0.647A1
- (a) P(Type II error) = 1 - 0.647 = awrt 0.353A1
- (a) clearly labels which value is the power and which is P(Type II error), avoiding confusing the twoB1
- (a) Answer: power = awrt 0.647; P(Type II error) = awrt 0.353
- (b) correctly states the power is higher when the true mean is 3 (awrt 0.647) than when the true mean is 5 (awrt 0.265, from Question 5)B1
- (b) gives a valid reason referencing distance from H0 / overlap of distributions, e.g. a true mean of 3 is further from the hypothesised mean of 8 than a true mean of 5 is, so the null (λ=8) and alternative distributions overlap less, and a greater proportion of the alternative distribution falls inside the critical region X≤3 (oe)B1
- (b) states the general principle: for a fixed significance level and sample size, the power of a test increases as the true parameter value moves further from the value stated in H0 (oe)B1
- (b) Answer: The power is higher when the true mean is 3 (0.647) than when it is 5 (0.265, from Question 5), because a true mean of 3 is further from the hypothesised value of 8, so the null and alternative distributions overlap less and more of the alternative distribution falls in the critical region X≤3; in general, power increases as the true parameter moves further from the value stated in H0.
Question 22
- (a) states H0: λ = 6, H1: λ ≠ 6, and searches the lower tail comparing P(X≤1) and P(X≤2) with 0.05M1
- (a) lower critical region X ≤ 1 (i.e. {0, 1}), actual lower-tail level = awrt 0.0174A1
- (a) searches the upper tail comparing P(X≥10) and P(X≥11) with 0.05M1
- (a) upper critical region X ≥ 11 (i.e. {11, 12, 13, ...}), actual upper-tail level = awrt 0.0426A1
- (a) combined critical region stated as {0, 1} union {11, 12, 13, ...}A1
- (a) total actual significance level = 0.0174 + 0.0426 = awrt 0.0600A1
- (a) Answer: Critical region: {0, 1} union {11, 12, 13, ...}. Total actual significance level = awrt 0.0600.
- (b) 10 does not lie in the critical region {0, 1} union {11, 12, ...} found in part (a) (ft)B1
- (b) do not reject H0; there is insufficient evidence, at the 10% level, that the mean number of faults per board has changed (ft, in context)B1
- (b) Answer: 10 is not in the critical region, so H0 is not rejected; there is insufficient evidence at the 10% level that the mean has changed.
- (c) identifies the acceptance (non-rejection) region as 2 ≤ X ≤ 10, using the true distribution Po(9)M1
- (c) uses P(2≤X≤10 | λ=9) = P(X≤10 | λ=9) - P(X≤1 | λ=9)M1
- (c) awrt 0.705A1
- (c) Answer: P(Type II error) = awrt 0.705
- (d) power = 1 - 0.705 = awrt 0.295 (ft from part (c))B1
- (d) Answer: power = awrt 0.295
- (e) valid comment referencing the low power value, e.g. the power is only awrt 0.295, so the test would fail to detect this particular increase in mean on roughly 70% of occasions; the test is not very sensitive to a change of this size (oe)B1
- (e) Answer: The power is only about 0.295, so the test would fail to detect an increase in mean to 9 on around 70% of occasions; it is not a very sensitive test for a change of this size.
Mark your answers
Check answers without an account. Some answers can be checked automatically; use the mark scheme to mark the rest.