Revision Library
Choose a subject
Menu
Maths
Need a new topic?
Missing a topic, found a bug, or want something added?
A Level Maths · Topic guide

Statistics: Hypothesis Testing Depth

As part of A-level Statistics, Hypothesis Testing Depth takes binomial and correlation tests into full two-tailed working: constructing BOTH critical regions of a two-tailed binomial test from cumulative probabilities and reporting the actual (not just the nominal) significance level, testing for negative as well as positive correlation using the product moment correlation coefficient, and giving the formal definitions of a Type I and a Type II error and identifying which has occurred in a scenario. The examiner is testing whether a student can build a critical region from scratch rather than being handed one, and whether the final conclusion is written in the context of the original claim rather than as a bare 'reject H0'.

A LevelStatisticsEdexcelAQAOCRWJECCCEA

Before you start

Make sure you're comfortable with these topics first:

Method

  1. Decide the approach from what the question gives. A single observed value with its exact probability given: compare that probability (the p-value) directly with the significance level. Asked to find a critical region: use cumulative binomial probabilities to find the most extreme values whose total probability does not exceed the significance level (or half of it, for each tail of a two-tailed test). A correlation claim: compare the sample PMCC, r, with a critical value for the given sample size and tail(s). Asked about an error: use the formal definitions of Type I and Type II errors.
  2. To build a two-tailed critical region, split the significance level equally between the two tails (e.g. 5% at each end of a 10% test). For the lower tail, find the largest k with P(X<=k) not exceeding that tail's significance level. For the upper tail, find the smallest k with P(X>=k) not exceeding that tail's significance level.
  3. State the ACTUAL significance level of a test as the sum of both tail probabilities actually used (or just the one tail probability, for a one-tailed test); because the binomial distribution is discrete, this is often not exactly the nominal level (e.g. not exactly 10%).
  4. The p-value method: compare the exact probability of a result at least as extreme as the one observed with the significance level (or half of it, for one side of a two-tailed test). This is equivalent to checking whether the observed value lies in the critical region, and always gives the same conclusion as that method.
  5. For a PMCC hypothesis test, state H0: rho = 0 against a one-tailed H1: rho > 0 (positive correlation) or rho < 0 (negative correlation), or a two-tailed H1: rho is not equal to 0. Compare the sample value r (or |r|, for a two-tailed test) with the critical value for the given sample size and significance level; for a negative correlation test, compare r with the NEGATIVE of the tabulated critical value.
  6. Definitions: a Type I error is rejecting H0 when H0 is actually true; a Type II error is not rejecting H0 when H0 is actually false. The significance level of a test IS the probability of a Type I error.
  7. Always write the final conclusion of a hypothesis test in context: state whether there is (or is not) significant evidence to support the claim being tested, referring back to the original scenario, not just 'reject H0'.

Common mistakes

  • MistakeQuoting the nominal significance level as the actual significance level of a binomial test.

    InsteadThe binomial is discrete, so the critical region rarely has a probability exactly equal to the nominal level. Add the probabilities of the values actually in the critical region; that sum is the actual significance level.

  • MistakeWriting that H0 has been proved true when the observed value is not in the critical region.

    InsteadNot rejecting H0 means there is insufficient evidence against it at that significance level, not that it is true. Write the conclusion in context, for example 'there is not enough evidence that the pass rate is lower than the school claims'.

  • MistakeWriting hypotheses in terms of the sample result, such as H0: x = 7 or H0: p-hat = 0.4.

    InsteadHypotheses are statements about the population parameter, p for a binomial proportion or rho for correlation. State them before looking at the sample, and set H1 from the claim being tested, not from the sample.

Worked examples

Example 1

A manufacturer claims that 30% of customers who receive a discount voucher use it within a month. A researcher doubts this and wants to test, at the 10% significance level, whether the true proportion differs from 30% (a two-tailed test). A random sample of 25 customers who received a voucher is checked. Let X be the number, out of 25, who use their voucher within a month, so under H0, X ~ B(25, 0.3). Given that, under H0, P(X<=3)=0.0332, P(X<=4)=0.0905, P(X>=11)=0.0978 and P(X>=12)=0.0442 (all to 4 dp), find the critical region for this test, and state the actual significance level.

  1. Since this is a two-tailed test at 10%, each tail is assigned half: 5% (0.05).
  2. For the lower tail, compare the given probabilities with 0.05: P(X<=3)=0.0332 does not exceed 0.05, but P(X<=4)=0.0905 does, so the lower critical region is X<=3.
  3. For the upper tail, compare the given probabilities with 0.05: P(X>=12)=0.0442 does not exceed 0.05, but P(X>=11)=0.0978 does, so the upper critical region is X>=12.
  4. The critical region is X<=3 or X>=12.
  5. The actual significance level is the sum of both tail probabilities used: 0.0332 + 0.0442 = 0.0774 (7.74%).

Example 2

A delivery firm claims that 45% of its parcels arrive before 9 am. A customer believes the true proportion is lower. She checks a random sample of 21 parcels, and X is the number that arrive before 9 am. She tests H0: p = 0.45 against H1: p < 0.45 at the 5% significance level. Given that, under H0, P(X <= 4) = 0.0126, P(X <= 5) = 0.0389 and P(X <= 6) = 0.0964, (a) find the critical region and the actual significance level of the test. (b) In fact the true proportion is 0.25. Given that P(X <= 5) = 0.5666 when p = 0.25, find the probability of a Type II error.

  1. Under H0, X ~ B(21, 0.45). This is a one-tailed test, so the critical region is the largest set X <= k with P(X <= k) not exceeding 0.05.
  2. P(X <= 5) = 0.0389 does not exceed 0.05, but P(X <= 6) = 0.0964 does, so the critical region is X <= 5.
  3. The actual significance level is P(X <= 5) under H0, which is 0.0389 (3.89%).
  4. A Type II error means H0 is not rejected although it is false, so X falls outside the critical region: X >= 6.
  5. When p = 0.25, P(X >= 6) = 1 - P(X <= 5) = 1 - 0.5666 = 0.4334.
  6. Final answer: (a) critical region X <= 5 with actual significance level 0.0389; (b) the probability of a Type II error is 0.433 (3 s.f.) when the true proportion is 0.25.

Practice questions

Q1[2 marks]State the definitions of a Type I error and a Type II error in a hypothesis test.Show answer

Answer: A Type I error is rejecting H0 when H0 is actually true. A Type II error is not rejecting H0 when H0 is actually false (i.e. when H1 is actually true).

Got it right?
Your answerWork it out on paper, then show the answer and mark yourself.
Q2[1 mark]A hypothesis test checks whether a coin is biased towards heads, using H0: p=0.5 against H1: p>0.5. The test concludes there is no significant evidence of bias, but the coin actually is biased. State which type of error has occurred.Show answer

Answer: A Type II error, since H0 was not rejected (no significant evidence found) but H0 is actually false (the coin genuinely is biased).

Got it right?
Your answerWork it out on paper, then show the answer and mark yourself.
Q3[1 mark]State the relationship between the significance level of a hypothesis test and the probability of a Type I error.Show answer

Answer: The significance level of a test IS the probability of making a Type I error (rejecting H0 when it is actually true).

Got it right?
Your answerWork it out on paper, then show the answer and mark yourself.
Q4[4 marks]For 12 pairs of data values, the product moment correlation coefficient is r = -0.68. Test, at the 5% significance level, whether there is evidence of negative correlation, given that the critical value for a one-tailed test at the 5% level with n=12 is 0.4973.Show answer

Answer: H0: rho=0, H1: rho<0. Compare r=-0.68 with -0.4973. Since -0.68 is more negative than -0.4973, reject H0; there is evidence of negative correlation.

Got it right?
Your answerWork it out on paper, then show the answer and mark yourself.
Q5[2 marks]A one-tailed hypothesis test at the 5% significance level gives an exact p-value of 0.083 for the observed result. State, with a reason, the conclusion of the test.Show answer

Answer: Since 0.083 > 0.05, do not reject H0; there is insufficient evidence at the 5% level.

Got it right?
Your answerWork it out on paper, then show the answer and mark yourself.
Q6[1 mark]A hypothesis test for a population proportion uses the model X ~ B(n, p0) under H0. State one assumption needed for this model to be valid.Show answer

Answer: Any one of: the trials are independent of each other; each trial has the same, constant probability p0 of success; the sample is random.

Got it right?
Your answerWork it out on paper, then show the answer and mark yourself.
Q7[3 marks]Under H0, X ~ B(18, 0.4). Given that P(X<=3)=0.0328 and P(X<=4)=0.0942 (both to 4dp), find the critical region for a one-tailed test H1: p<0.4 at the 5% significance level, and state the actual significance level of the test.Show answer

Answer: Since P(X<=3)=0.0328 does not exceed 0.05, but P(X<=4)=0.0942 does, the critical region is X<=3, with actual significance level 0.0328 (3.28%).

Got it right?
Your answerWork it out on paper, then show the answer and mark yourself.

Exam-style questions

Written in the style of a A Level Maths exam paper, with a full mark scheme.

Exam tips

  • For each tail, list the cumulative probabilities either side of the boundary next to the tail's significance level, so that both ends of the critical region are visible.
  • Describe an error in context by starting from what the test concludes and then what is true: 'the test concludes ... when in fact ...'.
  • Start by defining the sample statistic and its distribution under H0, such as 'Let X be the number ... ; under H0, X ~ B(n, p0)'.
Q1[8 marks]A charity claims that 35% of people who visit its website make a donation. Following a website redesign, the charity wants to test, at the 10% significance level, whether the proportion donating has changed. A random sample of 20 visitors is monitored. Let X be the number, out of 20, who donate, so under H0, X ~ B(20, 0.35). (a) State suitable null and alternative hypotheses for this test. (2) (b) Given that, under H0, P(X<=3)=0.0444, P(X<=4)=0.1182, P(X>=11)=0.0532 and P(X>=12)=0.0196 (all to 4dp), find the critical region for the test. (4) (c) In the sample, 10 of the 20 visitors donated. State, with a reason, the conclusion of the test, referring to the charity's claim. (2)Show mark scheme

Tick each line you got.

Nothing ticked yet - 8 available

Got it right?
Q2[5 marks]A researcher measures, for 14 UK towns, the average commute time (x, minutes) and a measure of reported stress (y, out of 100). The product moment correlation coefficient for the sample is r = -0.61. (a) State suitable hypotheses to test, at the 5% significance level, for a two-tailed test of correlation between commute time and stress. (2) (b) Given that the critical value for a two-tailed test at the 5% level, with n=14, is 0.5324, determine whether there is significant evidence of correlation, justifying your conclusion in context. (3)Show mark scheme

Tick each line you got.

Nothing ticked yet - 5 available

Got it right?
Q3[4 marks]A quality-control test on a production line uses H0: p=0.05 (the historic defect rate) against H1: p>0.05, at the 5% significance level, where p is the true proportion of defective items. (a) Describe, in context, what a Type I error would mean for this production line. (2) (b) Describe, in context, what a Type II error would mean for this production line. (2)Show mark scheme

Tick each line you got.

Nothing ticked yet - 4 available

Got it right?

See real A Level Maths past-paper questions, with official mark schemes 

This topic in real past papers

We have tagged some boards' past papers question by question (not every board or year yet). This topic comes up in:

Free printable worksheet

Want more practice on paper? Download the Statistics: Hypothesis Testing Depth worksheet pack - 8 pages of exam-style questions with a full mark scheme. Free to download, no card or account. Print it for personal and classroom use.

This topic is chapter 10 of A Level Maths Statistics Essentials, the whole course as one free printable PDF. Its worksheet is also chapter 10 of A Level Maths Statistics Topic Workbook and chapter 10 of A Level Maths 10-Minute Tests 2, free printable practice books.

Keep going

Not quite what you needed?

Tell us what is missing on Statistics: Hypothesis Testing Depth, or which topic to write up next.

Build a full practice pack.