Statistics: Hypothesis Testing Depth
As part of A-level Statistics, Hypothesis Testing Depth takes binomial and correlation tests into full two-tailed working: constructing BOTH critical regions of a two-tailed binomial test from cumulative probabilities and reporting the actual (not just the nominal) significance level, testing for negative as well as positive correlation using the product moment correlation coefficient, and giving the formal definitions of a Type I and a Type II error and identifying which has occurred in a scenario. The examiner is testing whether a student can build a critical region from scratch rather than being handed one, and whether the final conclusion is written in the context of the original claim rather than as a bare 'reject H0'.
Before you start
Make sure you're comfortable with these topics first:
Method
- Decide the approach from what the question gives. A single observed value with its exact probability given: compare that probability (the p-value) directly with the significance level. Asked to find a critical region: use cumulative binomial probabilities to find the most extreme values whose total probability does not exceed the significance level (or half of it, for each tail of a two-tailed test). A correlation claim: compare the sample PMCC, r, with a critical value for the given sample size and tail(s). Asked about an error: use the formal definitions of Type I and Type II errors.
- To build a two-tailed critical region, split the significance level equally between the two tails (e.g. 5% at each end of a 10% test). For the lower tail, find the largest k with P(X<=k) not exceeding that tail's significance level. For the upper tail, find the smallest k with P(X>=k) not exceeding that tail's significance level.
- State the ACTUAL significance level of a test as the sum of both tail probabilities actually used (or just the one tail probability, for a one-tailed test); because the binomial distribution is discrete, this is often not exactly the nominal level (e.g. not exactly 10%).
- The p-value method: compare the exact probability of a result at least as extreme as the one observed with the significance level (or half of it, for one side of a two-tailed test). This is equivalent to checking whether the observed value lies in the critical region, and always gives the same conclusion as that method.
- For a PMCC hypothesis test, state H0: rho = 0 against a one-tailed H1: rho > 0 (positive correlation) or rho < 0 (negative correlation), or a two-tailed H1: rho is not equal to 0. Compare the sample value r (or |r|, for a two-tailed test) with the critical value for the given sample size and significance level; for a negative correlation test, compare r with the NEGATIVE of the tabulated critical value.
- Definitions: a Type I error is rejecting H0 when H0 is actually true; a Type II error is not rejecting H0 when H0 is actually false. The significance level of a test IS the probability of a Type I error.
- Always write the final conclusion of a hypothesis test in context: state whether there is (or is not) significant evidence to support the claim being tested, referring back to the original scenario, not just 'reject H0'.
Worked example
A manufacturer claims that 30% of customers who receive a discount voucher use it within a month. A researcher doubts this and wants to test, at the 10% significance level, whether the true proportion differs from 30% (a two-tailed test). A random sample of 25 customers who received a voucher is checked. Let X be the number, out of 25, who use their voucher within a month, so under H0, X ~ B(25, 0.3). Given that, under H0, P(X<=3)=0.0332, P(X<=4)=0.0905, P(X>=11)=0.0978 and P(X>=12)=0.0442 (all to 4 dp), find the critical region for this test, and state the actual significance level.
- Since this is a two-tailed test at 10%, each tail is assigned half: 5% (0.05).
- For the lower tail, compare the given probabilities with 0.05: P(X<=3)=0.0332 does not exceed 0.05, but P(X<=4)=0.0905 does, so the lower critical region is X<=3.
- For the upper tail, compare the given probabilities with 0.05: P(X>=12)=0.0442 does not exceed 0.05, but P(X>=11)=0.0978 does, so the upper critical region is X>=12.
- The critical region is X<=3 or X>=12.
- The actual significance level is the sum of both tail probabilities used: 0.0332 + 0.0442 = 0.0774 (7.74%).
Practice questions
Try each question, then tap to reveal the answer.
Q1State the definitions of a Type I error and a Type II error in a hypothesis test.Show answer
Answer: A Type I error is rejecting H0 when H0 is actually true. A Type II error is not rejecting H0 when H0 is actually false (i.e. when H1 is actually true).
Q2A hypothesis test checks whether a coin is biased towards heads, using H0: p=0.5 against H1: p>0.5. The test concludes there is no significant evidence of bias, but the coin actually is biased. State which type of error has occurred.Show answer
Answer: A Type II error, since H0 was not rejected (no significant evidence found) but H0 is actually false (the coin genuinely is biased).
Q3State the relationship between the significance level of a hypothesis test and the probability of a Type I error.Show answer
Answer: The significance level of a test IS the probability of making a Type I error (rejecting H0 when it is actually true).
Q4For 12 pairs of data values, the product moment correlation coefficient is r = -0.68. Test, at the 5% significance level, whether there is evidence of negative correlation, given that the critical value for a one-tailed test at the 5% level with n=12 is 0.4973.Show answer
Answer: H0: rho=0, H1: rho<0. Compare r=-0.68 with -0.4973. Since -0.68 is more negative than -0.4973, reject H0; there is evidence of negative correlation.
Q5A one-tailed hypothesis test at the 5% significance level gives an exact p-value of 0.083 for the observed result. State, with a reason, the conclusion of the test.Show answer
Answer: Since 0.083 > 0.05, do not reject H0; there is insufficient evidence at the 5% level.
Q6A hypothesis test for a population proportion uses the model X ~ B(n, p0) under H0. State one assumption needed for this model to be valid.Show answer
Answer: Any one of: the trials are independent of each other; each trial has the same, constant probability p0 of success; the sample is random.
Q7Under H0, X ~ B(18, 0.4). Given that P(X<=3)=0.0328 and P(X<=4)=0.0942 (both to 4dp), find the critical region for a one-tailed test H1: p<0.4 at the 5% significance level, and state the actual significance level of the test.Show answer
Answer: Since P(X<=3)=0.0328 does not exceed 0.05, but P(X<=4)=0.0942 does, the critical region is X<=3, with actual significance level 0.0328 (3.28%).
Exam-style questions
Written in the style of a A Level Maths exam paper, with a full mark scheme.
A charity claims that 35% of people who visit its website make a donation. Following a website redesign, the charity wants to test, at the 10% significance level, whether the proportion donating has changed. A random sample of 20 visitors is monitored. Let X be the number, out of 20, who donate, so under H0, X ~ B(20, 0.35). (a) State suitable null and alternative hypotheses for this test. (2) (b) Given that, under H0, P(X<=3)=0.0444, P(X<=4)=0.1182, P(X>=11)=0.0532 and P(X>=12)=0.0196 (all to 4dp), find the critical region for the test. (4) (c) In the sample, 10 of the 20 visitors donated. State, with a reason, the conclusion of the test, referring to the charity's claim. (2)
Show mark scheme
Tick each line you got. Your score builds from the marks on the scheme.
Nothing ticked yet - 8 available
A researcher measures, for 14 UK towns, the average commute time (x, minutes) and a measure of reported stress (y, out of 100). The product moment correlation coefficient for the sample is r = -0.61. (a) State suitable hypotheses to test, at the 5% significance level, for a two-tailed test of correlation between commute time and stress. (2) (b) Given that the critical value for a two-tailed test at the 5% level, with n=14, is 0.5321, determine whether there is significant evidence of correlation, justifying your conclusion in context. (3)
Show mark scheme
Tick each line you got. Your score builds from the marks on the scheme.
Nothing ticked yet - 5 available
A quality-control test on a production line uses H0: p=0.05 (the historic defect rate) against H1: p>0.05, at the 5% significance level, where p is the true proportion of defective items. (a) Describe, in context, what a Type I error would mean for this production line. (2) (b) Describe, in context, what a Type II error would mean for this production line. (2)
Show mark scheme
Tick each line you got. Your score builds from the marks on the scheme.
Nothing ticked yet - 4 available
Free printable worksheet
Want more practice on paper? Download the statistics: hypothesis testing depth worksheet pack - 9 pages of exam-style questions with a full mark scheme. One email opens every download in this browser for 14 days - no account, no card. Print it for personal and classroom use.
Next topics
Not quite what you needed?
Tell us what is missing on statistics: hypothesis testing depth, or which topic to write up next. Every request is read, and we reply to every one.
Build a full practice pack.
This topic is one of hundreds in the library - pick the ones a student needs and generate a printable PDF in minutes.