Research methods: variables, hypotheses, choosing a statistical test and the sign test
Variables, hypotheses and test selection are the core planning steps in psychological research: researchers must identify independent and dependent variables and write directional, non-directional and null hypotheses before running a study, then use experimental design and level of measurement to choose an appropriate statistical test, such as the sign test, to analyse their results.
Before you start
No prerequisites. You can start here.
Method
- Learn to identify the independent variable (IV) and dependent variable (DV) in a described study, and name the experimental design used (independent groups, repeated measures, or matched pairs).
- Learn to write a fully operationalised hypothesis, in both directional and non-directional form.
- Learn to write the corresponding null hypothesis for a study.
- Learn to identify a plausible extraneous variable in a study and describe a control for it.
- Learn to read a test-choice table by design (independent groups, repeated measures, or correlation) and by level of measurement (nominal, ordinal, or interval/ratio, and whether data is normally distributed) to select the correct inferential test.
- Learn to carry out a sign test: assign a sign to each participant's change, exclude any tied (zero-change) scores, calculate S, and compare it with a critical value.
- Learn to interpret a significant or non-significant result correctly, including the difference between statistical significance and how large or consistent an effect looks in the raw data.
Common mistakes
MistakeWriting a hypothesis such as 'music affects concentration' without saying how either variable is measured or scored.
InsteadOperationalise both variables: state what is changed and what is counted or scored, for example 'participants who work with music playing will complete more sums correctly in three minutes than participants who work in silence'. A directional hypothesis also names the direction.
MistakeWriting the null hypothesis as the opposite prediction, for example 'participants who work with music playing will complete fewer sums'.
InsteadThe null hypothesis says there is no difference (or no relationship) between the conditions and that any difference found is due to chance. A prediction in the opposite direction is a different alternative hypothesis, not the null.
MistakeTaking S in a sign test as the more frequent sign, so that six plus signs and three minus signs are reported as S = 6.
InsteadS is the total of the LESS frequent sign, counted after ties are removed. Six plus signs and three minus signs give N = 9 and S = 3, which is the value compared with the critical value.
MistakeDeciding significance for a sign test with the chi-squared rule, saying the result is significant because the calculated value is bigger than the critical value.
InsteadThe rule depends on the test. For the sign test, like the Wilcoxon and Mann-Whitney tests, the result is significant when the calculated value is less than or equal to the critical value. For chi-squared, Spearman's rho and Pearson's r it must be greater than or equal to it.
Worked examples
Example 1
A teacher tests whether a short breathing exercise reduces exam stress, using a repeated measures design with six students. Each student rates their stress from 0 (no stress) to 10 (extreme stress) before and after a 10-minute breathing exercise: Student 1: before 8, after 5. Student 2: before 6, after 7. Student 3: before 9, after 4. Student 4: before 7, after 7. Student 5: before 8, after 3. Student 6: before 5, after 2. Using a sign test, decide whether the breathing exercise produced a significant reduction in stress (the critical value of S for n = 5, p < 0.05, one-tailed, is 0).
- Work out the direction of change for each student: Student 1 decrease (-), Student 2 increase (+), Student 3 decrease (-), Student 4 no change (0), Student 5 decrease (-), Student 6 decrease (-).
- Exclude any student showing no change: Student 4 is excluded because their score did not change, leaving n = 5.
- Count how many of the remaining signs are the less frequent sign: there are 4 decreases (-) and 1 increase (+), so the less frequent sign is the +, occurring once.
- State the calculated value of S: S = 1.
- Compare S with the given critical value for n = 5 (which is 0), remembering the result is only significant if the calculated value is less than or equal to the critical value.
- Since S (1) is greater than the critical value (0), state the final decision: the result is not significant, so the null hypothesis is retained: the breathing exercise did not produce a significant reduction in stress in this sample.
Example 2
A psychologist asks 12 participants to rate how much they enjoy a fruit juice from 1 (dislike) to 9 (like very much), once served cold and once served at room temperature. She has no reason to predict which will be preferred. The ratings (cold, room temperature) are: P1 7, 5; P2 6, 6; P3 8, 6; P4 5, 7; P5 9, 6; P6 7, 4; P7 4, 4; P8 8, 5; P9 6, 3; P10 7, 8; P11 8, 5; P12 6, 4. Write the hypotheses, choose a suitable test and carry out a sign test at p <= 0.05. The critical value of S for N = 10, two-tailed, is 1.
- Write the alternative hypothesis in non-directional form: there will be a difference in enjoyment ratings between cold juice and room-temperature juice. The null hypothesis is that there is no difference, and any difference found is due to chance.
- Choose the test: each person rates both juices (a repeated measures design) and a 1 to 9 rating is not truly equal-interval data, so only the direction of each person's difference is used. A sign test is suitable.
- Give each participant a sign for cold minus room temperature: P1 +, P2 0, P3 +, P4 -, P5 +, P6 +, P7 0, P8 +, P9 +, P10 -, P11 +, P12 +.
- Exclude the two ties (P2 and P7), so N = 12 - 2 = 10. There are 8 plus signs and 2 minus signs.
- S is the less frequent sign, so S = 2.
- Compare with the critical value of 1 (N = 10, p <= 0.05, two-tailed). For the sign test the result is significant only if S is less than or equal to the critical value. 2 is greater than 1, so the result is not significant.
- State the conclusion: the null hypothesis is retained, since no significant difference in enjoyment was found. Eight of ten people preferred the cold juice, which looks lopsided, but with only ten scores that split can still happen by chance; the test needed S = 1 or 0, meaning a split of 9 to 1 or stronger.
Practice questions
Q1[1 mark]A study measures how the number of hours of revision (2, 4, or 6 hours) affects quiz scores, using three separate groups of participants. What is the IV?Show answer
Answer: The number of hours of revision (2, 4, or 6 hours).
Q2[1 mark]In a repeated measures design, are the participants in each condition the same people or different people?Show answer
Answer: The same people; they take part in every condition.
Q3[1 mark]Which statistical test should be used for a repeated measures study comparing normally distributed interval data from two conditions?Show answer
Answer: The related (paired) t-test.
Q4[1 mark]A researcher measures the correlation between hours of sleep and reaction time, using interval data that is normally distributed. Which test should be used?Show answer
Answer: Pearson's product-moment correlation coefficient.
Q5[1 mark]In a sign test, what should be done with a participant whose score does not change between the two conditions?Show answer
Answer: Their data is excluded from the analysis, and n is reduced accordingly.
Q6[2 marks]What is a non-directional (two-tailed) hypothesis?Show answer
Answer: A hypothesis that predicts there will be a difference or relationship between conditions or variables, without stating the direction of that difference.
Exam-style questions
Written in the style of a A Level Psychology exam paper, with a full mark scheme.
Exam tips
- When asked to choose a statistical test, give your reasons in order: the design (related, unrelated or correlation), then the level of measurement, then the name of the test. A bare test name gives no reasoning to follow.
- Write each hypothesis using the study's own conditions and measures instead of the labels 'IV' and 'DV', and put a direction in the wording only when the question states or implies one.
- Write the critical value with its conditions beside it (N, one- or two-tailed, p level) so the comparison can be seen, then state the decision about the null hypothesis in words.
Q1[5 marks]A researcher tests whether owning a pet (yes or no) is associated with a person's self-reported stress category (low or high), using one sample of adults in which each adult is counted once, in one pet-ownership category and one stress category, and the frequency counts in the four combinations are compared. The calculated chi-squared value is 3.1. The critical value at p < 0.05, for the appropriate degrees of freedom, is 3.84. State whether the result is significant and explain what this means.Show mark scheme
Tick each line you got.
Nothing ticked yet - 5 available
Q2[6 marks]A student carries out a repeated measures study comparing 8 participants' reaction times before and after drinking caffeine. The reaction-time data is interval level, but a normality check shows the data is not normally distributed. The student decides to use a related (paired) t-test to analyse the results. Explain why the student's choice of test is incorrect, and identify the test they should have used instead.Show mark scheme
Tick each line you got.
Nothing ticked yet - 6 available
See real A Level Psychology past-paper questions, with official mark schemes
This topic in real past papers
We have tagged some boards' past papers question by question (not every board or year yet). This topic comes up in:
- Eduqas A Level Psychology past papers: 14 questions across 7 papers, latest tagged Summer 2025
- WJEC A Level Psychology past papers: 12 questions across 10 papers, latest tagged Summer 2025
- AQA A Level Psychology past papers: 7 questions across 6 papers, latest tagged June 2024
Free printable worksheet
Want more practice on paper? Download the Research methods: variables, hypotheses, choosing a statistical test and the sign test worksheet pack - 7 pages of exam-style questions with a full mark scheme. Free to download, no card or account. Print it for personal and classroom use.
This topic is chapter 14 of A Level Psychology Essentials 2, the whole course as one free printable PDF. Its worksheet is also chapter 2 of A Level Psychology Topic Workbook 4 and chapter 21 of A Level Psychology 10-Minute Tests 2, free printable practice books.
Other versions of this worksheet: