Pilot Surveys (Foundation) - Worksheets, Questions and Revision

18 original exam-style questions - 8 pages of questions with a full mark scheme - free printable PDF.

Download PDF Jump to mark scheme (page 9)Read the revision guide

Next in this strand

Revision Library

revisionlibrary.co.uk

FOUNDATION

F12 Pilot Surveys (Foundation)

EDEXCEL 1ST0 · Calculator allowed · about 75 minutes

Total Marks

Name: _______________________________ Date: ____ / ____ / ______

Answer ALL questions. Show all your working.

Define what is meant by a pilot survey.

(Total for Question 1 is 1 mark)

For each statement about pilot surveys, write down whether it is True or False.

(i)A pilot survey is carried out on a small group of people, rather than the whole population being studied in the main survey.(1)

(ii)A pilot survey takes place after the main survey has been completed, in order to check the results.(1)

(Total for Question 2 is 2 marks)

Before carrying out a large survey, a researcher first carries out a pilot survey.

Give four different reasons why she might do this.

(a)Give one reason for carrying out a pilot survey, relating to how clear the questions are.(1)

(b)Give a different reason for carrying out a pilot survey, relating to how long it takes to complete.(1)

(c)Give a different reason for carrying out a pilot survey, relating to the response options offered.(1)

(d)Give a different reason for carrying out a pilot survey, relating to the cost of the main survey.(1)

(Total for Question 3 is 4 marks)

A pilot survey can check many things before the main survey takes place.

(a)Which of these is NOT a typical reason for carrying out a pilot survey?(1)

A) To check that the questions are clearly worded
B) To check how long the questionnaire takes to complete
C) To make sure the final results match what the researcher wants to find
D) To check that the response options do not overlap

(b)Give a reason why the statement in option C would not be a valid reason for carrying out a pilot survey.(1)

(Total for Question 4 is 2 marks)

As part of a pilot survey, a researcher posts questionnaires to 25 people. 20 of the 25 questionnaires are completed and returned.

(a)Work out the response rate for this pilot survey, as a percentage.(2)

(b)State whether a response rate of 80% would generally be considered a high or a low response rate for a survey, giving a reason.(1)

(Total for Question 5 is 3 marks)

In a different pilot survey, only 9 of the 30 questionnaires sent out were returned.

(a)Give one reason why a person might not respond to a questionnaire.(1)

(b)Explain why a low response rate, found during a pilot survey, could be a problem if the same questionnaire were used, unchanged, for the main survey.(2)

(Total for Question 6 is 3 marks)

A pilot questionnaire for a supermarket includes the question shown below.

(a)State the type of problem with this question.(1)

(b)Explain why this is a problem for the pilot survey.(1)

(c)Rewrite the question so that it is no longer leading.(2)

(Total for Question 7 is 4 marks)

A different question on the same pilot questionnaire asks: "How often do you exercise?"

Response options given: Never, Sometimes, Often, A lot

(a)Identify the problem with the response options given for this question.(1)

(b)Suggest a better set of response options for this question, using numerical ranges.(2)

(Total for Question 8 is 3 marks)

The same pilot questionnaire asks respondents to tick a box to show their age, as shown below.

(a)A pilot respondent who is exactly 20 years old is unsure which box to tick. Identify the problem with the response options shown, and explain why it occurs.(2)

(b)Rewrite the response options so that they no longer overlap, and still cover every possible age.(2)

(Total for Question 9 is 4 marks)

A student is investigating how quickly ice cubes melt at different room temperatures. Rather than using a questionnaire, she plans to collect her results using a data collection sheet, on which she will record the time taken for each ice cube to melt completely.

Before her main investigation, she carries out a pilot of her data collection method.

(a)State one difference between piloting a questionnaire and piloting a data collection sheet or method, such as the student's.(2)

(b)Give an example of a problem that a pilot of the student's data collection method might reveal, that a pilot questionnaire would not.(2)

(Total for Question 10 is 4 marks)

A statistical investigation follows the statistical enquiry cycle: plan, collect, process, interpret.

(a)Name the stage of the statistical enquiry cycle at which a pilot survey would normally be carried out.(1)

(b)Explain how the results of a pilot survey, carried out at the plan stage, can affect how the collect stage is then carried out.(2)

(Total for Question 11 is 3 marks)

A gym is designing a customer satisfaction questionnaire, and pilots it on 10 members before the main survey. Three of the draft questions are shown below.

Question 1: "How satisfied are you with our excellent new equipment?"
Question 2: "How many times do you visit the gym?" Response options: Rarely, Sometimes, Regularly, Very often
Question 3: "What is your age?" Response options: Under 18, 18-30, 30-45, 45 and over

(a)Identify the problem with Question 1, and rewrite it so the problem is fixed.(2)

(b)Identify the problem with Question 2, and rewrite its response options so the problem is fixed.(2)

(c)Identify the problem with the response options for Question 3, and rewrite them so the problem is fixed.(2)

(Total for Question 12 is 6 marks)

The gym's pilot survey from Question 12 was sent to 10 members, of whom 8 responded.

(a)Write down the response rate for the pilot survey, as a percentage.(1)

(b)The gym plans to send the main, improved questionnaire to all 250 members. Assuming the same response rate as the pilot, work out how many completed questionnaires the gym should expect to have returned.(2)

(Total for Question 13 is 3 marks)

A national retailer plans to survey 5000 customers about a new loyalty scheme. Before the main survey, it runs a small pilot survey with 40 customers.

Discuss the benefits and limitations of using a pilot survey in this situation.

(Total for Question 14 is 3 marks)

A scientist is investigating the strength of a type of rope by measuring the force needed to break samples of it, using a testing machine. Before the main investigation, she pilots her data collection method by testing a small number of samples.

(a)State two things the scientist could check for during this pilot of her data collection method.(2)

(b)Explain why piloting this data collection method is different from piloting a questionnaire.(2)

(Total for Question 15 is 4 marks)

A charity's pilot survey of 30 supporters reveals two problems: one question was found to be leading, and only 15 of the 30 questionnaires were returned.

Recommend what the charity should do next, referring to at least two stages of the statistical enquiry cycle (plan, collect, process, interpret) in your answer.

(Total for Question 16 is 3 marks)

A student claims: "Carrying out a pilot survey guarantees that the main survey will have no problems."

Evaluate this claim.

(Total for Question 17 is 2 marks)

A local council pilots a questionnaire about a proposed new cycle path. It is sent to 40 residents, of whom 24 respond. One of the questions asks: "Don't you think a new cycle path would be a great improvement to our town?"

(a)Work out the response rate for this pilot survey, as a percentage.(2)

(b)Identify the problem with the quoted question.(1)

(c)The council now plans the main survey, sending the questionnaire to all 6000 residents. Recommend two changes the council should make before doing so, referring to your answers to parts (a) and (b).(2)

(Total for Question 18 is 5 marks)

Mark scheme · F12 Pilot Surveys

Question 1

B1 a small-scale trial version of a survey/questionnaire, carried out before the main survey, oe
Answer: A pilot survey is a small-scale trial run of a survey or questionnaire, carried out on a small group of people before the main survey takes place.

Question 2

(i) B1 True cao
(i) Answer: True
(ii) B1 False cao
(ii) Answer: False

Question 3

(a) B1 checks that the wording of the questions is clear and easy to understand, oe
(a) Answer: To check that the questions are clear and easy for people to understand.
(b) B1 checks how long the questionnaire takes to complete, oe
(b) Answer: To find out how long the questionnaire takes people to complete.
(c) B1 checks that the response categories/options given are suitable and cover all possible answers, oe
(c) Answer: To check that the response options cover every possible answer, with no gaps.
(d) B1 identifies problems early, avoiding wasting time and money running a flawed survey on the full population, oe
(d) Answer: To identify and fix any problems before spending time and money on the full-scale survey.

Question 4

(a) B1 C cao
(a) Answer: C) To make sure the final results match what the researcher wants to find
(b) B1 a pilot survey should test the process objectively, not be used to force a particular result, oe
(b) Answer: A pilot survey should be used to test and improve the survey method fairly, not to steer the results towards an answer the researcher wants.

Question 5

(a) M1 20/25, oe
(a) A1 80% cao
(a) Answer: 80%
(b) B1 high, with an acceptable reason, e.g. most people who received a questionnaire returned it
(b) Answer: High. Most of the people who were sent a questionnaire (20 out of 25) returned it.

Question 6

(a) B1 any valid reason, e.g. the questionnaire is too long, oe personal/sensitive questions, oe no interest in the topic, oe the person did not receive it
(a) Answer: The questionnaire might take too long to complete, so some people do not bother finishing or returning it.
(b) B1 identifies that too few responses would be collected in the main survey, oe
(b) B1 identifies that the people who do respond might not be representative/typical of the whole population, leading to biased conclusions
(b) Answer: Too few people would respond to give a large enough sample, and the small number who do respond might not be typical of the whole population, so the conclusions of the main survey could be biased or unreliable.

Question 7

(a) B1 it is a leading question cao
(a) Answer: It is a leading question.
(b) B1 it encourages/pushes the respondent towards a particular answer (agreeing), so the results would be biased and not reflect people's true opinions
(b) Answer: The question is worded to push people towards agreeing, so the answers would not reflect what customers really think, they are biased.
(c) B1 question no longer contains wording that suggests an expected answer
(c) B1 question remains clear and answerable, e.g. offers a genuine comparison or opinion scale
(c) Answer: e.g. "How do the new self-service checkouts compare with the old checkouts?" with response options such as Much better, Better, About the same, Worse, Much worse.

Question 8

(a) B1 the options are vague/ambiguous - they are not clearly defined, so different people could interpret 'Sometimes' or 'Often' differently, oe
(a) Answer: The options are ambiguous. Words like 'Sometimes' and 'Often' mean different things to different people, so answers would not be comparable.
(b) B1 response options are given as clear, non-overlapping numerical ranges, e.g. based on times per week
(b) B1 ranges cover every possible answer, including 0 (never) and a top open-ended range
(b) Answer: e.g. 0 times a week, 1-2 times a week, 3-4 times a week, 5 or more times a week

Question 9

(a) B1 the response categories overlap
(a) B1 the value 20 appears in both the '10-20' and the '20-30' box, so it is unclear which one to tick
(a) Answer: The categories overlap. Someone aged exactly 20 could tick either the '10-20' box or the '20-30' box, so it is not clear which one to choose.
(b) B1 boundaries no longer shared between adjacent categories, e.g. 10-19, 20-29, 30-39
(b) B1 categories cover every age with no gaps, including an open-ended top category, e.g. 40 and over
(b) Answer: e.g. 10-19, 20-29, 30-39, 40 and over

Question 10

(a) B1 piloting a questionnaire tests the wording/clarity of written questions and their response options
(a) B1 piloting a data collection sheet/method tests the practical process of measuring/recording data (e.g. equipment, timing, layout of the recording sheet), rather than question wording
(a) Answer: Piloting a questionnaire tests whether written questions and their response options are clear. Piloting a data collection sheet or method tests whether the practical process of measuring and recording results, such as the equipment used or the layout of the sheet, actually works.
(b) B1 identifies a practical/equipment problem, e.g. the stopwatch is hard to read, or the ice cubes are not all the same size to begin with
(b) B1 links the problem to an effect on the results, e.g. this would make the times recorded inconsistent/unfair to compare
(b) Answer: For example, the pilot might reveal that the ice cubes used are not all exactly the same size, which would make the melting times unfair to compare between temperatures.

Question 11

(a) B1 plan cao
(a) Answer: The plan stage.
(b) B1 any problems found during the pilot (e.g. unclear questions, overlapping categories, low response rate) are corrected before the main survey
(b) B1 the improved/finalised questionnaire or method is then used to collect data from the full sample/population, making the collect stage more reliable/efficient
(b) Answer: Any problems the pilot uncovers, such as unclear wording or a low response rate, can be fixed before the main survey. This means the collect stage then uses an improved questionnaire or method, so the data collected is more reliable.

Question 12

(a) B1 identifies Question 1 as a leading question (the word 'excellent' suggests the expected answer)
(a) B1 gives an improved, unbiased version, e.g. 'How satisfied are you with our gym equipment?'
(a) Answer: Question 1 is leading, because "excellent" suggests the answer expected. Improved: "How satisfied are you with our gym equipment?"
(b) B1 identifies that the response options for Question 2 are vague/ambiguous (undefined terms like 'Rarely' or 'Regularly')
(b) B1 gives improved options using clear numerical ranges, e.g. 0 times a week, 1-2 times a week, 3-4 times a week, 5 or more times a week
(b) Answer: Question 2's options are ambiguous. Improved: 0 times a week, 1-2 times a week, 3-4 times a week, 5 or more times a week.
(c) B1 identifies that the age categories for Question 3 overlap (30 appears in both '18-30' and '30-45')
(c) B1 gives improved, non-overlapping categories that still cover every age, e.g. Under 18, 18-29, 30-44, 45 and over
(c) Answer: Question 3's categories overlap at 30. Improved: Under 18, 18-29, 30-44, 45 and over.

Question 13

(a) B1 80% cao
(a) Answer: 80%
(b) M1 250 x 0.8, oe
(b) A1 200 cao
(b) Answer: 200

Question 14

B1 a clear overall judgement is given (e.g. the pilot is worthwhile, even though it has some limitations), rather than only listing points
B1 a benefit is explained, e.g. testing on 40 customers first can uncover unclear questions or a low response rate cheaply, before the expense of surveying all 5000 customers
B1 a limitation is explained, e.g. 40 customers is a very small sample compared with 5000, so the pilot might not reveal every problem that could occur in the full survey
Answer: Running a pilot survey is worthwhile. It lets the retailer find and fix problems, such as unclear questions or a low response rate, on a small, low-cost group of 40 customers before spending time and money surveying all 5000. However, because 40 is a very small sample compared with 5000, the pilot might not reveal every problem, such as rarer issues, that could still occur in the full survey.

Question 15

(a) B1 a check relating to the equipment, e.g. whether the testing machine gives a reading that can be read accurately/is working correctly
(a) B1 a different check relating to the procedure, e.g. whether the samples are being fixed in the machine consistently/safely, or whether the test takes a practical amount of time to carry out
(a) Answer: She could check that the testing machine gives a clear, accurate reading, and that each rope sample can be fixed into the machine safely and consistently, in a practical amount of time.
(b) B1 piloting the data collection method tests a practical, physical procedure/equipment rather than the wording of questions
(b) B1 there are no respondents/response options to consider, so the checks focus on whether the measurements produced are accurate, safe and repeatable
(b) Answer: Piloting this method tests a practical procedure and piece of equipment, rather than the wording of questions asked to people. There are no respondents or response options to worry about, so the pilot instead checks whether the measurements themselves are accurate, safe to carry out, and can be repeated consistently.

Question 16

B1 a clear recommendation is given, e.g. return to the plan stage to revise the questionnaire before collecting data from the full population
B1 a justified point linked to the plan stage, e.g. the leading question should be reworded to remove bias before it is used again
B1 a justified point linked to the collect stage, e.g. the charity should consider ways to improve the response rate for the main survey, such as reminders or a shorter questionnaire, since only half of the pilot questionnaires were returned
Answer: The charity should return to the plan stage and reword the leading question so it no longer biases responses. It should also review how it will carry out the collect stage for the main survey, for example by sending reminders or shortening the questionnaire, since only 15 of the 30 pilot questionnaires (a 50% response rate) were returned.

Question 17

B1 the claim is correctly judged to be false/not guaranteed
B1 a valid justification, e.g. a pilot can only reveal problems that happen to occur in its small sample; other issues (such as rare problems, or ones caused by using a much larger population) might still arise during the main survey
Answer: The claim is false. A pilot survey greatly reduces the chance of problems, by fixing issues found in a small trial, but it cannot guarantee there will be none, since some problems, such as rare issues, or ones that only appear with a much larger group of people, might not show up in a small pilot.

Question 18

(a) M1 24/40, oe
(a) A1 60% cao
(a) Answer: 60%
(b) B1 it is a leading question cao
(b) Answer: It is a leading question.
(c) B1 recommends rewording the leading question identified in part (b) to remove bias
(c) B1 recommends action to improve the response rate found in part (a) (60%), e.g. sending reminders, or offering more than one way to respond (online and paper), before surveying all 6000 residents
(c) Answer: The council should reword the leading question so it no longer suggests the expected answer, and should take steps to improve on the 60% pilot response rate, for example by sending a reminder or offering both online and paper versions, before sending the survey to all 6000 residents.

Mark my answers