We are all aware of magazine and television "surveys" where the public is invited to write or phone in to give their opinion on a current issue. Often the results are extreme. This is partly because only a very small proportion of the public who could participate actually do, and those who do often come from a particular viewpoint. These are examples of surveys with a very low response rate, often well under 1%. They represent an extreme example, however many other surveys also suffer from low response rates, and their results may also be misleading.
In contrast, some surveys are quoted as having a "high response rate". But what exactly is a response rate? And how does it influence the quality of the survey results?
The response rate of a survey is a measure of how many people approached, (i.e. 'sampled') actually completed the survey (expressed as a percentage from 0% to 100%). It is usually assumed that the higher the response rate, the more likely the results are representative of the population, provided the sampling is appropriate in the first place (and that people who don't respond are roughly the same in their opinions as the people who do respond).
Most people assume that if everyone approached took part in the survey then the results are representative. Actually, it depends on the way that the sample was designed. Sampling theory establishes procedures to ensure samples are chosen to avoid biases and, just as importantly, quantifies how different a sample might be from the whole population.
Response rates are strongly affected by the method of data collection. In general, the more interaction between the potential respondent and the people collecting the data, the higher the response rate. There are at least four common data collection methodologies:
- Self-completion and mail back
- Telephone interview
- Face-to-face interviews
- Internet questionnaires
Self-completion surveys often have lower response rates than telephone or face-to-face interviews, because the respondent is left to return the survey. It is also more work for the respondent, as they have to fill in the questionnaire themselves without assistance, and then return it to the research company.
What is a Response?
When someone is approached to participate in a survey, there are a number of possible outcomes. This is an area of survey theory with confusing and sometimes conflicting terminology, but a reasonable set of definitions are: