Surveys are critical in informing decision making in Australia, particularly in the health and community services sectors.  In such contexts it is crucial that the surveys give a truly representative picture, while at the same time not being prohibitively expensive to carry out.

Telephone surveys have filled this role very well for the past thirty years, when virtually every Australian household had a standard phone.  The technology of the telephone made this possible, largely replacing the more expensive door-to-door surveys of the past.  Such surveys are still not easy, involving many issues of sampling, follow up protocols for numbers that do not answer, how to handle answering machines and much more.  Additionally, the growing proportion of unlisted numbers has led to the development of the more expensive Random Digit Dialling (RDD) procedures used in cases where it is essential to get greater coverage.  

Now further changes in technology are creating new challenges. An example is the recent phenomena of "mobile only" households where virtually all calls are made and received using mobile phones.  There might be a landline, but it is only used for the Internet or a fax machine.  Traditional phone surveys will generally miss these households.  This is not a major problem if the survey subject matter can be assumed to be independent of whether the household is "mobile only" or not, or if the bias can be managed during analysis.

However, recent research has suggested that people in "mobile only" households in Australia are different for some subjects such as health.  They are likely to be young (and hence healthier), more likely to smoke (less healthy) and more likely to engage in risk taking behaviour such as consuming excess alcohol and taking illicit drugs. Traditional telephone surveys could give very misleading results for these and other areas. 

Including mobile phones in a survey can be problematic. Surveys often target a specific geographical area, but for mobiles there is a disconnect between numbers and location - in contrast landlines have an exchange prefix that indicates the rough location. Mobile phones also have the property that they are attached to a single person rather than a household, giving a very different sampling perspective. If a random number dialling system is used with mobiles, a problem arises that every wireless data device ranging from an iPad to a dongle has a SIM card with a mobile number. Even if a list of mobile numbers is available for sampling, there is no way of knowing the time zone it might be in (or, with roaming, even the country), and safety considerations include calls received when the phone user is driving or in a dangerous work environment.  

Beyond mobile phones there are even newer technologies.  Voice over Internet Protocol (VoIP) and Skype create virtual telephones that might not even have numbers and are almost impossible to sample.  There is also a coming generation for whom voice itself may come second to text messaging. 

There are multiple solutions to these problems.  One is that surveys should consider several modes of response - traditional phones, mobile phones, mail and the Internet. Surveys should also consider multiple sampling frames such as having an intercept sample as well as a telephone sample.  Thought must be given to the "hard to get" segments and sometimes additional items of information need to be included in the questionnaire to allow for bias adjustments.  When making survey recommendations, Data Analysis Australia seriously considers these issues, while still recognising practicalities and "fit for purpose" solutions. 

Data Analysis Australia is keeping at the forefront of these developments in Australia.  We were invited to contribute to a recent workshop into these issues organised by the Social Research Centre of Melbourne and the Institute for Social Science Research of the University of Queensland. The workshop brought together a number of stakeholders concerned about best survey practice in Australia, as well as a major report from Professor Paul Lavrakas on his experience as chair of the Cell Phone Task Force for the American Association of Public Opinion Research.

For further information, please Contact Us.

June 2011