In areas covering a wide variety of issues such as health, road safety, transport use and consideration of the environment, there are campaigns to change the way that people behave.  It may be to drink less alcohol, to drive more safely, to use the car less or to reduce energy consumption.  Perhaps the most famous in the Australian context were the ‘Life. Be in it.’ and the ‘Slip! Slop! Slap!’ campaigns.  In all these situations there is a need to assess how well the campaigns worked, in order to measure their effects and to determine how to do better next time.

Measuring behaviour change is always a challenge because people are rarely objective about their own behaviour.  Far too often people believe – or at least say they believe – that they have “done the right thing” and made a change for the good when in reality they have made little or no change.  If change has occurred, it can sometimes be difficult to say it was due to the campaign and not due to something else such as changes in economic circumstance, health, or even the weather.

Data Analysis Australia uses two key principles when evaluating behaviour change.  

The first principle is to measure real and relevant outcomes.  For example, in road safety, it is a reduction in crashes, injuries and fatalities.  In water efficiency projects, it is a measurable drop in water consumption.  Focusing on real change removes the subjectivity of both the public (“we think we have changed”) and the campaign personnel (“we think they have changed”).  

The second principle is to use proven statistical methods for eliminating extraneous influences that may be confused with the effect of the campaign.  Statistical solutions include:

• The use of a control group — a separate but similar group not exposed to the campaign, but likely to be exposed to similar extraneous factors; and 
• The use of a statistical model to adjust for known extraneous factors — for example a regression model for the effect of weather.  

Sometimes change can be measured using existing data sources.  Data Analysis Australia has often used large billing databases for campaigns such as water conservation, as the billing data shows real measures of the relevant change.  Existing historical data can often establish good baselines against which to measure change.

To measure behaviour change, data is needed from both before and after the campaign or intervention.

To measure change, the need to have data from before the campaign may sound obvious, but it is still easily overlooked in the rush to start a campaign.  Obtaining this data may involve the consideration of ethical and other non-statistical issues and should take into account the type of statistical methodology going to be used.  If data is not already available or existing data is inadequate, time must be allocated to collecting information to establish a basis for comparison.

Data Analysis Australia has worked on many behavioural change projects. An example of one that required data to be collected before an intervention in order to create a baseline data set was the TravelSmart program, which was designed to increase people’s awareness of their travel options and encourage travel choices which would reduce their use of the private car. To measure travel behaviour change associated with the TravelSmart campaign, Data Analysis Australia used a longitudinal survey.  For each household, actual travel was recorded for one day of the week before the campaign and on exactly the same day of the week exactly one year later.  This was done for the population affected by the campaign and a control group that would be subject to the same trends in travel, allowing the differences in the respective changes for the two groups to be statistically evaluated, thus satisfying the two principles above.

For further information, please Contact Us.

June 2013