Data Analysis Australia
STRATEGIC INFORMATION CONSULTANTS
Copyright © 2012
Data Analysis Australia
Statistics in the Media
On a daily basis we are confronted with a multitude of statistics in the media.
In most cases little or no justification is given for these figures and we are left to make a decision on whether to accept them as facts or pursue our own exploration of their validity. Unfortunately the first exposure one has to an issue can influence their opinion and "mis-facts" may become accepted as "facts".
Responsible journalists are expected to check their sources and substantiate their information. In a time when much of what is reported is in the form of numbers, surely that responsibility extends to the numbers they quote. Is it too much to ask that they be held accountable for misleading the public using poorly quoted statistics?
Tourism in Crisis?
Consider the following article posted on the ABC News website on July 15, 2009. There was no author name attached to the article and no reference to the figures quoted.
To explore the validity of the article, an investigation of the relevant Australian Bureau of Statistics (ABS) data is necessary. It is often assumed that it is hard to get more reliable data than that presented by the ABS and there is a temptation to simply take the figures and use them. However, no matter where data is sourced from, it is important for the user to be aware of how the data was collected and any inclusions, omissions or assumptions that have been made.
Today, ABS provides a large range of information and statistics to the general public on its website free of charge. Each table comes with explanatory notes to help the user understand the context in which the data was collected, and any assumptions made. To make informed, reliable inferences and to avoid misinterpretation, there is no excuse for not accessing the full data and its accompanying notes.
Although there is no reference to the actual data used in the tourism article above, it appears that the data relates to the ABS release of a quarterly Survey of Tourist Accommodation (STA). The STA is a census of all in-scope accommodation establishments within Australia. In general, response rates of at least 80% have been achieved. This method of collecting the data suggests that it should be reliable, provided that most employment is in larger establishments. The most current results at the time were released on 14 July 2009 (ABS Cat No. 8635.5.55.001DO001_200903) and refer to the March 2009 quarterly survey. It is natural to assume the article is comparing this release with the previous quarter or some other previous release.
A summary of the results of the STA survey for the March 2009 quarter is given in Table 1, along with the previous four quarters. It is immediately apparent that the article cannot be comparing March 2009 quarterly figures with those from December 2008, as the total number of jobs in WA's regional accommodation sector increased by five between December 2008 and March 2009, while those in Perth decreased by 225.
* These values were not available from the ABS catalogue.
On the other hand when comparing September 2008 with December 2008 quarterly results, the jobs in the accommodation sector in regional WA decreased by 246 while those in Perth only decreased by 72, and so 77% of jobs lost in Western Australia's accommodation sector occurred in regional areas. It appears most likely that these are the results the article refers to, but the relevance of putting them in a July 2009 article on unemployment is questionable.
A more detailed study of the results shows that 291 jobs were lost in the hotel sector of the North West region from September 2008 to December 2008, with other regional WA accommodation sectors actually having an overall increase in jobs over this period. This further demonstrates the need for proper investigation when quoting figures such as these. It is not surprising that the North West has experienced a change between these quarters given the seasonal nature of the tourism industry in the region. The October-December quarter is the beginning of the wet season and consequently the off peak tourist season. On the other hand, the July-September quarter includes the peak tourist season when extra seasonal staff are employed to service the industry. This leads to an expected decrease in tourism employment between the two quarters. Before any real comparison can be made between quarterly data, seasonality must be considered. One way is to use seasonally adjusted data. An easier and in many ways more reliable method is to use year-on-year comparisons.
A comparison of jobs from March 2008 to March 2009 shows that in the Perth area, the number of jobs in the accommodation industry decreased by 428, whereas the number of jobs in regional WA decreased by 146. Along the same lines as the news article, this means that 75% of jobs lost in Western Australia's accommodation sector occurred in Perth over the year to March 2009. Does this mean we should be concerned about Perth's accommodation industry rather than regional WA?
Perhaps, but this highlights another important issue. Is it sensible to look at the percentage of job losses in Perth as a proportion of the total lost in all of WA? As the Perth accommodation industry accounts for more jobs that regional WA it is natural to expect the job losses in Perth to be higher than those lost in regional WA, although perhaps 75% is a little high.
Instead, perhaps it is more meaningful to compare the percentage of jobs lost between March 2008 and March 2009 in the Perth accommodation sector at 5.5% or (428 out of 7,843) with 2.3% of jobs lost in the rural accommodation sector (146 out of 6,383).
Either way, the inference is almost the opposite of that suggested in the article.
Another example can be seen in a recent article on mobile broadband usage in Australia.
The ABS has published the results of the Internet Activity Survey (IAS), which include quarterly figures as at December 2008 and June 2009. (Cat No. 8153.0, Jun 2009). For these quarters, the survey is a census of all Internet Service Providers (ISPs) operating in Australia with at least 1,000 active subscribers. Libraries, Internet kiosks, Internet cafes and hot-spots which provide Internet access on a casual basis were excluded from the census. As with the Tourism example, this suggests that the data should be reliable, although the effect of the response rate that ABS reports (93%) would depend upon the size of the non-responding ISPs.
The article above quotes directly from the summary released by the ABS with the results of the survey. According to the ABS data the number of mobile wireless broadband connections has indeed increased by 51% (up to 1,961,000 from 1,298,000) in the six months from December 2008 until June 2009. Further if we define "market share" as the percentage of non-dialup Internet subscriptions that are mobile broadband, then market share has increased from 20% to 27%, as the article states. Table 2 and Table 3 provide extracts from the ABS summary as well as the percentage change and market share details.
Table 2. Internet Subscribers (rounded to the nearest 1,000) by access connection, for ISPs with more than 1,000 active subscribers in Australia. Source: ABS Cat No. 8153.0.
Table 3. Internet Subscribers (rounded to the nearest 1,000) and percentage by non-dialup access connection, for ISPs with more than 1,000 active subscribers in Australia. Source: ABS Cat No. 8153.0.
There is no question that mobile broadband use is increasing, which is not surprising given it has only become available in the past couple of years, with Telstra introducing the first national HSDPA service in late 2006. However, by discussing "market share", the article suggests that mobile broadband is replacing other Internet connections. The general definition of market share is the percentage or proportion of the total available market being serviced. What is the appropriate market here and how do you measure proportions? In other words, the questions here relate to what the data might mean.
Fixed line broadband is a fast non-portable method of accessing the Internet, while mobile broadband is a relatively expensive, slow portable method of accessing the Internet. Many users find advantage in both forms of Internet access with business especially likely to want portable access to supplement a fast fixed line access. Students in share situations and travellers may use mobile broadband as their sole Internet service, but this may not mean they have taken business away from fixed line services, as a fixed line option may have never been available for them. It is also worth noting that while it was only a 1% rise, fixed line broadband connections have actually increased from December 2008 to June 2009. This suggests that market share should consider customers, not connections. For other purposes, such as designing infrastructure, it might be more appropriate to consider how much is downloaded.
A second feature of the data is that at the same time that wireless Internet use has increased from 6,639,000 to 7,333,000, dial-up access has decreased from 1,286,000 to 1,087,000. It is conceivable that some of the 694,000 rise in wireless subscriptions comes from the 199,000 drop in dial-up subscriptions, that is customers replacing their dial-up access with mobile broadband. However, the ABS data cannot confirm or refute such conjectures.
Hence to form an inference about whether mobile broadband is replacing fixed line service, further data is necessary, data that a survey of ISPs cannot give since a customer may deal with several ISPs. What is needed is a survey of customers themselves. This might not have the precision of the ABS census of ISPs, but it might collect far more relevant data.
A Numerate Mediawatch?
Every day seems to produce new examples of such misuse of statistical data. Perhaps Australia needs a "Numerate Media Watch" to shame the producers of what is really such poor journalism.
 Australian Bureau of Statistics 2009, Tourist Accommodation, Small Area Data, Western Australia, Releases: Mar 2008, Jun 2008, Sep 2008, Dec 2008, Mar 2009, 'Table 5, 9, 11& 13: persons employed by region, data cube: Excel spreadsheet, Cat. No. 8635555001DO001_200903, viewed 21 September 2009.
 Since March 2005, the scope of the STA has been defined as licensed hotels and resorts with facilities and 5 or more rooms, motels, private hotels and guest houses with facilities and 5 or more rooms, serviced apartments with 5 or more units, caravan parks with 40 or more powered sites, holiday flats, units and houses of letting entities with 15 or more rooms or units, visitor hostels with 25 or more bed spaces.
 Australian Bureau of Statistics 2009, Internet Activity, Australia, Jun 2009, Table 1 Internet Subscribers by access connection, for ISPs with more than 1,000 active subscribers, data cube: Excel spreadsheet, Cat. No. 8153.0, viewed 21 September 2009.
 Results are also included for prior quarters but the scope of the survey has changed.
 Australian Bureau of Statistics 2009, Internet Activity, Australia, Jun 2009, Main Features, Cat. No. 8153.0, viewed 21 September 2009.