Surveys have often suffered from a focus on “bigger is better”. Today, internet based surveys are promoted as “fast, low cost and informative”, and the low cost has often led to large sample sizes that give an impression of accurate results.
The fallacy of large sample sizes giving accuracy was dramatically shown in the polls of the 1936 US presidential election. The Literary Digest, a well-respected magazine and one that had a history of accurately predicting the winners of presidential elections, conducted its poll by sending out 10 million post cards asking people how they would vote. Approximately 2.3 million were returned. The overwhelming conclusion from this poll was that Alfred Landon (57%) would win over Franklin Roosevelt (43%). History tells us this was not the case and Roosevelt won convincingly over Landon. What went wrong?