A magazine once polled millions on the presidential election - and got the results dead wrong

Donald Trump's supporters have been looking for signs of hope in the polls. Even with a slight tightening of the race nationally this week, Democratic presidential nominee Hillary Clinton has a comfortable lead over Republican Trump. That's among the most statistically rigorous surveys, and even more so at the state level.

Conservative websites, though, have been focusing on less reliable surveys as "shock polls" showing a narrowing race (so much so that the right-leaning Federalist had to debunk the practice this summer).

But to those looking at less savory surveys, remember this: Alf Landon never became president of the United States.

This shouldn't be news, but if you had been a loyal subscriber to a magazine called Literary Digest in 1936, it would have seemed a stunning upset.

Instead, it became an infamous milestone in the history of election year polling.

Today, most polls sample about 1000 people. We know now that if statisticians choose a group that size with demographics and characteristics that match the larger population, they can get a rather accurate view of what people think at that moment in time.

But back then, Literary Digest conducted its poll very differently. It sent surveys to 10 million people. And of those, 2.4 million responded. It may be one of the biggest polls in history. And it was dead wrong.

The magazine predicted Republican candidate Landon would handily defeat the incumbent Democrat Franklin D. Roosevelt:

• Landon: 57%
• Roosevelt: 43%

This is how the election actually turned out that November.

• Roosevelt: 62%
• Landon: 38%

What went wrong?

It had two major flaws:

1. The 10 million people it polled did not look like the population as a whole. This was the Great Depression, but Literary Digest sent surveys to its subscribers, and Americans who owned telephones and cars. Those people tended to be wealthier, and more Republican, giving Landon a false edge. Stats nerds call this "selection bias."
2. Less than one quarter of the sample responded. People who take the time to respond to a questionnaire may be different than those who throw them in the trash. This is called "non-response bias," and it's why pollsters today work hard to combat it.

It was around this time that another pollster was trying a different way to survey the public. George Gallup was getting far more accurate results using smaller samples. Initially, the public was skeptical. But he called the race correctly with a sample of thousands rather than millions. A small sample with the right people is always better than a huge sample that's skewed.

Literary Digest didn't fare as well. It folded a year after Roosevelt took the oath of office again in 1937. What was left of it was absorbed by Time magazine. Gallup, however, is still going.

Read more about what can go wrong with polls: