The other important thing I’m going to say is that if Comey’s words are true, then he actually needs to listen OK Election projections put the figure more like 70%. Therefore, this becomes an argument for further predictions.
So, what is a “good” forecast? If we go back to 2016, as you said, Nate Silver’s prediction gave Trump a 30% chance of winning. Other models put Trump’s chances at 1 percent or lower single digits. The feeling is that because Trump won, Nate Silver was “right” for it. But of course, we can’t really say that. If you say something has a 1 in 100 chance of happening, and it happens, that might mean you underestimated it, or it might just mean a 1 in 100 chance of hitting.
It’s a matter of figuring out whether election prediction models are properly tuned to real-world events. Back in 1940, there were only 20 presidential elections in our sample size. So there is no real statistical reason to justify exact probability here. 97 vs 96 – Our limited test scale makes it hard to know if these things are properly calibrated to 1%. I think this whole process is more uncertain than the media, leading consumers to believe in polls and forecasts.
In your book, you talk about Franklin D. Roosevelt’s pollster, who was an early pollster genius — but even his career, in the end, went on fire, right?
This man, Emil Hurja, was Franklin Roosevelt’s pollster and election forecaster. He designed the first collection of polls, the first tracking polls. A very charming character in the voting story. He was very accurate at first. In 1932, he predicted that Franklin Roosevelt would win by 7.5 million votes, although others predicted Roosevelt would lose. He won with 7.1 million votes. As such, Hurja was more accurate than other pollsters at the time. But then he failed in 1940, and later he was basically as accurate as the average pollster.
When it comes to investing, it’s hard to outperform for a long time. Likewise, with polls, you have to constantly rethink your methods and assumptions. Although early on Emil Hurja was known as the “Wizard of Washington” and the “Crystal Gazer of Crystal Falls, Michigan,” his record slipped over time. Or, maybe he got lucky early on. It’s hard to know if he really is this genius prophet.
I’m asking this question because — well, I’m not trying to scare you, but probably your biggest mistake is somewhere in the future, but it’s not here yet.
That’s the lesson here. What I want people to think about is that just because the polls have skewed in one direction in the past few elections doesn’t mean they will be biased in the same way for the same reasons in the next election. The smartest thing we can do is read every poll and look at how the data is generated. Are the questions properly worded? Does this poll reflect American demographic and political trends? Is this outlet a reputable outlet? Is there something going on in the political environment that could lead Democrats or Republicans to answer phone calls or answer online surveys at a higher or lower price than the other? You must consider all of these possible outcomes before accepting data. So it’s an argument that treats polls with more uncertainty than how we’ve treated polls in the past. I think that’s the unspoken conclusion of past elections. But more importantly, how pollsters arrive at their estimates is more realistic. At the end of the day, they are uncertain estimates; they are not basic facts about public opinion. That’s the way I want people to think.
