A lot of what I have said in the past few weeks may give you the idea that I think Nate Silver is a fraud, or at best a non-serious person. Neither of those things is true.
I recently have had, let us say, some of my fellow-traveling partisans tell me they find Nate to be those things, and I have had to defend him.
Nate Silver is a very smart and honest guy. A very impressive guy, really.
Two recent statements of friends I normally agree with have proved Nate’s “coupling” hypothesis about “Village liberals” sort of correct:
- “[…H]e’s no longer a reliable pollster and is no longer at 538. He does non-professional-pollsterish stuff like discounting convention bounces qualitatively, according to people who know about his methodology. His polls are no longer included in legit aggregated polls….”
- “I’ve seen him interviewed about this— a serious pollster would explain where things went wrong instead of taking up a side gig about gambling. Maybe he’ll make more money with this book but to me, this just makes me take him less seriously.”
Several points here:
- First of all, Silver is not a “pollster” at all. He is an aggregator of the polling of others. He has created a model that (among other elements, such as demographics) incorporates the measured accuracy of all the polls he uses, weighting them according to their previous validity (or lack thereof). (In addition, he created 538, then left it in 2023, ten years after selling it, presumably for a large amount, to ABC/ESPN.)
- Second, Nate’s “qualitative” treatment of things like “convention bounces” (which, again to be fair, is not really represented entirely accurately here) is part of the whole Bayesian approach. With Bayes, you start with certain prior assumptions, and you alter them as events come in. There’s a qualitative element to that to start with – I think in this case, he openly stated that he was not including the normal Democratic Convention bounce in his model, ironically because it tended to make Kamala Harris’ odds look worse (the bounce was far smaller than previous ones, because the electorate is so polarized that little movement can be expected from even a very successful convention anymore).
- Third, his life course is pretty much the opposite of what the second commenter was saying. He started off doing baseball statistical analysis, as I recall, and (following Bill James) created a lot of the field of “Sabermetrics,” modeling likely baseball player performance. Then he got into on-line poker, made a bunch of money initially, and then, as he said in The Signal and the Noise, lost it, and got into modeling political races (again, NOT polling – he rates other people’s polls and incorporates them into his predictive models).
- Fourth, as a predictor, he is about as good as it gets. Who else had Trump with anything like as high as a 29% chance in 2016? Almost no one, certainly no one seen as serious. 29% chances come in all the time.
So Nate Silver did NOT fail in 2016; prediction itself failed. It was useless and paralyzing.
I think the fact that he is now back into playing poker – and winning at it, ranked in the top 100 in the world recently – makes him, if anything, more impressive. He’s still handicapping the presidential and congressional races, though you have to pay to see what his real latest odds are.
I think he’s been “explaining where things went wrong” for eight years now, and he’s TECHNICALLY right that the failure was overwhelmingly on the part of those consuming his polls.
HIS failure was that he thought he was in some Bayesian statistical salon discussing the arcana of probability theory with well-educated peers who were well aware that the essence of probabilistic forecasting is UNcertainty.
Instead, he was, almost certainly unwittingly, chumming the waters with hope for non-mathematicians, purveying an addictive drug to voracious certainty-addicts.
Once we knew that Trump COULD win back in 2016, we all should have logged off of Nate’s models and started planning for the apocalypse (either way – the one thing both sides agreed upon was that the election of the other candidate might be the end of the world).
We didn’t, of course… so Nate got a lot of unexpected bitterness in response.
P cubed: Polls Paralyze People.
