(C) Daily Kos This story was originally published by Daily Kos and is unaltered. . . . . . . . . . . The 538 and RCP polling aggregates have no credibility: The deep dive [1] ['This Content Is Not Subject To Review Daily Kos Staff Prior To Publication.'] Date: 2024-11-04 I’ve been posting comments about the lack of credibility of the 538 and RCP polling aggregates, and it’s time for a deep dive, with plenty of links, quotes, and screen grabs. If you know anyone who relies on 538 or RCP to frame their understanding of the state of the presidential race, this should be a good place to direct them. The 2022 red wave that wasn’t Remember the big red wave election of 2022? The polling aggregates and the media’s designated Really Smart People Who Know Things told us the Republicans were going to sweep to a large House Majority, flip and take control of the Senate, and win some big governor’s races, too. Of course, it didn’t turn out that way. But here is how some of those Really Smart People predicted the 2022 election: Steve Shepard, Politico's chief polling analyst: Republicans are likely to gain upward of 15 House seats, and they have a good shot of taking full control of Congress. Nate Silver played a cutesy game of arguing against Redd and Bleu on the outcome of the election. It’s not clear if he bet on the outcome: Silver ultimately decided "Redd's case is stronger than Bleu's just because it's much simpler," though "Bleu raises a few solid points." Nate Cohn, to his credit, was more on the fence. Nate Cohn, The New York Times' chief political analyst, believes that the Republican's lead in the House is clear based on public polls, and the Democrats may be facing an increasingly tense battle for Senate. (Yeah, The Week’s editors missed that crime against apostrophes) Charlie Cook: My personal hunch is Democrats suffer net losses of at least 20 seats, but in the Senate, the difference between either party picking up or losing a seat or two could easily be minimal. Frank Luntz: For the first time, the GOP has taken a Senate lead. And in the House, my new projection is 231-236 seats. Emily Ekans of Cato and Fox: Expect a larger red wave than anticipated Scott Rasmussen: Republicans [will take] 53 Senate seats, GOP [will gain] 30 seats in House." And there will be "at least one surprising upset in [the governor] races.” Dave “I’ve seen enough” Wasserman: We believe a Republican gain of 15 to 25 seats is most likely, but it wouldn't be terribly surprising if the Toss Ups broke mostly their way, pushing GOP gains even higher. Oops. To be fair, most of them were basing their predictions on those same aggregates that everyone was watching. They shouldn’t have so done. Pollsters were traumatized by their underestimations of Trump’s support in 2016 and 2020, but may not understand the obvious reasons why they underestimated that support In 2016, when everyone assumed Hillary Clinton was sweeping to a substantial victory, an unprecedented event happened less than two weeks before the election. It’s called the Comey Letter. In a stunning display of arrogance and irresponsibility, FBI director James Comey released to Republicans what he knew they’d release to the public: a letter explaining that a laptop recently obtained in a different investigation contained emails from Clinton that were being reviewed to make sure they didn’t add new evidence to the since closed investigation of her having had a private email server. Yes, it’s mind-blowing to write that. It’s mind-blowing that Clinton’s utterly trivial private email server became a media obsession that, in the end, destroyed her candidacy. So, of course when Comey released his letter the media were like flies on shit. We all remember the infamous New York Times front page that made the letter seem a devastating scandal. A few days later, the FBI reported that the Clinton emails on the laptop were merely copies of emails they’d already seen, adding nothing new to the closed investigation, and the investigation remained closed. Most of the same media outlets that made the letter the biggest story in the news conveniently buried that conclusion inside their papers and as brief asides on electronic media. The damage was done. I still remember the exact word a Clinton pollster used to describe their numbers the weekend after the release of the Comey Letter: gruesome. The article (which I can’t locate online) went on to quote the pollster as saying their numbers had stabilized. I remember that word, too. Because I took comfort in it. What only occurred to me later was that the word “stabilized” was not the same as the word “rebounded,” that the inference didn’t suggest that the Letter hadn’t done any serious damage, rather it hinted merely that the free fall had stopped, settling on a new, lower plateau. Low enough, as it turned out, that 80,000 votes in three critical states flipped the election to Trump. There has been plenty of analysis verifying that the Letter was responsible for Clinton’s loss. The story of 2020 is also easier to explain than many seem to understand. We were in a pandemic. A pandemic that Trump first ignored, then dismissed by saying it would just go away, then told people to combat by injecting bleach. The election came before there were vaccines, before the Biden administration’s astonishingly successful response to the pandemic got two hundred million vaccines into arms within a hundred days of their taking office; and because they aren’t anti-science nitwits, Democrats in 2020 were far likelier to take the health risks seriously. The 2020 Biden campaign took it so seriously that they largely eschewed in person Get Out The Vote efforts: Democrats are banking on this unprecedented push to make up for the lack of door-to-door canvassing that has normally been the party’s strong suit. This year, Republicans have been hitting the streets to reach voters, and the GOP is boosting a formidable ground game operation. A Trump campaign spokeswoman says they have knocked on 35 million doors in total this cycle, and numbers released by the RNC boasted 15 million voter contacts in seven days. Biden and the Democrats, in deference to public health guidelines, waited until October to start knocking on doors, and have only gone out in some states. Instead, they’ve betting big on digital outreach. Most of the virtual staging locations are hosted through Mobilize, a digital platform founded in 2017 to help Democrats organize online. Over the last three years, nearly 4 million people have signed up for more than 13 million volunteer shifts on the platform. Before the pandemic, Mobilize helped grassroots organizers organize their in-person events; now, it has pivoted mostly to digital events, for the Biden campaign as well as other campaigns and liberal organizations. Since May, volunteers have attempted almost 400 million voter contacts through Mobilize, and more than one million people have signed up to volunteer in the four days leading up to November 3. As Markos has stated, a good GOTV effort can be worth a few points, and the Biden team obviously thought their digital effort would be adequate— and it was. But maybe only because they had such a strong lead from the start. The polls saw it as an 8 point race, it turned out to be a 4 point race, and those few points lost by not engaging in a robust in person GOTV effort may have accounted for them. It’s also likely that a good number of Democrats, thinking the race was in the bag and worried about going to polling stations, sat it out. There were also reports at the time suggesting that Biden’s inner circle never believed it was an 8 point race, in the first place. And then came Dobbs The extremist activist Supreme Court majority’s Dobbs decision overturning Roe v. Wade changed everything. Not only did the Democrats have a historically successful mid-term election in 2022, but that trend continued in 2023, as they consistently over-performed their partisan share of registered voters in special elections, to an average of 11 points. As Nathaniel Rakich— of 538— noted: That’s more than just an impressive streak — it’s a potential sign of a Democratic wave election in 2024. In each of the past three election cycles, a party’s average overperformance in all special elections in a given cycle has been a close match for the eventual House popular vote in the eventual general election — albeit a couple of points better for Democrats. Michael A. Cohen: In 2022, Democrats consistently overperformed in special elections before the midterms. Those results offered ample evidence that a Republican red wave was unlikely to occur in November. And, of course, that’s precisely what happened. Perhaps the most important reason for optimism among Democrats is that they are overperforming in elections that are relatively low-turnout and often elite-driven votes. It takes effort to vote in a special election versus a midterm or presidential contest. One has to not only be politically engaged, but also have the time to get to the polls. Traditionally, Republicans have done best in such races because their supporters were more likely to turn out. But over the last several cycles, these voters — who tend to be college-educated and more affluent — have dramatically switched to the Democratic Party. Indeed, this shift of suburban, college-educated voters is one of the most important factors in why Democrats have performed so well in the last three election cycles. But pollsters haven’t bothered to figure out why they blew it so badly, underestimating Democrats in 2022 and 2023; they’re instead obsessed with correcting for their 2016 and 2020 misses, apparently forgetting all about the Comey Letter and the pandemic. Thumbs on the scale There have been plenty of articles about pollsters herding this year, but it was most revealingly exposed by New York Times pollster Nate Cohn on Friday. I appreciate his honesty. Perhaps the very best reason to think the polls might underestimate Kamala Harris this cycle is simply that many pollsters are so concerned — understandably — about underestimating Mr. Trump. It’s hard to overstate how traumatic the 2016 and 2020 elections were for many pollsters. For some, another underestimate of Mr. Trump could be a major threat to their business and their livelihood. For the rest, their status and reputations are on the line. If they underestimate Mr. Trump a third straight time, how can their polls be trusted again? It is much safer, whether in terms of literal self-interest or purely psychologically, to find a close race than to gamble on a clear Harris victory. At the same time, the 2016 and 2020 polling misfires shattered many pollsters’ confidence in their own methods and data. When their results come in very blue, they don’t believe it. And frankly, I share that same feeling: If our final Pennsylvania poll comes in at Harris +7, why would I believe it? As a result, pollsters are more willing to take steps to produce more Republican-leaning results. (We don’t take such steps.) Got that? Despite their consistently underestimating Democratic support since Dobbs, pollsters are most worried about underestimating Trump, and therefore have their thumbs on the scales to make the data look friendlier to him. This isn’t science, rather they’re making completely subjective decisions. Despite 2022 and 2023, and despite Trump consistently throughout the Republican primaries underperforming his polls and particularly underperforming in the often swingy suburbs, the pollsters continue to assume that they’re underestimating him rather than the Democrats. It’s also worth noting Cohn’s parenthetical claim that he doesn’t put his thumbs on the scales the way other pollsters do, because the internals of the NYT/Siena polls have been so dubious this year that Joe Scarborough derisively scoffed at them. And a quick peek at the crosstabs of their final polls, Sunday, shows such odd numbers as their likely voter pool in MI being only +5 women. The 2020 MI electorate was +8 women, so their likely voter model has women turning out less than before Dobbs. The early MI vote this year is +10.6 women. Despite Cohn’s disclaimer, that’s exactly what taking steps to produce more Republican-leaning results looks like. They didn’t release their broader registered voter results, so we can’t even see how their likely voter model affected their results. The aggregates So, back to the polling aggregates that define so much of the media narrative about the state of the race. Simon Rosenberg has been trying to keep tabs on the amount of shit red wave pollsters that are flooding the aggregates, and on Sunday he said this: x The red wave pollsters dropped 17 polls today alone, at least 75 since last Sunday - a majority of polls in final week - Trafalgar & InsiderAdvantage have each released 30+ polls since late August - Emerson poll to counter Selzer in IA today was paid for by "RealClearDefense" 1/ pic.twitter.com/1HJbQaF4yA — Simon Rosenberg (@SimonWDC) November 3, 2024 The aggregators know better. They “weigh” pollsters based on past performance, but when shit pollsters are flooding the aggregates with shit polls, quantity outweighs any possible quality in the weighing. They shouldn’t weigh shit pollsters, they should exclude them. To use the most obvious example, the ridiculous Trafalgar is rated by 538 as the 279th best pollster. 279th! And yet they continue to use them in their aggregates. And as Rosenberg noted, Trafalgar produces a ton of “polls.” They have so little transparency about their internals and methodology that Dave Wasserman, interviewed in yet another article predicting the 2022 red wave that never was, had this to say: I spoke with the founder of the Trafalgar Group last week. He thinks he’s got it figured out. What do you make of his work? I would be happy to offer an opinion as soon as he lets me observe a poll in person, and explains exactly how he gets from survey design to results. My guess is that Trafalgar doesn’t actually poll at all, and just makes numbers up. And yet there they are in the polling aggregates. And pay special attention to their margins in the 2022 aggregates listed below. 2022 Now, for the 2022 aggregates. Which were bad. Really, really bad. And as I wrote a couple days ago, yes, they had some misses on the GOP side, such as having Desantis up only 12.1 when he won by 19.4, launching him into the stratosphere of the media narrative as leader of the Republican Party. Until people around the country got a look at him. But there were many more and much starker misses on the Democratic side, particularly in the battleground states where they often got the winner wrong, and in other states that were supposed to have been getting ominously close in what hadn’t been expected to be battlegrounds— and ended up not being close at all. I apologize that the images aren’t more clear. I took good screen grabs, and uploaded them, but when I embed them they come out smaller than they actually are. I’m not good enough at this stuff to know why. Arizona Governor : 538 Lake (R) +2.4 RCP Lake (R) +3.5 Election Result Hobbs (D) +0.6 Arizona Senate : 538 Kelly (D) +1.5 RCP Masters (R) +0.3 Election Result Kelly (D) +4.9 Colorado Senate : 538 Bennet (D) +8.5 RCP Bennet (D) +5.7 Election Result Bennet (D) +14.6 Trafalgar was off by only 12.6. Oops. Georgia Senate : 538 Walker (R) +1 RCP Walker (R) +1.4 Election Result Warnock (D) +0.9 Michigan Governor: 538 Whitmer (D) +4.8 RCP Whitmer (D) +1.0 Election Result Whitmer (D) +10.6 Yes, Trafalgar actually had Dixon winning, and she lost by 10.6. Oops. Nevada Senate: 538 Laxalt (R) +1.4 RCP Laxalt (R) +1.4 Election Result Cortez Masto (D) +0.9 New Hampshire Senate: 538 Hassan (D) +2.2 RCP Hassan (D) +1.4 Election Result Hassan (D) +9.1 Trafalgar had Bolduc winning, and he lost by 9.1. Oops. Pennsylvania Senate: 538 Oz (R) +0.5 RCP Oz (R) +0.4 Election Result Fetterman (D) +4.9 Washington Senate: 538 Murray (D) +4.7 RCP Murray (D) +3.0 Election Result Murray (D) +14.5 At least Trafalgar had the right winner, here. Murray by a scary 1 point. She won by 14.5. Oops. Wisconsin Governor: 538 Michels (R) +0.4 RCP Michels (R) +0.6 Election Result Evers (D) +3.3 By missing on so many of the key races, and by framing several other races as close when they weren’t remotely close, 538 and RCP played into the narrative that the Democrats were in deep trouble, and a big red wave was coming. And despite missing on so many of the key races and despite embarrassingly framing several other races as close when they weren’t remotely close, 538 and RCP continue to use in their aggregates the pollsters that made them look so ridiculous. And despite 538 and RCP missing on so many of the key races and embarrassingly framing several other races as close when they weren’t remotely close, analysts and pundits continue to use their ridiculous aggregates to define the narratives about the state of political races. 2012 Bonus And as a special bonus, because some of the genuinely smart people I know say this year feels a lot more like 2012 than 2016, I’ll add the RCP aggregate from that year, too. As far as I can tell, 538 didn’t do one. Yes, they had Barack Obama up only 0.7 over Mitt Romney. Rasmussen and Gallup actually had Romney ahead. Politico/GWU/Battleground, CNN/Opinion Research, and Monmouth/SurveyUSA/Braun had it even. Obama won by 3.9 nationally, with 332 Electoral votes. And finally... I can’t tell you Kamala Harris is going to be elected. I can tell you that Nate Cohn implied that the national pollsters have had their thumbs on the scales for Trump all along. I can tell you that the polling aggregators that have shown a coin flip race, with Trump building momentum this month, have no credibility. I can tell you that despite the GOP making a much more serious effort to get their people out early this year, the early vote partisan ballot margins in the battlegrounds look very good for the Democrats, even as the Republicans have significantly cannibalized their previous election day vote, thus reducing the late margins on which they’ve relied. I can tell you that polls of the early vote and polls of late deciding voters show Harris with significant leads. I can tell you that recent high quality polls show registered independent voters favoring Harris, potentially by significant margins. I can tell you that the Harris-Walz GOTV effort may be the most robust and efficient ever, while the Trump-Musk GOTV effort appears to be as successfully innovative as a hyperloop. I can tell you that for the past week or two, news reports have suggested a quiet confidence growing in the Harris campaign, and the Trump campaign already starting to find people to blame. I can tell you that campaigns are able to poll with much greater specificity than national polls can, and often have a better sense of the state of the race than do national pollsters and the pundits who depend on them. I can tell you that for the past couple weeks, Kamala Harris has looked and acted like someone who knows she’s winning. I can tell you that for the past couple weeks, Trump has looked and acted like someone who knows he’s losing. He wouldn’t already be whining about the election being stolen if he thought he was winning. Since I began drafting this post a couple days ago, the gold standard Selzer Iowa poll released a bombshell showing Harris winning Iowa by 3. This followed an also shocking high quality poll from deep red Kansas showing Trump up only by 5 there. Don’t be surprised if the national polls start herding to show Harris with sudden momentum. They don’t want to be caught flat footed. And if they’ve had their thumbs on the scales all along, all they need to frame a narrative of late momentum for Harris is to take their thumbs off the scales. Maybe their numbers have shown her winning all along. If Democrats continue to turn out the way they have been, if those who haven’t yet voted get to the polls, this will not be a close race. By any credible metric by which we gauge the state of a presidential race, Kamala Harris is winning. Potentially by a comfortable margin. The polling aggregates are not a credible metric, and yet they continue to define the media narrative. At least one pollster already is panicking to adjust his numbers, so he doesn’t look like the fool he has been. Let’s see if others catch up. Let’s ensure that Democratic voters vote. [END] --- [1] Url: https://www.dailykos.com/stories/2024/11/4/2281797/-The-538-and-RCP-polling-aggregates-have-no-credibility-The-deep-dive?pm_campaign=front_page&pm_source=trending&pm_medium=web Published and (C) by Daily Kos Content appears here under this condition or license: Site content may be used for any purpose without permission unless otherwise specified. via Magical.Fish Gopher News Feeds: gopher://magical.fish/1/feeds/news/dailykos/