Previewing the Next NYT/Siena Poll

It’s a busy week in New York Times election-land — we’re wrapping up our second national poll of the cycle.

The last interviews will be complete by the time you read this — the poll is still in the field as I write this — and it should be interesting to see how it contrasts (or doesn’t) with our last poll. In July, in our last survey, President Biden’s approval rating was 33 percent, one of his worst results of the cycle.

But a lot has changed in the last few months. Gas prices have plummeted. Mr. Biden’s legislative agenda was suddenly revived. According to FiveThirtyEight, Democrats have gained around a net three percentage points in the generic ballot, while Mr. Biden’s approval rating has risen by five percentage points.

This Times/Siena poll also has a twist: a Hispanic “oversample,” which is a fancy way of saying that we surveyed a lot more Hispanic voters than we normally do. We’ll have more on this in coming days.

If you’re subscribed to this newsletter — and you should be! — we’ll send you an email with our findings as soon as we get them. We’re probably still a few days from publishing the results, so no need to refresh your inbox just yet.

On Tuesday, I asked whether anyone had a good historical analogy for the way the Supreme Court’s decision to overturn Roe v. Wade had shaken up this year’s midterm elections — an example in which the party out of power achieved the biggest policy success of a president’s first term.

It’s not an exact analogy, but here’s a good answer from Matt Grossmann, a professor at Michigan State University who often has great insights into the dynamics of American electoral politics.

His comparison: the backlash against the Republican effort to impeach Bill Clinton in 1998.

No, it’s not exactly a policy triumph like the court’s overturning of Roe. But if we think of the impeachment through Congress as something like a legislative initiative, you can see the similarity: Republicans were making a major push to change the status quo in Washington, and a backlash against a Republican-favored initiative became a key point in the election.

For Democrats, it’s a pretty favorable analogy: Democrats picked up five seats in 1998, making it the first time the president’s party gained House seats in a midterm since 1934.

Yesterday, the venerable Marquette Law School poll found the incumbent Republican senator Ron Johnson leading the Democrat Mandela Barnes by one percentage point among likely voters.

That’s not exactly great news for Democrats, but there’s a case that it’s decent news for polling in light of the newsletter we published on Monday.

As I mentioned, the polls have shown Democrats overperforming in exactly the same states where the polls overestimated President Biden in 2020. The last Marquette Law School poll was my main example: The poll showed Mr. Barnes faring about eight points better than the FiveThirtyEight “fundamentals” index, a figure that was eerily similar to the polling error last cycle in the state.

Not this time. The new results are much closer to what we might expect for a Senate race in Wisconsin in this political environment, perhaps because Republicans have been directing attack ads against Mr. Barnes since his primary win last month.

Is this good news for pollsters and polling? Quite possibly. Back in 2016, Mr. Johnson led only one poll in the RealClearPolitics average. Donald J. Trump didn’t lead a single Wisconsin poll over the final two months of the race in 2020, even though he ultimately lost by less than a percentage point.

After Monday’s newsletter, a number of people asked why I used 2020, rather than 2018, as the baseline for thinking about polling error. In principle, I’m agnostic about whether 2018 or 2020 ought to be the better comparison. But in practice, that newsletter entry focused on 2020 because the current polling resembles the 2020 polling error — with Democrats often seeming to fare better than they ought to in the very states where the polls erred in 2020.

That’s a little less true after this Wisconsin poll, and that means the “warning sign” is one poll dimmer.

Echelon Research, a Republican polling firm, released some of the most eye-popping numbers of the campaign season on Tuesday. Democratic Senate candidates led by jaw-dropping margins, including 15 points in Arizona, 21 points (!) in Pennsylvania, 10 points in Georgia and six points in Ohio. You can see the full results here.

The wildly favorable figures for Democrats are strange enough, but what’s really perplexing is that the numbers often seem eminently reasonable in the context of the last presidential race in the very same states. Take Ohio: In the Echelon poll, Donald J. Trump led Mr. Biden, in a projected 2024 rematch, by a very 2020-like eight-point margin.

Echelon’s Patrick Ruffini told me that the poll did a lot to control for “super engaged Democrats” who might ordinarily be cited as the reason for left-leaning poll results. The poll did this by weighting on partisanship from the voter file and by adjusting the survey to have the “right” number of voters who said they voted for Mr. Trump or Mr. Biden in the last election.

So what could be going on here? How do you find so many Trump voters who back Democratic Senate candidates? This is a bit of a wonky explanation, but one thing that caught my eye was in the methodology: It’s an online panel that has been matched to a voter registration file.

Online panel surveys are pretty common — you might have seen some, whether by YouGov or Morning Consult. But online data matched to the voter file is somewhat rarer in public polling, even though it’s common in private surveys. Voter-file-matched data is coveted by campaigns, which rely on voter registration files for basically everything, from door-to-door voter contact to modeling and polling.

Is it possible that this kind of data has some idiosyncratic biases? Maybe. Over the last few years, pollsters have been taking note of a variety of biases in typical online panels, like certain people who take surveys over and over and dominate the sample. (Yes, these people do exist.)

The voter-file-matched sample may be especially odd. These frequent survey-takers may answer polls often enough that the sample vendor knows exactly who they are. And since these kind of respondents are coveted by the campaigns, they might be especially likely to have taken a political poll in these same Senate races — they might have even been asked to watch advertisements about these races.

If a group of survey panelists has been exposed to ad tests from dozens of campaigns and PACs, who knows how their views of the Senate candidates might differ from the rest of the population? In the survey world, this is an extreme form of “panel conditioning” — in which the experience of being on the panel changes the attitudes of the respondents.

There’s good reason for pollsters to be experimenting in this area. Telephone surveys are increasingly expensive, and political polling really does benefit from the use of voter file data. Mr. Ruffini might well be right that “this is the future of polling.”

But as Mr. Ruffini added, challenges remain: “The unweighted samples lean to the left.” And it’s not the first time we’ve seen outlying, left-leaning results from an experimental online data source matched to the voter file, like a University of North Florida email poll finding Marco Rubio trailing in the Florida Senate race.

This kind of data may be the future, but it may not be quite ready yet.