Looking at Monday’s polls of the Alabama Senate race, Republican Roy Moore is either up 9 points, according to Emerson College, or Democrat Doug Jones is up 10, according to Fox News. Both Fox and Emerson produce reputable polls that have a decent track record. But, they can’t both be right. One of them is spectacularly wrong.
The 19 point gulf between Fox and Emerson’s results probably has a lot to do with differences in the methods used to poll people and who pollsters assume will show up to vote.
“There are, to oversimplify a bit, two important parts to polling,” the Washington Post’s Phillip Bump writes. “The first part is getting a sense for how people plan to vote. The second part is matching those results to who actually will vote.”
A tough nut to crack
Special elections have relatively small voter turnout. Figuring out which group of people will actually show up is a tough nut to crack. In polls of conventional races, past election results are usually a pretty good predictor of the electorate. Polls are usually closely grouped because they make roughly similar assumptions of the voters that turnout. Not so here.
In the Alabama race, unprecedented national media attention and salacious allegations throw an additional monkey-wrench into predicting an already difficult to predict special election turnout. Are Democrats more fired-up to defeat Moore? Are Moore voters shy about revealing their intention to vote for an alleged sexual predator?
How pollsters answer questions like these will strongly impact their results. Given all the uncertainty, Monmouth University’s pollsters threw up their hands and simply released results under a variety of turnout scenarios. Under their 2017-based turnout model, Moore and Jones are tied at 46%. Assuming the electorate looks like they did in the 2014 mid-terms, Moore wins 48-44. But, if the electorate reflects voters in the 2016 election, Jones is up 48-45.
Survey Monkey took a similar approach, releasing results for a range of turnout assumptions . “Minor differences in the methods used to model or select the likely electorate produce wildly varying estimates in Alabama,” Survey Monkey CEO Mark Blumenthal wrote. “Data collected over the past week, with different models applied, show everything between an 8 eight percentage point margin favoring Jones and a 9 percentage point margin favoring Moore.”
Blumenthal reckons that releasing a range of results is probably a better way to do it in an unconventional race like this. The lesson from 2016, he told Politico, is that pollsters were too preoccupied with prognostication. “We should not be getting into the minutiae of what fraction of a percent the lead is for one candidate or the other,” he said.
Compounding the problems, pollsters are finding it increasingly difficult to get a representative sample of voters, which is forcing greater reliance of weighting to get their sample to reflect how they expect the electorate to look. For example, pollsters may have trouble reaching African-Americans in sufficient numbers, so will “weight” the votes of the ones they do have more heavily. This adds a degree of subjectivity into the mix.
Another issue that seems to be causing a lot of variation between polls is how the questions are asked. As Nate Silver of FiveThirtyEight explained in an extensive analysis Monday, automated polls that auto-dial voters and have them respond by pressing keys or answer computerized voice prompts, have tended to show Moore ahead. Polls using live interviewers have tended to favor Jones.
As Silver points out, one big difference is that automated polls are prohibited by law from calling cell phones. Because just over half of Americans no longer have a land-line, IVR polls exclude a substantial number of people. And, those with only cell phones tend to vote differently. Fox News’ poll showed a 30% advantage for Jones among cell phone callers. Automated pollsters are not unaware of this problem and use weighting to try to compensate for its effect. But, as fewer and fewer people have landlines, it’s possible that automated polls are undercounting a meaningful slice of the electorate.
However, there is also some thought that people might be less willing to cop to voting for a controversial candidate when talking to a real person on the phone. While studies have shown that the evidence for this is weak, in the unique situation of a candidate accused of molestation of teenage girls, perhaps it could be a factor. If so, how people answer automated poll questions might better reflect how they will behave in the voting booth.
“It’s worth keeping in mind that one difficulty in polling Alabama’s electorate is that very few, if any, pollsters have a track record there.” Monmouth’s Patrick Murray told Politico. “This lack of familiarity is further compounded by the unpredictability of special elections.”
Analysts say Moore is the likely winner…maybe
Polls show Democrats have an advantage in excitement about the race. In Fox’s poll, among likely voters, 50% of Democrats said they were “extremely interested” in the race versus 45% of Republicans. Among likely voters who were “extremely interested,” Fox’s poll has Jones ahead 53-40, a commanding 13% lead. But, enthusiasm expressed to a pollster doesn’t always translate to actually going to the voting booth, and reliable voters for Republicans may just be unenthusiastic about their candidate but turn up and vote for him anyway. And as weird as this election year period in history is, a good many may turn up tomorrow and make up their minds as they walk in. Fox News’ poll found that 8% were still undecided. That could be significant.
If a normal electorate for an Alabama special election turns out tomorrow, Moore probably wins. That leads most analysts to give the edge to Moore in spite of Jones’ lead in the Fox Poll. Nate Silver says that Moore is more likely to prevail, but cautions that Jones has about as good a shot at winning (30%) as Trump did in 2016. And we all know how that turned out. Monmouth’s Murray agrees, adding that “there is still an opening for Jones.”
Still, the bottom line is anyone who tells you they know for sure what’s going to happen Tuesday is probably just guessing.