Stace’s one-question poll

On Monday night, Stace Nelson’s U.S. Senate campaign put out an automated call to 837 South Dakota Republican households and got results that made Nelson pretty happy.

In a head-to-head matchup between Nelson and frontrunner Mike Rounds, 40 percent of those longtime Republican primary voters picked Nelson. Rounds had a double-digit but surmountable lead at 54 percent support.

"It’s appearing that our hard work is paying off, and the more South Dakotans are understanding the choices at hand, the more that they are looking for the conservative alternative," Nelson said.

Except there’s a lot more to the story of this poll, a perfect example of why polling numbers shouldn’t simply be taken at face value — particularly when it’s internal campaign polling. That’s because unlike public pollsters, internal polling isn’t released whatever the result is. Campaigns only release the results that make them look good. So even if their polling is perfectly scientific and accurate (and even perfectly scientific polls will fluctuate up and down within a range based on issues of sampling) internal polls can’t be trusted as an accurate picture of the race.

(For a similar anecdote about the data you have vs. the data you don’t, read the story of Abraham Wald and the shot-up British bombers.)

There’s also more to Nelson’s poll. The night before, a South Dakota voter tweeted at me that she had received a poll she thought was Nelson’s, a single question with unusual wording:

When I asked Nelson about this, he said that sounded like his poll, which was just one question.

Obviously, this is what’s called a loaded or slanted question. If a public pollster like Gallup were polling the South Dakota race, they’d simply ask respondents whether they preferred Mike Rounds or Stace Nelson in a head-to-head matchup. Asking Republican voters to choose between the “conservative Stace Nelson” and the “moderate Mike Rounds” nudges any of them who identify as conservative to pick Nelson.

So as a snapshot of what voters are feeling now, this poll is pretty useless. That’s not to say it doesn’t have uses — if viewed as a message-testing poll, it could be held up as evidence that South Dakota Republicans, if persuaded that Stace Nelson is the conservative candidate and Mike Rounds is them moderate, are relatively closely divided between the two despite Rounds’ much higher name recognition. And that message is exactly what Nelson is going to be spending the next nine months telling people from one end of the state to the other. (Though it’s worth noting that even with this leading question, Nelson didn’t win — an indication of the huge built-in advantages Rounds has.)

Whenever I write about polling in this manner, I like to repost a clip from an old British comedy show, “Yes Prime Minister,” in which the characters discuss how you can make polls show whatever you want them to say by shaping the questions:

(Also worth noting: Nelson didn’t ask at all about other GOP candidates Larry Rhoden and Annette Bosworth. “We’re not worried about the other two campaigns,” Nelson said. “Neither one of them are conservatives.” Both Rhoden and Bosworth dispute the latter statement.)

Furthermore, Nelson’s poll is not particularly scientific. It was only one question long. Real polls, even ones interested in only one answer, ask several demographic questions — the respondent’s age, race, gender, etc. In addition to giving you interesting breakdowns — maybe the poll would show that voters over 65 support Nelson more strongly when told that he’s the conservative and Rounds the moderate, which could then let you focus your message on seniors — that also lets a pollster check his sample against the broader population. If your sample is 60 percent male while the population is 50 percent male, then males are oversampled. And if males, say, were more likely to support Nelson than females wore, that oversampling of males would boost Nelson’s support. So a professional pollster would use that demographic data to weight his results so female respondent’s answers counted more heavily in the final mix, to produce a more accurate result.

Now, Nelson says he’s planning on conducting “more detailed polling” in the future, which may give him better results.

But from what Nelson told me, it’s also fair to say his polling wasn’t just intended to get a result Nelson can use to argue that the race is tight.

"We wanted to get out to those who aren’t aware of our campaign, and get the name out there," Nelson said. "We want to be pushing the fact that we’re the conservative in the race."

The key phrase in that quote is “pushing the fact.” There’s a very good reason to view this as, at least in part, a “push poll” — by the correct definition of the term. A push poll isn’t a synonym for a “slanted” or “biased” poll (though as discussed above this is also that). It’s a poll whose primary purpose is not to get data from the electorate, but to push out a message to voters. It is, as pollster Mark Blumenthal wrote, “a telemarketing smear masquerading as a poll.”

Now, as telemarketing smears go, calling Rounds a “moderate” is not the most vicious stuff, though it does indicate the vitriol with which Nelson and certain segments of the Republican base view the term “moderate.” The classic example of a push poll in American politics was in the 2000 GOP presidential primary, when South Carolina voters received a “poll” asking them, ”Would you be more likely or less likely to vote for John McCain for president if you knew he had fathered an illegitimate black child?” (He had not.)

One grain of salt to this whole theory: a true push poll in this case would have dialed far more than 837 Republican households. It would have called tens of thousands of households. So this poll can’t be dismissed as merely a telemarketing ploy. It’s got some limited value as a message testing tool. But a better message test would have asked voters who they support with neutral language, then given them a bunch of negative information about Rounds, and then asked, “With what you know now, who do you support?” In fact, someone — we don’t yet know who — conducted a poll like that recently in the South Dakota Senate race.

So what do you take away? Rounds has some vulnerabilities if his opponents can persuade voters he’s not conservative, though even that (a charge Rounds will be firing back against, do not forget) doesn’t actually make Rounds trail. Candidates in South Dakota are experimenting with all sorts of techniques as the race ramps up. And polls, while a very useful tool, are also ALWAYS deserving of scrutiny to determine exactly what information you should and should not take away from them.