The Minnesota Poll, here we go again
I spent a good portion of my last post on the Minnesota Poll discussing the issue of question wording and how it can effect the results of polls.
This post will be similar.
Mason-Dixon (3/5, no trend lines):
Minnesota state law currently bans same-sex marriage. Do you think the state Legislature should or should not legalize it?
Should 38
Should Not 53
Not Sure 9
(MoE: ±3.5%)
For supporters of Marriage equality these are not good numbers. In addition they are similar to the numbers from a recent SurveyUSA poll of the issue. A recent PPP poll however paints a completely different picture.
Here are the approval minus disapproval numbers from these three recent polls in a table:
| M-D | SUSA | PPP |
|---|---|---|
| -15 | -12 | +2 |
By looking at the way the three pollsters worded their questions though, it’s easy to see why there is such a wide disparity in the results.
| M-D | SUSA | PPP |
|---|---|---|
| Minnesota state law currently bans same-sex marriage. Do you think the state Legislature should or should not legalize it? | Should the Minnesota state law that defines marriage as between one man and one woman be changed to allow same-sex couples to marry? Or should it be left as it is? | Do you think same-sex marriage should be allowed in Minnesota, or not? |
When people are simply asked if they support same sex marriage or not you get an even split. As soon as you start talking about existing law, and changing it, you get a roughly 15 point swing towards disapproval.
It’s important to be aware of this distinction in how the questions of these polls are worded. This is not to say that the wording in the Hubbard sponsored SurveyUSA poll and the StarTribune sponsored Mason-Dixon poll are incorrect. But they are clearly less straightforward than PPP’s question.
The case of the SurveyUSA poll is an interesting one. This is what I wrote at the time:
This question is not like the others. While all the other issue questions that SurveyUSA asks don’t mention that fact that existing law would need to change, this one does.
The other questions simply ask for a persons opinion on a policy proposal. This question though, goes in an entirely different direction, asking weather the person supports changing the existing law to make room for the policy proposal.
Why not ask, as with all the other questions, weather a person supports same sex marriage or not? Why go through the whole rigmarole about changing the existing law?
This is not an idle question. Question wording goes to the very heart of survey design and can influence a response more so than demographic or partisan characteristics of a sample. As much as possible you want to have consistent question structure so as not to confuse the respondents.
This question is deliberately constructed in a way that is different from all the other issue questions in the survey. I can’t imagine what the reason is for this, but there is no good one that I can think of.
It’s a bit less interesting now that Mason-Dixon has essentially done the same thing. In their questions about taxes not once do they bring up existing law and how it would need to change. Instead, and as is usually the case with polls, they just ask people if they are in favor of a policy proposal or not.
Why SurveyUSA and Mason-Dixon both decided to take a completely different approach to question wording on the single question of Marriage equality will remain an open question for now. But because of this rather curious approach I would be dubious of the results.
Thanks for your feedback. If we like what you have to say, it may appear in a future post of reader reactions.