Since we talked about something so mechanical and technical for the last two columns, today, we're going to talk about something that doesn't involve SPSS nor does it involve any kind of programming.  Today, we're going to talk about why every poll that isn't a push poll uses such milquetoast, mild language when they ask questions.

There's more.

To just about anyone reading this site, politics and geekery are probably like hydrogen and oxygen: essential to life.  Because of that, we've probably watched debates and become angry at how moderators frame questions in such a way that's not directly beneficial to our point of view, and we've probably had the same reaction when looking at polls. When you see questions like this, it's hard not to get irritated!

When thinking about the war in Iraq, do you think that Senator McCain showed political courage in opposing President Bush's and Secretary of Defense Rumsfeld's policies and calling for "the surge"?

1. Yes, a lot.

2. Yes, somewhat.

3. Not really

4. No, not at all

5. Don't Know

To any Democrat, and most Republicans, who is politically engaged, it's obvious how many false assumptions are packed into that question.  To those Democrats reading the site, please, please hold back your rage.  There's a reason that the question was asked that way, and it's not to carry water for McCain or to push the themes of the RNC's biography driven adverts.  The question is asked that way, because for better or for worse, that's how people think of the issue. If you want an accurate measurement, you have to ask questions that accurately reflect the general sentiments around an issue.  While most of us would like to see the question asked as follows, it probably wouldn't help us at all, and would introduce another problem.

When thinking about the colossal disaster that is our Smirking Chimp President's illegal invasion of Iraq for craven political purposes, to what degree do you think that Senator McSame's crass posturing about disagreeing with Rumsfeld and Shrub while at the same time claiming that success was within a Friedman Unit to anyone who was listening not only reflected what a douche the guy is, but was also a transparent ploy to set up his Presidential campaign that we all knew was coming?

1. Absolutely. What a jerk.

2. It was posturing, but I don't think he was trying to set up his campaign. He's not that smart.

3. Honestly, I hate him so much for his environmental policies that I hadn't even thought about that.

4. Why does anyone care about that? I hate him for making Joe The Plumber so popular he got a country music album.

5. I'm a Paultard.

This question, while pretty much summarising what most of us really think, would not give us an accurate measurement of the sentiment of the public at large because the assumptions in it differ so much from the way that most people think about the issue. Ultimately, this leads to the real problem: asking questions like that creates an incentive for your respondent to answer in a certain way, or, to use the language of social science, it introduces response bias.   Once people know how you want them to respond, they will either respond in that way or terminate the interview.  Either way, that interview is now useless to you.

When polling, you are trying to collect information on how people actually think, not convince them. Convincing people is what phone-banking and canvassing is for, and doesn't actually require a pollster. The only utility that your poll has is the accuracy of its measurements. Now, obviously, you're never going to have a bias free poll.  Most people don't just sit around thinking about politics and issues for ten or twenty minutes at a time, and the simple fact that they're doing that while being polled could affect the way in which they answer their questions.  Maybe. It's contentious.  But the simple fact that we can't make a poll 100% bias free doesn't mean that we shouldn't control what we can.  There's no need for unforced errors!

This, ultimately, is why you should write your questions in such a way that they're not likely to offend anyone. You don't want the response of someone when he's offended and thinking about the issue defensively or the response of someone who's asked to think about the issue in a way differently than he currently does - you want the immediate, reflexive response.  This tells you how the respondent actually thinks about the issue.

So, as much fun as it would be to have polls with questions exactly like the one I wrote, it's never going to happen.

Dirty D