Building a Poll

This is the first on-going series by Dirty D. In this series, we will discuss the concepts that go into designing a political poll and how to put things together.

This will appear regularly on Friday mornings.

What is the difference between public polling and strategic polling?

Tell anyone that you work in polling and the first response is to ask about what the latest Gallup numbers actually mean. (A close second is to ask what the phrase “margin of error” actually means, but that's a whole different conversation.) There are a number of differences, but these are the most important:

Building a Poll Part 1: Thinking Through a Poll

In my previous post, I discussed what I hope to be an ongoing series of posts: Building a Poll. What I hope to discuss in this series is everything that goes into fielding a poll from step one, which is saying, “Hey, we should poll!” to the final step, which is saying, “Hey, that was expensive, and I don't like what I'm seeing!” (Just kidding. Sometimes, people don't find it that expensive.) The important thing in polling is to determine what you want to know, from whom and when. You should also have a pretty good justification for why you want to know this, and we can figure out the how once you've answered everything else. In some sense, fielding a poll is like being a journalist conducting an interview, but on a massive scale. Try to imagine five hundred(1) junior reporters conducting structured interviews with interview subjects, and a senior reporter then looking at all the collected data and making sense of it.

There's more....

Building a Poll Part 2: Why Do People Poll?

Admin Note: This post was submitted in time, but accidentally left in draft form. I have recently discovered the error and moved it to the front page.

 

We now get further into the role-playing aspect of this series. Let's pretend that you're trying to design a polling strategy for a Democratic Congressional candidate running for an open seat in the state of Mississippi. It is going to be in time for the 2008 elections, and not a special election.  You are the campaign manager, and it's your job to figure out what you need to do with the polls.  You have no primary opponent, and are able to begin running against the Republican from day one.  In fact, your candidate had an exploratory committee up and going by March 07, and declared his intentions for the seat in June 07.  The Republican incumbent of fourteen years is retiring, and they're facing a primary of three candidates.  Their primary is not going to be messy and fun. It's going to have the party leadership step in and make their executive decisions, so you'll be running against someone by January 08.

There's more...

Building a Poll Part 3: Catching up with all our assumptions

This has been a fun series to write so far, and I'm hoping that all of you are getting as much out of reading it as I am from writing it. In my few posts so far, we've stated a few assumptions and a few things for role-playing. I'd like to collect this all together and put it all in context.

There's more...

Building a Poll Part 4: How do I know what to ask about?

There is one thing about polling that makes it incredibly frustrating for a pollster and great for journalists and other pollsters who seek to discredit your findings: there is no such thing as a perfect methodology.  There is absolutely nothing that you can do to ensure that your poll is perfect. Not too long ago, The American Prospect had an informative review of a recent book on the war on science. I'd advise that you take a look at it and think about it before we proceed.  Your poll will never be perfect - come to terms with this. Someone will always be able to complain that you should have asked X or should not have asked X, that the ordering of your questions introduced bias into the poll by creating incentives for certain responses, etc.   Some of these complaints are nothing more than standard trolling and attempts to discredit the findings of a poll as described in the article I linked to, but some of them are legitimate. So long as your poll falls into the range of best practices (which we're going to be describing in this blog), you're fine.

 How do we know what to ask about? 

There's more...

Build a Poll Part 5: Using Your List Strategically

Today's post brings us to a fun intersection.  So far, my colleague,Blue Leader, and I have been writing in parallel.  Our posts appear around the same time, but they don't really intersect with each other.  This is the post that you've been waiting for! The clash of the Titans! The Rumble in the Jungle! Okay, it's nothing like that.  In fact, it's really nerdy.  What we are going to discuss today is making strategic use of your voter file when designing your poll, which brings us to an intersection with Blue Leader's excellent series, Building a Voter File.

In my last post, we started the discussion of knowing what to ask about.  You obviously need to divvy up your time between a certain amount of dependent and independent variables, and that's what we're going to talk about today.  Stepping into your role as campaign manager, you want to commission a message testing poll. You only have the money for 500 respondents at fourteen minutes length.  You don't want to cut out any of the dependent variables you're testing, but at the same time, you don't want to lose that valuable contextualising information.  What are you to do?   This is where having data in which you have high confidence becomes invaluable.

There's more...

Building a Poll Part 6: Using Your List Strategically (Continued)

Admin Note: We apologize for the tardiness of this post. Dirty D got swamped in his day job and forgot to send me his contribution for editing and posting.  We'll try to be on time in the future.

In my last post, we began a conversation about using your list strategically.  This was the beginning of the intersection with Blue Leader's awesome series, Building a Voter File. I'd like to continue that conversation today.  When we left off, we were talking about using variables on a list in lieu of questions in a poll, and how you could trust what's on there.  I'd like to continue that conversation a bit today.

There's more...

Building a Poll Part 7: Evaluating Lists

These last few installments in these series have focused heavily on how a pollster makes use of the lists given to her, and it's for a pretty good reason.  That reason is not, by the way, to promote Blue Leader's series on Building a Voter File.  The reason is that polling would really, really, suck if there were no lists. This is why our tag line reads "Analytics, data, modeling..." these three fields are so closely interrelated that I'm tempted to call them areas of specialty in one large area, and not three distinct undertakings.  Without list vendors out there, we pollsters would be severely limited in what we could do.

This brings us to what I'd like to talk about today, which is how you pick a data vendor. 

There's more...

Building a Poll Part 8: Random sampling

It's been a while since we've had an update in the Building a Poll series, and I'm excited to pick it up again. When we last left off, we were talking about using a list strategically. I'd like to continue on the subject of using lists, but we're going talk about something more mechanical aspect of using the list: building a random sample.

Granted, if you're a pollster, you're likely to have purchased a random sample from your data vendor, but it never hurts you to understand the mechanics of how this works.  Heck, maybe one of your clients stuck you with a membership list or some other targeted list and asked you to call a random sample of that list.  You can't very well send that list off to your data vendor and ask them to sample it for you!

There's more...

Building a Poll: More on random sampling

Hi, everyone.  Last time, we discussed the mechanics of random sampling in SPSS. To summarize, we discussed how you randomize a list and then how you pick out some 500 records from the list at random. Today, we're going to talk a little bit more about what we did and why we did it that way.

There's more...