It's commonly said that there's no such thing as a dumb question, and this is probably true. However, there is certainly such thing as a bad survey question.

A bad survey question is one that contains confusing construction, poorly phrased language, or other elements that leads respondents to answer in ways that are less than helpful. Bad survey questions can skew your dataset or even make the data unusable. Unfortunately, we often become so absorbed in our data-collection process that survey questions with bad construction and wording inevitably sneak in.

Here's a list of three good examples of bad questions, along with some suggestions about how to correct them.

The Double-barreled Question

There's nothing inherently wrong with the double-barreled question. We use them all the time in conversation. For example, your friend or spouse would find no fault if you asked them, "Would you like to go out for dinner tonight, and where would you like to go?" Our brains can easily negotiate a layered question such as this in our everyday interpersonal conversations. However, within the formal constraints of a survey, a respondent is likely to be tripped up and trapped by such an inquiry because it is both a yes-no question and an open-ended one.

To fix a double-barreled survey question, simply break the question into its component pieces. The first question, in this case, would be, "Would you like to go to dinner tonight?" (Answered with yes or no.) The second question, of course, would be, "If you would like to go to dinner, where might you like to go?" (Answered open-endedly or with a list of discrete options to choose from.)

The Loaded Question

The loaded question bears some similarity to the double-barreled question because they can muddle survey results in much the same way. Again, your friend or spouse would hardly notice the presumptiveness of asking, "Where would you like to go to dinner tonight?" This question is "loaded" with the assumption that your friend or spouse wants to go to dinner in the first place. This is not a problem in ordinary conversation, but in a formal survey, it renders the respondent who does not want to eat out unable to meaningfully answer at all, because answering "No" is not a valid response.

There are at least two ways to fix a loaded question. The first would be to unload any assumptions and determine them with separate questions--in much the same way as we fixed our double-barreled question. We would first ask, "Would you like to eat out tonight?" Then we would ask, "If you answered yes to the previous question, where would you like to go? (If you answered no, skip this question)."

Another way to fix a loaded survey question would be to furnish a range of responses that includes an acknowledgment of the assumption, like this: "I do not want to eat out."

The Double-negative Question

In everyday conversation, using double negatives (or even triple negatives) can lead to confusion and misunderstandings. In data collection, it can be positively disastrous because it can result in a response that is the exact opposite of what a survey respondent intends.

Consider this question: "Do you not think it's a good idea to go out to dinner tonight?" Even your easy-going friend or spouse might need to think a while before replying to this one. That's because the answer "Yes" means that they would not care to dine out, while "No" means they would. But even if your friend at first misunderstands, the confusion will be cleared up when one of you gets ready to leave and the other starts making dinner. In a survey question, however, the respondent might misunderstand the question, and respond directly contrary to their own opinion without realizing anything is amiss.

The way to fix a question containing a double negative is to--surprise--remove the double negative. However, the way to avoid double negatives (and other confusion, for that matter) is to simplify the language you use in all of your questions. When writing surveys (and other forms of technical writing), we have a tendency to make our writing more complicated than necessary. Sometimes, while trying to sound professional, precise, or important, we make survey questions too complicated. Or, maybe the subject matter itself really is complicated. In any case, keep it simple. Simplicity will almost always win the day. So, in short, to correct this double-negative question, we revert to simply asking, "Do you want to go out to eat tonight?"

This discussion will continue in part 2 of this three-part series.

‍