We've acknowledged that, while there is no such thing as a bad question, there are lots of bad survey questions. In part 1 of this three-part series, we provided examples of the Double-barreled Question, Loaded Question, and Double-negative Questions. We now continue with good examples of bad survey questions by introducing the Leading Question, the Jargon-filled Question, and the Question with Faulty Response Options.

The Leading Question

A leading question contains an inherent bias that "leads" the respondent to a certain answer. In part 1 of this discussion, we were in the middle of a hypothetical conversation about whether to go out to eat. A non-leading question would be something like, "Would you like to go out to dinner this evening?" It's a yes-or-no question no value placed or implied either way. However, if the question were phrased this way: "Wouldn't it be great to go out to dinner this evening?" it becomes a leading question, implying that "yes" is "great," while answering "no" is not.

One way to erase any biased or leading language in survey questions is to simplify them. Stick to the bare inquiry, avoiding extra words such as adjectives and adverbs. Even asking something like, "Would you like to go out to a nice place for dinner this evening?" contains a slight bias, so strip everything down to the bare minimum of words necessary to ask your question.

The Jargon-filled Question

Jargon is language, terminology, and abbreviations that may be familiar to you but are unknown or poorly understood by those you are polling. This one is tricky because jargon can become so familiar to us that we may forget it's jargon at all. If, for example, you were to ask your spouse or friend, "Do you want to go to Rick's tonight?" This assumes that everyone knows that Rick's is a restaurant and not a bar or the home of someone named Rick.

The problem is trickier in survey questions--when you use jargon in your survey questions, your respondent may have no way to ask about the jargon. Abbreviations, acronyms, and technical terms are particularly troublesome--they may be so common in your professional speech, you don't realize that not everyone knows what they mean.

So, screen your survey questions for jargon by running them past someone outside your professional circle. Ask your neighbor, dentist, or relative--anyone not in your field--to read possible examples of jargon, and modify your questions until they are easily and immediately understood: "Do you want to go to Rick's (a downtown BBQ rib joint) tonight for dinner?"

The Question with Faulty Response Options

Sometimes a question that seems to call for a yes-or-no answer is more nuanced. Here again, such subtleties are easily processed in everyday conversation. For example, if you ask your friend if she would like to go out to eat, she will realize that the question is yes-no, but she might answer with an opinion such as "maybe," "possibly," or "depends." Many survey questions are the same--they are not up-down, black-white questions, but more a matter of a preference or a range of opinions. So, instead of a straightforward, "Would you like to go out to eat this evening?" some questions should be more like, "How badly would you like to go out to eat?" And the answer to the question can range from "not at all" to "very much." When writing survey questions, you must be aware of (1) the many ways the respondent might feel about your questions, and (2) what kind of data you are attempting to collect.

Numbered scales are useful when a range of opinions might be expressed, but take care to use the same wording in both the question and possible responses. For example, if you ask about dining out in a survey question, you might put it this way: "How likely would you be to go out to dinner this evening?" Here you have decided that "likelihood" is what you're trying to measure, and so the word "likely" should appear in the responses, like this: "(1) Very likely, (2) Likely, (3) Somewhat likely, (4) Somewhat unlikely, (5) Unlikely, (6) Very unlikely."

Options for neutrality or uncertainty (such as, "I have no opinion either way" or "I never go out to eat") can be added, depending on the survey objectives and your potential respondents.

Levels of agreement or disagreement are also useful response options. Just be sure that the "question" is something that the respondent can clearly agree or disagree with.

In part 3 of this series, we'll discuss more options to make sure you capture the true opinions and feelings of your respondents, so that you are measuring the very most accurate "pulse" that is possible from your feedback-collection efforts.

‍