How to Get Interpretable Feedback: Two Best Examples

The keys to writing the best questions on your survey or evaluation form are: (1) choose a response scale that matches the question, and (2) carefully consider what you will learn if respondents choose each of the answers on your response scale.

For example, if I want to know if my 9-year old son emptied all the baskets of trash in the house, I would NOT frame the question like this:

Did you empty all the baskets of trash in the house? YES / NO

Learn the 9 Principles of Feedback to retain customers and grow your membership

Each time I administer this question his answer is “yes” because my respondent does not know what the scale means. If he ever said “NO” then I would know the answer. But since he says “YES” then I am not certain whether he emptied one basket or half of the baskets or all of the baskets. Some survey makers might be tempted to turn this question into a multiple choice grouping like this:

How many baskets of trash did you empty?

• 0
• 1 to 4
• 5 to 8
• 9 or more

Two of these choices are uninterpretable. If he responds “0” or “1 to 4” I know the answer is “No” because I know we have 8 baskets in our house. But even if he chooses the “correct” answer of “5 to 8”, I still do not know if he emptied all 8 or only 5 or 6. If he chooses “9 or more” than I am left wondering whether he did indeed complete the job (and also emptied one basket that was not trash) or if he just chose the highest number thinking that was the best answer to pick.

The best way to ask this question is allowing him to fill in a number like this:

How many baskets of trash did you empty? Write one number here: _____

All answers to this question are completely interpretable and I will know the meaning of any number he inserts (i.e., 8 = Completed; Less than 8 = Not Completed; More than 8 = Lying)

Sometimes the best response scale has only 2 points. Many times my clients hesitate to use such a scale, thinking it is always better to offer their respondents more choices. But consider this example:

Did you walk the dog today? YES / NO

This is the most interpretable and best way to ask this question of my 8-year old son. Adding more choices beyond YES and NO will yield confusing results. Like some of my clients, my son also prefers to have more choices, something like this:

Did you walk the dog today? (check all that apply)

• Yes
• No
• I think I did
• It wasn’t my turn
• You didn’t tell me
• I can’t find my shoes
• N/A
• We have a dog?
• Other (write your excuse here): ______________________

As with many survey questions, MORE choices are not always better. In this case, I can assume most of these answers actually mean “No”, but some might mean “Yes”. Others might mean that he did not walk the dog but his sister did. In person, I have the advantage of eventually learning the true answer through further interrogation, but you will not have the opportunity to clarify these ambiguous answers on your survey.

Do not wait until you collect your data to think through these issues or you will be left guessing at the intent of the answer or acting on misleading results.

Takeaways:

1. More choices and longer response scales are not always better – use the response scale that is most interpretable for that particular question
2. Keep your audience in mind when constructing your response scale – will they all interpret the anchors in the same way?
3. Consider each answer that can possibly be given and determine if you know exactly what it means – if any doubts, change your response scale or modify the question/Doc