September 5, 2006

Five Pitfalls of Survey Writing

For most communicators, survey writing is a skill that doesn't come naturally. While we recognize that surveys are an essential tool to understand the needs of employees and make our case to senior management, writing them is not the straightforward exercise we crave.

Take advantage of where we've seen others fall and sharpen your survey writing skills. Here are five pitfalls that we've witnessed over the years:

1. The double whammy. Two questions combined into one create confusion. For example, if you ask employees about their level of agreement with the statement: “E-mails from HR and Employee Communication are timely,” you're really asking two questions. While you may be interested in e-mails generally, employees may have different experiences with the two distinct sources. Here's another one, “Do you receive and regularly access the following vehicles?” An employee may receive a newsletter, but never open it.
   
   
2. Fuzzy logic. Ill-defined or vague questions are an easy trap. You may understand what you mean by a question like, “Do you believe you have enough information to support our strategies,” but will your respondent? What exactly is “support?” Be as concise as possible without rambling. Test your questions with an employee by asking him/her to explain the intent.
   
   
3. One potato, two potato. Asking the same question twice, but in a slightly different way doesn't provide any value; it only increases the length of your survey. The difference between these two examples is not clear (not to mention that “communications” is vague):
   
   • “The communications I receive help me do my job.”
     
   • “The communications I receive are relevant to my job.”
     
     
4. Lead a horse to water. Or an employee to the answer. Leading questions are a common problem that will destroy the integrity of your data. How could an employee disagree with these two examples?
   
   • “I am actively involved in setting my goals and objectives.”
     
   • “I never have enough time to take advantage of career development tools and courses.”
     
     
5. Open-ended gridlock. Too many open-ended questions will contribute to a condition called “Survey Shutdown”–a respondent leaving the survey before he/she is finished. Every survey should be limited to one open-ended question. Based on all of the comments we've read from hundreds of surveys, responses will be repeated if there are more than two open-ended questions. If you feel the need to include many open-ended questions, conducting a focus group is the better solution. Your research will be richer for it.