Putting your survey to the test: final checklist

You’ve set a goal for your survey, finalized your questionnaire, determined how you’re going to collect your responses, but are you ready to send your survey to respondents?

Before you do, make sure your survey is in tip-top shape for public consumption by testing it out. Here’s a checklist with ten things to look for before you hit send:

The most important thing to do before sending out your survey is to preview it. When testing your survey out, keep these things in mind:

Time how long it takes you to finish. Keep in mind that your respondents will perceive the time to finish as much longer than you will. If it seems too long, it is. You might consider cutting out some questions or adding skip logic to streamline the experience.

Preview the survey on a mobile device. Surveys look different and completion times vary depending on whether you are on a computer, tablet, or smartphone. Surveys even look different on Android and iOS smartphones, so you may need to change the format of your questions if you think that a lot of your respondents will be completing the survey on a smartphone.

Get feedback from a friend or colleague who didn’t work on the survey. They can give you a sense of whether it is too long and also catch any mistakes that you may have missed.

An introduction is a great way to let your respondents know what the topic of your survey is and also thank them for their time, so don’t forget to include one. A simple sentence will do. At a loss what for what to write? Just borrow it from your survey goal! Try something like:

“Your feedback on our iPhone app will help us improve it and fix any problems. Thank you for taking the time to complete our survey.”

And if you already have an introduction, make sure it isn’t too long. A long, text-heavy intro can turn off respondents, and they may be less likely complete the survey.

Survey creators often agonize over the content in their survey and are so focused on the questionnaire that they often forget to make the survey relatable to their respondents. So put yourself in the shoes of your respondent and be on the lookout for questions that:

Are you providing all possible answer options to your questions, or should you add a “Not applicable” option? By not offering all possible answer options, your respondents may feel like they are forced to randomly pick an answer option to your question, giving you bad data.

Or are you using scales? If so, make sure you are consistently using the same number of scale points throughout your whole survey. If one of your scales only has four answer options, but the others have five, you may be losing an opportunity to compare that question with others.

Make sure to check for any spelling or grammatical mistakes. An easy way to do this is by reading the survey aloud to yourself!

Give your respondents a mental breather by adding in pages to your survey. Often survey creators have ten or more questions on a page and respondents don’t have time to give their brains a rest, making survey fatigue much more likely. Having too many questions on a page can also make surveys harder to take for respondents on mobile devices.

But there is a fine line here. Each question doesn’t need to be on its own page (unless you are using skip logic), and having too many pages can require a lot of load time.

Check all possible paths of your skip logic! Mistakes in the programming can skip respondents to the wrong page, which will harm your data. We’ve even seen logic errors that send respondents in an infinite loop so there is no way “out” of the survey.

If you haven’t used skip logic, think about whether it makes sense for everyone who responds to your survey to answer every single question. For questions that don’t apply to everyone, add in skip logic.

Randomization is good way to get rid of order bias. Think about whether your questions would benefit from either having the answer options randomized, or if the questions themselves should be randomized.

If you do use randomization, make sure to double check that it is working properly. Sometimes survey creators forget to anchor the last answer option (by not clicking “Don’t randomize last choice”, which means that the “None of the above” option doesn’t show up at the bottom and instead is randomly placed in the list), or think that they randomized something when they actually didn’t.

If you are emailing out a link to your survey or using our email collector, make sure that the content of your email is appropriate. This is often the first time that you interact with your survey respondent, so make a good impression by writing a short email that has the following information: a sentence about the topic of the survey, when you’d like them to complete the survey by, and of course, a link to the survey (or information on how they can complete it if you aren’t sending a link).

You can also think about using your introduction sentence in the body of the email instead of in your survey. Also, try sending out a few test emails to make sure that the content of your email or the subject line doesn’t get caught in anyone’s SPAM filters.

You did it, you’ve got a great survey. Now congratulate yourself, send it out to your respondents, and watch their answers roll in!

So before you send out your next survey, make sure you’ve gone through this checklist and completed all the ten items on this list! Once you have and made your final edits, you’re ready to start collecting data!

This article is part of SurveyMonkey’s Surveys 101 project. We hope to help more people create smart surveys. Learn more about the project and our involvement in the research community.