How to create great surveys that
minimise dropout rates!
28 Jan 2021
Michaela Faber
Though questionnaires are among the day-to-day tools of market researchers, marketers, PR specialists, brand and product managers it is quite an art to design a market survey questionnaire that guarantees the results you need. After all, what you want are valid and standardized answers that help you drive your business forward.
I often participate in different online market surveys to see what is out there and I am shocked as to how many questionnaires fail to hold my attention long enough, to actually follow through to the end. Why? Because they are either lacking explanation, they are way too long or the questions don’t make sense to me.
So how to create a great questionnaire for your market survey that holds consumer attention and minimises the dropout rate?
There are some rules regarding length, structure of the questionnaire and the questions themselves that you need to bare in mind when creating your questionnaire:
I often participate in different online market surveys to see what is out there and I am shocked as to how many questionnaires fail to hold my attention long enough, to actually follow through to the end. Why? Because they are either lacking explanation, they are way too long or the questions don’t make sense to me.
So how to create a great questionnaire for your market survey that holds consumer attention and minimises the dropout rate?
There are some rules regarding length, structure of the questionnaire and the questions themselves that you need to bare in mind when creating your questionnaire:
1. Length of the questionnaire:
Keep it as short as possible!
We recommend keeping it at about 10 questions per survey. People lose focus quite easily and drop out when the questionnaire is too time consuming. This is the reason why we designed our standard questionnaires with 10 questions so the average length of the interview is less than 3 minutes. This ensures high quality answers and almost zero dropout rate.
We recommend keeping it at about 10 questions per survey. People lose focus quite easily and drop out when the questionnaire is too time consuming. This is the reason why we designed our standard questionnaires with 10 questions so the average length of the interview is less than 3 minutes. This ensures high quality answers and almost zero dropout rate.
2. Design of the questionnaire:
Start with general questions and then go into more detail as you move along.
Use the type of answer that will fit the question, your respondent and the results.
For example if you want to run a study about a sensitive subject such as income, you will usually find it is responsible for a high dropout rate when dealt with incorrectly. So when you ask about income you may want to start with such questions as: How satisfied are you with your annual income? Then move on to the more detailed question: What is your average monthly income?
When you ask more detailed questions about income it is usually best to give respondents scaled answers with ranges from — to, i. e What is your average monthly income '10.000 -20.000' '21.000−30.000' etc.
Use the type of answer that will fit the question, your respondent and the results.
For example if you want to run a study about a sensitive subject such as income, you will usually find it is responsible for a high dropout rate when dealt with incorrectly. So when you ask about income you may want to start with such questions as: How satisfied are you with your annual income? Then move on to the more detailed question: What is your average monthly income?
When you ask more detailed questions about income it is usually best to give respondents scaled answers with ranges from — to, i. e What is your average monthly income '10.000 -20.000' '21.000−30.000' etc.
3. Design of the questions:
Designing questions is not as easy as one may think. There are various things to consider. For that we have put together some easy to follow tips:
1. Have clear instructions in regards to what you want the respondent to answer. As a first step, write down what you want to achieve: i. e: I want the respondent to compare A, B and C in the ad visual and tell me which option they would choose based on the colours of the label. Then articulate the question you want the respondent to answer i. e: What colour do you like best on the label, when comparing these three samples. Please choose the one you prefer.
2. Be single minded when articulating the questions. Use the above example: if you ask your respondent which colour they like best don’t also ask them why in the same question. For example, do not ask: What colour on the label do you like best, when comparing the three samples and why do you like this colour best? These should be two separate questions.
3. Design the answers so they really reflect the question. You think this is easy? Well, you’d be surprised how often we find that this is not the case i. e if you ask: What type of food do you feed your dog? The answer options should not be: Pedigree, Royal Canine, Purina hence brands. Instead the answers should be types of food i. e: canned food, raw meats and bones, dry food.
4. Use easy to understand, day to day language. Ask a colleague from a different department or a friend to read the questionnaire and confirm that all questions are clear.
5. Keep the questions as short as possible. Read the questions and strike through all filler words. Then check if it still makes sense. Keep doing this until your question is short and punchy.
6. Use a point in time to make the question less vague: i.e. Which brands of shampoo have you used? can be pretty vague for respondents as the time frame is not clear. Which brands of shampoo have you used within the last 4 months?, is longer but more precise so you will get more reliable and meaningful answers.
7. In case you want to use hypothetical questions, make sure they are very clear and lead to clear answers. If you design them too loosely you will get vague answers. A bad example would be: Imagine you were looking for a new shampoo? What would you be looking for? A good example would be: Imagine you are looking for a new shampoo, which one of the brands presented would most likely catch your attention?
8. Remember the difference between open questions and closed questions.
Open questions are about 'why'. They help you get some deeper reflections in the words of your respondents. Though they may not replace a qualitative study, they do allow you some great, deeper insights. Example: In your own words, tell us, what do you think about the situation with coronavirus.
Closed questions are most commonly used in market surveys to quantify data. If you need a straight yes/no like/dislike etc. answer, use closed questions. Example: Are you afraid of coronavirus? Yes/ no/not sure
Open questions are about 'why'. They help you get some deeper reflections in the words of your respondents. Though they may not replace a qualitative study, they do allow you some great, deeper insights. Example: In your own words, tell us, what do you think about the situation with coronavirus.
Closed questions are most commonly used in market surveys to quantify data. If you need a straight yes/no like/dislike etc. answer, use closed questions. Example: Are you afraid of coronavirus? Yes/ no/not sure
9. When using multiple choice questions, be sure to find the right answer for everyone. Brainstorm this thoroughly. If you are not sure that the list is complete, add "other" at the end as the last possible answer.
10. Always use neutral questions that don’t create bias. Check that the wording of the questions does not impose any point of view on the respondent i.e. Many are already donating to charity, have you donated in the last four weeks?
You may want to keep in mind that bias can be created by:
Another way of asking delicate questions is to use agree or disagree statements in the question i.e: I don't mind how animals are treated as long as I can afford to buy meat: Then use 1-5: totally agree, agree, neither agree nor disagree, disagree, totally disagree.
- Having a subliminal message in your question: i.e. How much do you like vegetarian meat alternatives? This type question contains the subliminal message, that the respondent is supposed to like vegetarian meat alternatives. Instead ask: What do you think of vegetarian meat alternatives?
- Answers that are skewed towards one direction. If you go from 'like very much' to 'like just a little bit', leaving out neutral or disapproving answers, your questionnaire won't work. So make sure to use the Likert scale which contains all options from 'like very much' via 'neither like nor dislike' to 'dislike very much'.
- Feelings of guilt or other negative emotions. People tend to demonstrate socially approved behaviour rather than telling the truth if there is a negative emotion or social judgement attached to it.
Another way of asking delicate questions is to use agree or disagree statements in the question i.e: I don't mind how animals are treated as long as I can afford to buy meat: Then use 1-5: totally agree, agree, neither agree nor disagree, disagree, totally disagree.
11. Asking people about experiences that were way in the past will likely falsify the results of your research. We know that the human brain fabricates stories around actual events that are stored in long term memory. So make sure you ask them within a timeline that is realistic i. e if you want to find out how many face masks people buy in supermarkets you are best off to ask them how many they bought in the past four weeks. If you ask them how many they bought in the past 12 month, they will most likely give falsified answers.
12. When you use scaled answers ensure that the direction of the scale is consistent throughout the entire questionnaire e. g: 1= very bad, 5= very good. 1 should not be very good in a different question. This is one mistake I often find in market surveys.
13. It is also very useful to use evaluation questions. They allow you easy insights and feedback as to how consumers perceive and like/dislike your product. Some examples would be: How clear is…, How likely is it that…, How probable would it be that you…. You best use evaluation questions with a five point scale: For example:
How clear is this idea?
- Completely clear.
- Rather clear.
- In some ways it is clear, in others it is not.
- Rather not clear.
- Completely not clear.
The great benefit of working with automated online market survey platforms like Fastuna is that a lot of the above mentioned is already taken care of from our end. All you have to do is define your goals and think of questions that help you to get the answers you need.
Fastuna is an automated online plattform, be assured that you are still supported by humans or as my dear colleague Maria always puts it: Fastuna is an online market survey platform with a human interface.
To learn more about Fastuna or find out how we can support you, please feel free to contact myself: michaela@fastuna.com or my colleague maria@fastuna.com.
Fastuna is an automated online plattform, be assured that you are still supported by humans or as my dear colleague Maria always puts it: Fastuna is an online market survey platform with a human interface.
To learn more about Fastuna or find out how we can support you, please feel free to contact myself: michaela@fastuna.com or my colleague maria@fastuna.com.
“
If I compare Fastuna with a vehicle, it would be a Swiss train. You don’t need to worry about anything, you just upload your stimuli and then sit back and relax. You will get to your destination promptly, within just a couple of hours, getting all the data you need to make a fast decision on a concept, a product or a packaging.
“
Fastuna is a great partner to work with. The platform is very intuitive and easy to use, and the results are available in just a couple of hours. Great support from the team. I look forward to seeing how they develop even further in a truly agile way!
Innovate with confidence
Validate product and marketing decisions with real people in 24h using our hassle-free testing tool