D.7 Step 7: Questionnaire design

As stated before, the way you design your questionnaire is closely related to the method you choose for data collection. However, independently of the collection method applied, any questionnaire for market data collection needs to be standardized. This means that every person participating in your survey will get exactly the same questions and will have identical answering options to choose from. You need to use the same questions and response categories to be able to compare results over time, between different groups of respondents or between different countries. You can only analyse your results statistically if your data from different sources are comparable (e.g. by being of the same type).

Usually, you will design your questionnaire based on already existing questionnaires which you have found during secondary research. Still, while the adoption of single questions is often to be recommended, you should not adopt whole questionnaires. Keep in mind which questions are really relevant for your survey aims. When you set up your questionnaire, you should already have your data analysis in mind. Many statistical analyses (e.g. regression analysis) can only be carried out if your response categories are based on ‘ratio’ scales (e.g. when you ask for the exact number of goats on a farm, so an answer could be ‘40’ or ‘67’) instead of ordinal scales (with categories as answering options, e.g. number of goats:  < 20, 20-39, 40-59, ≥ 60) or nominal scales (farm with goats: yes/no).

You should always create a questionnaire in such a way that the members of your population of interest can fill it in without needing any extra information or clarification. While your questionnaire needs to be self-explanatory, it is necessary to add a cover letter to all surveys for market data collection. The cover letter explains the background and reasons for this survey and includes a statement of confidentiality. This assures participants that their information will not be passed on to third parties and that the results will be anonymised in any publication. Before distributing the questionnaire to your population of interest, you should always conduct at least one pretest with a small number of people who are not part of the sample. This way, you will be able to detect problems (e.g. through unclear questions) and can solve them before starting collecting data on a larger scale. If the first pretest reveals substantial problems, a second pretest is advisable. 

In some cases, you ask potential participants a filter question before you begin with your ‘real’ questionnaire.  Filter questions ensure that only those people who are able to give relevant answers will fill in the questionnaire (or certain questions in the questionnaire).

• Example
  Your population of interest are regular consumers of organic products. You are approaching people in front of a supermarket. Because you want to exclude those people from your survey who only occasionally buy organic products, you start with a filter question, e.g.  "Do you buy organic products at least once a week?" with the answering options ‘Yes’ and ‘No’. The person is only asked to participate in your survey and answer the questionnaire if the answer is ‘Yes’.

Your questionnaire should start with those kinds of questions that are easy to answer and make it easier for respondents to get into the survey. The main part of the questionnaire should then follow a logic which is easy to follow for the respondent, e.g. by structuring it according to topics. To reduce participants’ unnecessary input, you can also include filter questions in the questionnaire itself.

• Example
  Your total population of interest all of the organic dairy farmers in your country. Now, imagine you had a national support program for dairy farmers located in mountainous areas called “Organic Mountain Dairy”. In your questionnaire you could put a question like: “Do you participate in the “Organic Mountain Dairy” program?” with the answering options ‘Yes’ and ‘No’. Only those participants who answer with ‘Yes’ will then be asked to fill in some extra questions, while the other ones will be asked to skip these and continue with questions later on.

You need to phrase all questions very clearly in order to avoid misinterpretations (which will lead to nonresponse or measurement errors). In the complete questionnaire, you should avoid questions which include more than one aspect (double-choice questions).

• Example
  You want to find out how satisfied organic farmers are with the development of the domestic organic market in the last year. A double-choice question would be: “Have you been satisfied with the organic market development in 2013 and think that it will continue like this?” People might give a different answer to the first part of the question than to the second part. Thus, you should create two individual questions instead.

Only at the end of your questionnaire should you ask sensitive questions and ask people to give their sociodemographic data (e.g. year of birth, gender, household size, income). In many cases, people perceive questions on their income as sensitive. As such questions might irritate or even put off participants, asking them earlier is likely to result in additional nonresponse. Also, using bands or categories (e.g. age bands as 18-29 years, 30-44 years etc.) rather than the actual value might help overcome this problem.

The checklist contains the basic considerations for your questionnaire design.