If there is any possibility that the respondent may not know the answer to your question, include a "don't know" response category.
8. Does not imply a desired answer. The wording of a question is extremely important. We are striving for objectivity in our surveys and, therefore, must be careful not to lead the respondent into giving the answer we would like to receive. Leading questions are usually easily spotted because they use negative phraseology. As examples:
Wouldn't you like to receive our free brochure? Don't you think the Congress is spending too much money?
9. Does not use emotionally loaded or vaguely defined words. This is one of the areas overlooked by both beginners and experienced researchers. Quantifying adjectives (e.g., most, least, majority) are frequently used in questions. It is important to understand that these adjectives mean different things to different people.
10. Does not use unfamiliar words or abbreviations. Remember who your audience is and write your questionnaire for them. Do not use uncommon words or compound sentences. Write short sentences. Abbreviations are okay if you are absolutely certain that every single respondent will understand their meanings. If there is any doubt at all, do not use the abbreviation. The following question might be okay if all the respondents are accountants, but it would not be a good question for the general public.
What was your AGI last year? ______
11. Is not dependent on responses to previous questions. Branching in written questionnaires should be avoided. While branching can be used as an effective probing technique in telephone and face-to-face interviews, it should not be used in written questionnaires because it sometimes confuses respondents. An example of branching is:
1. Do you currently have a life insurance policy ? (Yes or No) If
no, go to question 3 2. How much is your annual life insurance premium ? _________ These questions could easily be rewritten as one question that applies to everyone:
1. How much did you spend last year for life insurance ? ______ 12. Does not ask the respondent to order or rank a series of more than five items. Questions asking respondents to rank items by importance should be avoided. This becomes increasingly difficult as the number of items increases, and the answers become less reliable. This becomes especially problematic when asking respondents to assign a percentage to a series of items. In order to successfully complete this task, the respondent must mentally continue to re-adjust his answers until they total one hundred percent. Limiting the number of items to five will make it easier for the respondent to answer.
Pre-notification Letters Many researchers have studied pre-notification letters to determine if they increase response rate. A meta-analysis of these studies revealed an aggregate increase in response rate of 7.7 percent. Pre-notification letters might help to establish the legitimacy of a survey, thereby contributing to a respondent's trust. Another possibility is that a pre-notification letter builds expectation and reduces the possibility that a potential respondent might disregard the survey when it arrives.
Pre-letters are seldom used in marketing research surveys. They are an excellent (but expensive) way to increase response. The researcher needs to weigh the additional cost of sending out a pre-letter against the probability of a lower response rate. When sample sizes are small, every response really counts and a pre-letter is highly recommended.
1. Briefly describe why the study is being done and identify the sponsors. This is impressive and lends credibility to the study.
2. Explain why the person receiving the pre-letter was chosen to receive the questionnaire.
3. Justify why the respondent should complete the questionnaire. The justification must be something that will benefit the respondent. For most people, altruism is not sufficient justification. If an incentive will be included with the questionnaire, mention the inclusion of a free gift without specifically telling what it will be.
4. Explain how the results will be used.
Cover Letters The cover letter is an essential part of the survey. To a large degree, the cover letter will affect whether or not the respondent completes the questionnaire. It is important to maintain a friendly tone and keep it as short as possible. The importance of the cover letter should not be underestimated. It provides an opportunity to persuade the respondent to complete the survey. If the questionnaire can be completed in less than five minutes, the response rate can be increased by mentioning this in the cover letter.
Flattering the respondent in the cover letter does not seem to affect response. Altruism or an appeal to the social utility of a study has occasionally been found to increase response, but more often, it is not an effective motivator.
There are no definitive answers whether or not to personalize cover letters (i.e., the respondents name appears on the cover letter). Some researchers have found that personalized cover letters can be detrimental to response when anonymity or confidentiality are important to the respondent.
The literature regarding personalization are mixed. Some researchers have found that personalized cover letters with hand-written signatures helped response rates. Other investigators, however, have reported that personalization has no effect on response.
The signature of the person signing the cover letter has been investigated by several researchers. Ethnic sounding names and the status of the researcher (professor or graduate student) do not affect response. One investigator found that a cover letter signed by the owner of a marina produced better response than one signed by the sales manager. The literature is mixed regarding whether a hand-written signature works better than one that is mimeographed. Two researchers reported that mimeographed signatures worked as well as a hand-written one, while another reported that hand-written signatures produced better response. Another investigator found that cover letters signed with green ink increased response by over 10 percent.
It is commonly believed that a handwritten postscript (P.S.) in the cover letter might increase response. One older study did find an increase in response, however, more recent studies found no significant difference.
1. Describe why the study is being done (briefly) and identify the sponsors.
2. Mention the incentive. (A good incentive is a copy of the results).
3. Mention inclusion of a stamped, self-addressed return envelope.
4. Encourage prompt response without using deadlines.
5. Describe your "confidentiality/anonymity" policy.
6. Give the name and phone number of someone they can call with questions.
Response Rate and Following up on Nonrespondents Response rate is the single most important indicator of how much confidence can be placed in the results of a survey. A low response rate can be devastating to the reliability of a study.
One of the most powerful tool for increasing response is to use follow-ups or reminders. Traditionally, between 10 and 60 percent of those sent questionnaires respond without follow-up reminders. These rates are too low to yield confident results, so the need to follow up on nonrespondents is clear.
Researchers can increase the response from follow-up attempts by including another copy of the questionnaire. When designing the follow-up procedure, it is important for the researcher to keep in mind the unique characteristics of the people in the sample. The most successful follow-ups have been achieved by phone calls.
Many researchers have examined whether postcard follow-ups are effective in increasing response. The vast majority of these studies show that a follow-up postcard slightly increases response rate, and a meta-analysis revealed an aggregate gain of 3.5 percent. The postcard serves as a reminder for subjects who have forgotten to complete the survey.