How to design Questionnaires .steps advantages disadvantages

Steps of designing a questionnaire courtesy my teacher assoc professor Dr rubina gul

1. Questionnaire By Ms. Safia Murad

2. Content Introduction Definition of questionnaire Importance of questionnaire Characteristics of good questionnaire Advantages and disadvantages of questionnaire Type of questionnaire Steps preceding questionnaire design Conclusion

3. Introduction Research is a quest for knowledge through diligent search or investigation or experimentation aimed at the discovery & interpretation of new knowledge Research problem comprises of various steps each being mutually exclusive

4. Introduction Steps include: The research design Problem definition Hypothesis generation Decision on the types of the study appropriate to the problem Decision on the method of data gathering Development of an analysis plan Data collection Performance of analysis Drawing conclusions and recommendations

5. The method of data collection is one of the important steps in the research process. Questionnaire is one such tool used to collect information which can cover almost all practical issues Questionnaires are extensively used in surveys and forms the backbone of the survey procedure

6. Questionnaire History The questionnaire was invented by Sir Francis Galton, a British anthropologist, explorer and statistician, invented questionnaires in the late 1800s

7. Definition A questionnaire is a research instrument consisting of a series of questions and other prompts for the purpose of gathering information from respondents.” Questionnaire is a measuring device used to query a population/sample in order to obtain information for analysis. A questionnaire is simply a list of mimeographed or printed questions that is completed by or for a respondent. (Health Research Methodology 2nd ed. 2001 WHO)

8. When to use a questionnaire When resources and money are limited When it is necessary to protect the privacy of the participants When corroborating other findings For validating the research document and data

9. Importance of questionnaire A questionnaire is the main means of collecting primary data A questionnaire enables research data to be collected in a standardized way so that data are internally consistent and coherent for analysis. A questionnaire ensures standardization and comparability of the data across interviewers , increases speed and accuracy of recording , and facilitates data processing

10. Characteristics of a questionnaire Elicits information from respondents Results can be tabulated Standardized across respondents Understandable to respondents Clear and concise

11. A good questionnaire must: Obtain the most complete and accurate information possible. Be organized and worded to encourage respondents to provide accurate, unbiased and complete information. Make it easy for respondents to give the necessary information and for the interviewer to record the answer, and it should be arranged so that sound analysis and interpretation are possible. keep the interview brief and to the point and be so arranged that the respondents remain interested throughout the interview

12. Advantages of Questionnaire Low cost even when the universe is large and is widely spread geographically. It is free from the bias of the interviewer; answers are in respondents own words. Respondents have adequate time to give well thought out answers Respondents who are not easily approachable can also be reached conveniently. Large samples can be made use of and thus the results can be made more dependable and reliable.

13. Easy to analyze Maintains respondents anonymity where required Scalability Free from time constraints Ease of replication

14. Disadvantages of Questionnaire Low rate of return of the duly filled in questionnaires It can be used only when respondents are educated and co- operating. The control over the questionnaire may be lost once it is sent It is difficult to know whether willing respondents are truly representative There is also the possibility of ambiguous replies or omission of replies altogether to certain questions This method is likely to be the slowest of all Respondents may misinterpret a question, thereby limiting the validity of the results

15. Dishonest answers Low validity Can generate bias to avoid legal issue Might be difficult to interpret No representation of respondent’s feeling and emotions

16. Brainstorming How can a researcher overcome the disadvantages generally related to research questionnaire

17. Answer Pilot testing the questionnaire with individuals who are similar to the sample population who have been chosen to participate in the study can minimize the disadvantage. These individuals can alert the investigator to any unclear items. After reviewing the results from the pilot study the researcher can revise the questionnaire accordingly before distribution to the population sample.

18. Types of questionnaire The general form of questionnaire can be three types. Structured Semi-structured Unstructured

19. Structured Questionnaire Structured questionnaires are those questionnaires in which there are definite, concrete and pre-determined questions. The questions are presented with exactly the same wording and in the same order to all respondents. Resort is taken to this sort of standardization to ensure that all respondents reply to the same set of questions Structured questionnaires have fixed alternative questions in which responses of the informants are limited to the stated alternatives

20. A highly structured questionnaire is one in which all questions and answers are specified and comments in the respondents own words are held to minimum. Structured questionnaires are simple to administer and relatively inexpensive to analyze. They are used in large interview programs (anything over 30 interviews and more likely over 200 interviews in number) and may be carried out over the tele- phone, face-to-face or self completion depending on the respondent type, the content of questionnaire and the budget.

21. Semi-structured questionnaires Semi-structured questionnaires comprise a mixture of closed and open questions. They are commonly used in research where there is a need to accommodate a large range of different responses from companies. The use of semi-structured questionnaires enables a mix of qualitative and quantitative information to be gathered. They can be administered over the telephone or face-to-face.

22. Unstructured Questionnaires Unstructured questionnaires are made up of questions that elicit free responses. These are guided conversations rather than structured interviews and would often be referred to as a “topic guide”. The topic guide is made up of a list of questions with an apparent order but is not so rigid that the interviewer has to slavishly follow it in every detail.

23. cont There are no hard-and-fast rules about how to design a questionnaire, but there are a number of points that can be borne in mind: A well-designed questionnaire should meet the research objectives. It should obtain the most complete and accurate information possible . A well-designed questionnaire should make it easy for respondents to give the necessary information and for the interviewer to record the answer , and it should be arranged so that sound analysis and interpretation are possible . It would keep the interview brief and to the point and be so arranged that the respondent(s) remain interested throughout the interview.

24. 1. Define your objectives and target audience. What information do you want to collect? Who will be responding to the questionnaire? 2. Develop your questionnaire. Write clear, concise, and unambiguous questions. Use a variety of question types (e.g., multiple choice, Likert scale, open-ended). Pilot test your questionnaire with a small group of people to identify any problems. 3. Evaluate content validity. Does your questionnaire measure what it is intended to measure? Involve subject matter experts in reviewing your questionnaire. 4. Evaluate face validity. Does your questionnaire appear to be valid on the surface? Ask potential respondents if the questions are clear and easy to understand.

25. 5. Evaluate construct validity. Does your questionnaire measure the underlying constructs you are interested in? Use statistical tests to assess the relationships between different parts of your questionnaire. 6. Evaluate criterion validity. Does your questionnaire correlate with other measures of the same construct? Compare your questionnaire results to other established measures. 7. Evaluate reliability. Is your questionnaire consistent over time and across different administrations? Use statistical tests to assess the internal consistency and test-retest reliability of your questionnaire. 8. Refine your questionnaire. Make changes based on your evaluation results. Pilot test your revised questionnaire again. 9. Document your validation process. Describe the steps you took to validate your questionnaire. Report your findings in your research report.

26. Cronbach's alpha coefficient alpha (α), is a statistic used to assess the internal consistency or reliability evaluate whether the questions consistently measure student engagement. A high alpha value (typically above 0.7) suggests that the items are measuring the same thing, while a low alpha value indicates that they might be measuring different aspects or that some items are not well-aligned with the intended construct

27. Remember Cronbach's alpha: It is not a measure of validity. While alpha tells if the items are internally consistent, it doesn't guarantee that they are measuring what is intend to measure. Its interpretation depends on the number of items. Generally, higher alpha values are desired, but the acceptable threshold can vary depending on the number of items in the test. There are limitations to its use. Alpha assumes that the items are measuring the same construct on an equal interval scale. It is not suitable for tests with items that have different response formats or measure different aspects of a construct

28. THE STEPS PRECEDING QUESTIONNAIRE DESIGN There are nine steps involved in the development of a questionnaire: Decide the information required. Define the target respondents. Choose the method(s) of reaching your target respondents. Decide on question content. Develop the question wording. Put questions into a meaningful order and format. Check the length of the questionnaire. Pre-test the questionnaire. Develop the final survey form.

29. Decide the information required It should be noted that one does not start by writing questions. The first step is to decide 'what are the things one needs to know from the respondent in order to meet the survey's objectives’ It should appear in the research brief and the research proposal.

30. Define the target respondents Demographic characteristics Job or social category Other relevant characteristics Secondary audiences

31. Demographic characteristics , such as age, sex, religion, urban/rural residence, income level, social class, education, employment status, and ethnic or language group Job or social category , such as policymakers, doctors, nurses, factory workers, religious leaders or university students

32. Other relevant characteristics: Some individuals or groups may be disproportionately affected by TB, such as persons living with HIV/AIDS, imprisoned people, homeless populations, drug users, or family members of people with TB; Secondary audiences: Your secondary audiences may include allies who can influence or provide access to your primary audience, such as community leaders or health authorities. Knowledge, attitudes and health-seeking practices may vary substantially among population groups, and according to social, cultural or economic characteristics.

33. Choose the method(s) of reaching your target respondents Face-to-Face Interview Téléphone Interviews Mail Questionnaires Internet Questionnaire

34. Face-to-Face Interview Face-to-face interviews or personal interviews are surveys conducted in person by an interviewer who usually travels to the person being surveyed. Pros —High response rates; can clarify questions, if necessary; control over respondent selection; can use longer, more complex questionnaire; and easier to motivate the respondent. Cons —High costs, time-consuming, and more administrative requirements (i.e., selecting and training interviewers, contacting respondents, travel arrangements). Also, there is a tendency for respondents to give socially acceptable answers.

35. Telephone Interviews Telephone interviews are usually conducted from a central office that places telephone calls to selected households or businesses. Pros —Good response rates, fast, some anonymity for respondents in answering questions, and control over respondent selection. If a comprehensive list of the target population is available, the likelihood of obtaining a representative sample is high. Cons —Questions must be short and not complex; cannot control interruption by others in household/ office; hard to find persons at home, and those that are at home may resent intrusion; there is mounting displeasure among households receiving unsolicited telephone calls; requires training and quality control monitoring of the interviewers; and is usually difficult to target a specific geographical location.

36. Mail questionnaires Mail questionnaires are written surveys that are sent through the mail to selected members of the population to be surveyed. Pros —Good response rates with rigorous follow-up procedures, relatively easy to obtain a listed population and locate respondents, can avoid interviewer bias and distortion, answers unlikely to be socially influenced, easy to administer and relatively low costs, can cover a wide geographical area, and more manageable for handling large samples. Cons —Questionnaire may be given to someone else to fill out or may not reach the desired respondent; most difficult type of questionnaire to design; hard to interpret open-ended questions; cannot control sequence in which respondents answer questions; and time consuming, given periodic mail- out requirements.

37. Internet questionnaire Internet questionnaire is a form of a written survey. Respondents may be invited to participate in the survey through email or because they visit a particular web page. Pros —Fast to conduct and tabulate, some software products allow questionnaires to be customized depending on the respondent’s answers, avoids interviewer bias and distortion, answers unlikely to be socially influenced, easy to administer, and relatively low costs. Cons —Information transferred via the Internet may not be confidential; poor control over respondent selection; follow-up difficult to conduct; difficult to obtain probability sample; and, like mail surveys, this is the most difficult type of questionnaire to design.

38. Decide on question content There are a series of questions that should be posed as the researchers develop the survey questions themselves: "Is this question sufficient to generate the required information“ "Can the respondent answer the question correctly“ "Are there any external events that might bias response to the question“ "Do the words have the same meaning to all respondents" For example, "How many members are there in your family?" There is room for ambiguity in such a question since it is open to interpretation as to whether one is speaking of the immediate or extended family.

39. "Are any of the words or phrases loaded or leading in any way" "Are there any implied alternatives within the question" "Will the question be understood by the type of individual to be interviewed“ "Is there any ambiguity in my questions“ "Are any words or phrases vague“ "Are any questions too personal or of a potentially embarrassing nature" "Do questions rely on feats of memory"

40. Develop the question wording Contingency questions/cascade format Matrix questions Closed ended questions Dichotomous, where the respondent has two options Nominal-polytomous, where the respondent has more than two unordered options Ordinal-polytomous, where the respondent has more than two ordered options (Bounded)Continuous Open ended questions

41. Contingency questions/cascade format several questions have been used to scale a response to the unasked but real understanding question. A question that is answered only if the respondent gives a particular response to a previous question. This avoids asking questions of people that do not apply to them (for example, asking men if they have ever been pregnant) Eg. 1. Do you use tobacco (a) Yes (Go to 1a Question) (b) No (Go 2 Question) 1 (a) How many packs of tobacco do you use? (a) ≥ 10 packs (b) 10 - 5 packs (c) ≤ 5 packs per day 2. Do you use alcohol (a) Yes (b) No.

42. Matrix questions Identical response categories are assigned to multiple questions. The questions are placed one under the other, forming a matrix with response categories along the top and a list of questions down the side. This is an efficient use of page space and respondents’ time.

43. CLOSED ENDED QUESTIONS Respondents’ answers are limited to a fixed set of responses. Four types of response scales for closed-ended questions are distinguished: Dichotomous , where the respondent has two options Nominal-polytomous , where the respondent has more than two unordered options Ordinal-polytomous, where the respondent has more than two ordered options (Bounded)Continuous , where the respondent is presented with a continuous scale. Examples of types of scales include the Likert scale , semantic differential scale , and rank-order scale

44. Open ended questions Completely unstructured - For example, “What is your opinion of questionnaires?” Word association - Words are presented and the respondent mentions the first word that comes to mind. Sentence completion - Respondents complete an incomplete sentence. For example, “The most important consideration in my decision to buy a new house is . . .” Story completion - Respondents complete an incomplete story. Picture completion - Respondents fill in an empty conversation balloon.  Thematic apperception test - Respondents explain a picture or make up a story about what they think is happening in the picture

45. Advantages of Closed ended questions It provides the respondent with an easy method of indicating his answer - he does not have to think about how to articulate his answer. It 'prompts' the respondent so that the respondent has to rely less on memory in answering a question. Responses can be easily classified, making analysis very straightforward. It permits the respondent to specify the answer categories most suitable for their purposes.

46. Disadvantages of Closed ended questions They do not allow the respondent the opportunity to give a different response to those suggested. They 'suggest' answers that respondents may not have considered before. They don’t represent respondent’s feelings and emotions They restrict the respondents within given parameters

47. Advantages of Open ended questions They allow the respondent to answer in his own words, with no influence by any specific alternatives suggested by the interviewer They often reveal the issues which are most important to the respondent, and this may reveal findings which were not originally anticipated when the survey was initiated. Respondents can 'qualify' their answers or emphasize the strength of their opinions.

48. Disadvantages of Open ended questions Respondents may find it difficult to 'articulate' their responses i.e. to properly and fully explain their attitudes or motivations. Respondents may not give a full answer simply because they may forget to mention important points. Some respondents need prompting or reminding of the types of answer they could give. Data collected is in the form of exact comments - it has to be coded and reduced to manageable categories. This can be time consuming for analysis and there are numerous opportunities for error in recording and interpreting the answers given on the part of interviewers. Respondents will tend to answer open questions in different 'dimensions '. For example, the question: "When did you purchase your tooth brush?", could elicit one of several responses, viz: "A short while ago" "Last month" "When I lost my last tooth brush". "When I bought the monthly groceries

49. PUTTING QUESTIONS INTO A MEANINGFUL ORDER AND FORMAT According to the three stages theory (also called the sandwich theory), initial questions should be screening and rapport questions. Then in the second stage you ask all the research specific questions. In the last stage you ask demographic questions. Opening questions Question flow Question variety Closing questions

50. Opening questions Opening questions should be easy to answer and not in any way threatening to THE respondents. The first question is crucial because it is the respondent's first exposure to the interview and sets the tone for the nature of the task to be performed. If they find the first question difficult to understand, or beyond their knowledge and experience, or embarrassing in some way, they are likely to break off immediately. If, on the other hand, they find the opening question easy and pleasant to answer, they are encouraged to continue

51. Question flow Questions should flow in some kind of psychological order, so that one leads easily and naturally to the next . Questions on one subject, or one particular aspect of a subject, should be grouped together. Questions should flow logically from one to the next. The researcher must ensure that the answer to a question is not influenced by previous questions. Questions should flow from the more general to the more specific. Questions should flow from the least sensitive to the most sensitive. Questions should flow from factual and behavioral questions to attitudinal and opinion questions. Questions should flow from unaided to aided questions.

52. Question variety Respondents become bored quickly and restless when asked similar questions for half an hour or so. It usually improves response, therefore, to vary the respondent's task from time to time. An open- ended question here and there (even if it is not analyzed) may provide much-needed relief from a long series of questions in which respondents have been forced to limit their replies to pre-coded categories. Questions involving showing cards/pictures to respondents can help vary the pace and increase interest.

53. Closing questions It is natural for a respondent to become increasingly indifferent to the questionnaire as it nears the end. Because of impatience or fatigue, he may give careless answers to the later questions. Those questions, therefore, that are of special importance should, if possible, be included in the earlier part of the questionnaire. Potentially sensitive questions should be left to the end, to avoid respondents cutting off the interview before important information is collected.

54. PRESENTATION AND LAYOUT OF THE INTERVIEW FORM Use of booklets Simple, clear formats Creative use of space and typeface Use of color coding Interviewer instructions

55. PILOTING / PRE-TESTING THE QUESTIONNAIRES whether the questions as they are worded will achieve the desired results whether the questions have been placed in the best order whether the questions are understood by all classes of respondents whether additional or specifying questions are needed or whether some questions should be eliminated whether the instructions to interviewers are adequate.