Art of Questioning and Construction of MCT

art of questioning guide to multiple choice question

1. ART OF QUESTIONING and CONSTRUCTION OF MULTIPLE CHOICE TEST Prepared by: MARY SUL A. MAGALLANES Dumalinao District

2. Learning Objectives 2 At the end of the session, participants are expected to: acquire questioning techniques to facilitate learning and meaning-making; discuss the value of questioning and discussion strategies in the literacy classroom; realize the importance of enhancing skills in questioning and asking questions in an English classroom.

3. Priming: The Best Question Ever 6 Take a good look at the photo If you could ask a question, what would it be? The goal is to learn as much as you possibly can about who this c haracter really is. Your question should not be too broad, nor too limiting .

4. Write at two (2) questions for each stage of the B-D-A instructional framework. Fill out the chart. 10 Literacy Instructional Framework Questions Before During After En g age

5. What’s in a question?

6. Questions Allow us to make sense of the world. They are the most powerful tools we have for making decisions and solving problems, for inventing, changing and improving our lives as well as the lives of others. Jamie McKenzie What’s in a question?

7. We ask questions to facilitate the comprehension of both the content and language of the lesson/subject matter. COMPREHENSION PRODUCTION

8. We are using questions to make connections: text to text, text to self, and text to world We are using questions to make predictions. We are using questions to make sure we understand what we have read Questions strengthen reasoning abilities of children as well as help them clarify/define their initial response to the text. 14

9. Questioning helps us to become better readers by making us THINK! “Books not discussed lose their value.” (Mortime Adler) Questions allow us to make sense of the world. They are the most powerful tools we have for making decisions and solving problems, for inventing, changing and improving our lives as well as the lives of others. (Jamie McKenzie) 15

10. I n f e r r i n g I’m questioning as I read to help me draw conclusions, making predictions, and reflecting on my reading. When the author doesn’t answer my questions, I must infer. Thinking Stems: Mayb e … P er h a p s… I think… I’m guessing… It means…

11. Questioning to Understand I’m asking questions and looking for answers *Before Reading *During Reading *After Reading Thinking Stems: I wonder… What if… W h y… I think I was confused when…

12. Determining the Importance I understand the main idea of the text and the author’s message. Thinking Stems: The text is mainly about… I learned… The important details are… I want to remember…

13. Visualizing I create pictures in my mind as I read. I see what I read. I feel what I read. I use my senses to help me make a movie in my mind. Thinking Stems: I’m visualizing… I’m picturing… I can imagine… I’m seeing…

14. S y n t h e s i z i n g I combine what I know with new information. I read to help me understand the text. I change my thinking along the way. Thinking Stems: Now I get it… At first I thought… But now I think… My new thinking is… I think the lesson or theme is…

15. Making Connections Text to self Text to Text Text to World I use what I know to understand what I’m reading. Thinking Stems: This reminds me of… because… This reminds me of the book… This reminds me of what I’ve heard…

16. What questions should we ask? 22

17. GPU: Gradual Psychological Unfolding 23 GPU is a discussion technique wherein the story is unravelled through a slow yet logical manner with questions and activities prepared by the teacher.

18. FIVE LEVELS/DIMENSIONS OF READING COMPREHENSION (based on Gray, Gates, Smith and Barett Models) Level 5 – CREATIVE READING Level 4 – INTEGRATION (Application to Self/Life) Level 3- EVALUATION (Critical Reading) Level 2- INTERPRETATION (Inferential) Level 1- LITERAL COMPREHENSION 24

19. 25

20. Dimensions of Reading Bloom’s Taxonomy Level 5 – CREATIVE READING Level 4 – INTEGRATION (Application to Self/Life) Level 3- EVALUATION (Critical Reading) Level 2- INTERPRETATION (Inferential) Level 1- LITERAL COMPREHENSION 26

21. 1. First Dimension: Knowledge or Literal Understanding A literal understanding of a selection provides the reader with the most basic structure through which the writer expresses his philosophy in life. Answers to information or Wh- questions are explicitly stated in the text. 27

22. Sample Questions: 28 1. What is the title of the story? Who are the characters? Where does the grasshopper stay? Why does the ant save food?

23. 2. Second Dimension: Comprehension or Interpretation (Grasping Fully the Writer’s Ideas) 29 The reader is tasked to read between the lines to make inferences. Sufficient clues are given to enable the reader to arrive at the writer’s ideas. Questions on the second dimension are still about the story though the answers are derived and not lifted from the selection.

24. Sample Questions: 30 Compare the grasshopper and the ant. W h a t word s desc r i b e t h e an t ? the grasshopper? What does the saying “Save for a rainy day” mean?

25. 3. Third Dimension: Application 31 T he reader u s es o r ap p li es learne d mate r i a l s i n new a n d concrete situations, processes, effects, conclusions. Sample Question: If you were the grasshopper, what would you have done? Why is it important to practice thrift and economy?

26. Fourth Dimension: Analysis 32 The reader break s dow n t he ma t erial i n t o component facts so that its organizational can be unde r s t ood su c h as h y po t hesi s , s t a t e me n t of f ac t s, structure e l eme n t s, others.

27. Sample Questions: 33 W h a t i s the wr i ter’s purpo s e for writing the story? Literary device did the writer use to make the selection interesting to the readers?

28. 5. Fifth Dimension: Synthesis 34 The reader puts parts together to form a whole new pattern, structure, or design. He suggests or makes plans of action .

29. Sample Questions: 35 What other things can you save in order to economize? What ways can you suggest to save on the following Food? Electricity? Water? Clothing? Other resources?

30. 6. Sixth Dimension: Evaluation 36 T h i s i s j udg i ng the valu e of something using internal criteria.

31. Sample Questions: 37 Prove that the ant did right in storing food for the rainy day. Did the grasshopper deserve to go hungry? Cite reasons for your answer What can you say about people who recklessly spend their salary without thinking of saving a portion of it for future use? What advice can you give them?

32. How should questions be asked? 38

33. Simple Clear Definite Relevant Challenging and though-provoking Adapted to the age, abilities and interests of the students Require an extended response Use common vocabulary Properly directed 39

34. Effective Questioning should: 40 Reinforce and promote the learning objectives Include “staging” questions to draw pupils towards key understanding or to increase the level of challenge in a lesson as it proceeds. Involve all pupils Engage pupils in thinking for themselves Promote justification and reasoning

35. Create an atmosphere of trust where pupils’ opinions and ideas are valued Show connections between previous and new learning Encourage pupils to speculate and hypothesize Encourage pupils to ask as well as to “receive” questions E n c o urage p u p il s t o l i s t en a n d re sp o n d t o ea c h ot h er as well as to the teacher 41

36. Questioning Techniques in the Classroom Encourage students to ask questions at any time. Give adequate consideration to all questions--never evade a question. Scatter questions over the entire class. 42

37. Pose questions within the ability of the student to whom the question is addressed. complete Ask questions to the inattentive. Re q u i re s t ude n t s t o g i v e answers. 43

38. D o not p e r m i t fre q ue n t g r oup responses. Ask open-ended questions A v o i d as k i n g q uest i ons t h a t can be answered by guessing. Use the key words of questioning- - h o w , w hy , w h e n , w he r e, w hat, which . 44

39. H a v e s t ude n t s s p eak lou d l y so that all may hear. U s e c o rr e ct g r am m a r and terminology. Keep questions on the subject. W r i t e q u e s t i o n s i n y our l e s son plan. 45

40. When should we ask? 46

41. WHEN to ask?Questioning Cycle Credit: Rosie Piller and Ian Beatty . BEFORE Setting up instruction 47 Motivate Discover Predict-and-show Provoke thinking Assess prior knowledge

42. WHEN to ask? Questioning Cycle Credit: Rosie Piller and Ian Beatty . DURING Developing knowledge Check Application k n owle d ge / compre h e n s i on Analysis E v a l u a ti o n Synthesis Elicit misconception Exercise s k i ll 48

43. WHEN to ask? Questioning Cycle AFTER Assessing learning Exit poll “Big picture” Demonstrate success Review / Recap Credit: Rosie Piller and Ian Beatty . 49

44. WHEN to ask? Questioning Cycle osi e P i ll e r an d I a n B ea . BEFORE Setting up instruction AFTER Assessing learning DURING D e v e lo p ing knowledge Check Application knowledge/comprehension Analysis E v a l u a ti o n Synthesis Elicit misconception Exit poll Exercise skill “Big picture” Demonstrate success Review / Recap P r e d i c t -a n d - sh ow Provoke thinking Assess prior knowledge Motivate D i s cover 47

45. What questions should we ask? First Dimension: Knowledge or Literal Understanding Second Dimension: Comprehension or Interpretation (Grasping Fully the Writer’s Ideas) Third Dimension: Application 4. Fourth Dimension: Analysis 5. Fifth Dimension: Synthesis 6. Sixth Dimension: Evaluation

46. When should we ask? BEFORE Setting up instruction DURING Developing knowledge AFTER Assessing learning

47. Multiple Choice Questions Guidelines for Writing Advantages & Disadvantages

48. Multiple Choice Test

49. MCQ Used to measure knowledge outcomes and various types of learning outcomes. They are most widely used for measuring knowledge, comprehension, and application outcomes.

50. Stem Keyed Response Distracters Options

51. 1. The capital city of Canada is V a n c o u v er Montreal Toronto *d. Ottawa Terminology: Multiple Choice 4 1 2 Stem: presents the problem Keyed Response: correct or best answer Distracters: appear to be reasonable answers to the examinee who does not know the content Options: include the distracters and the keyed response. 3

52. S t rengths Learning outcomes from simple to complex can be measured. Highly structured and clear tasks are provided. A broad sample of achievement can be measured. Scores are more reliable than subjectively scored items (e.g., essays). Scoring is easy, objective, and reliable.

53. S t rengths Item analysis can reveal how difficult each item was and how well it discriminated between the strong and weaker students in the class Performance can be compared from class to class and year to year Can cover a lot of material very efficiently (about one item per minute of testing time). Items can be written so that students must discriminate among options that vary in degree of correctness

54. Limita t ions Constructing good items is time consuming. It is frequently difficult to find plausible distractors. This item is ineffective for measuring some types of problem solving and the ability to organize and express ideas. Real-world problem solving differs – a different process is involved in proposing a solution versus selecting a solution from a set of alternatives. Scores can be influenced by reading ability. There is a lack of feedback on individual thought processes – it is difficult to determine why individual students selected incorrect responses.

55. Limita t ions Students can sometimes read more into the question than was intended. Often focus on testing factual information and fails to test higher levels of cognitive thinking. Sometimes there is more than one defensible “correct” answer. They place a high degree of dependence on the student’s reading ability and the instructor’s writing ability. Does not provide a measure of writing ability. May encourage guessing.

56. S t ems Considerations Use clear, straight forward language Aim to write as a complete sentence Avoid use of unnecessary content Avoid the use of negatives. If you must use them embolden them to make them stand out Avoid giving clues in the question e.g an/a Avoid veering away from the content/cognitive level that you aimed to address 59

57. Guidelines for Writing: Stems Place most of the subject matter in the Stem ensures full statement of problem Eliminate extraneous material from the Stem goal is to measure student achievement, not to present new material maximize use of time for demonstrating understanding, not reading ability Avoid Negatively phrased Stems students may miss the qualifier use only when learning outcome requires this type of differentiation

58. Example 1 The stem of the original item below fails to present the problem adequately or to set a frame of reference for responding . Original World War II was: The result of the failure of the League of Nations. Horrible. Fought in Europe, Asia, and Africa. Fought during the period of 19391945. Revised In which of these time period was World War II fought? A. 1914-1917 B. 1929-1934 C. 1939-1945 D. 1951-1955 E. 1961-1969

59. Guidelines for Writing: Multiple Choice Items State stem in the form of a question. Weak Canada is a) a country b) where you live c) between the Pacific and Atlantic oceans d) all of the above Better Between which two oceans is Canada located? a) Atlantic and Indian b) Atlantic and Pacific c) Pacific and Indian

60. Options Correct answer and Distractors Distractors should be worded in a similar way to the correct answer and of a similar length (but not too close to cause confusion) Avoid repeating the same content in the options – move it to the stem if possible Avoid ‘all of the above’/ ‘none of the above Aim to use common misconceptions/student errors as distractors Distractors should be plausible There should be a sufficient number of distractors. A correct statement but not the answer to the question is a good distractor 67

61. Alternatives should not overlap (e.g., in the original form of this 68 item, if either of the first two alternatives is correct, “C” is also correct.) Original 1. During what age period is thumb-sucking likely to produce the greatest psychological trauma? Infancy Preschool period Before adolescence During adolescence After adolescence Revised During what age period is thumb-sucking likely to produce the greatest psychological trauma? From birth to 2 years old From 2 years to 5 years old From 5 years to 12 years old From 12 years to 20 years old 20 years of age or older

62. There should be no grammatical clues to the correct answer. Original 1. Albert Eisenstein was a: Anthropologist. Astronomer. Chemist. Mathematician Revised 1. Who was Albert Einstein? An anthropologist. An Astronomer. A chemist. A mathematician.

63. Types of Multiple Choice Items Correct Answer* Only one correct response Best Answer requires examinee to select alternative closest to being correct fine distinctions Multiple Answer More than one correct or best answer

64. Components of an MCQ Go o d MCQ Well f ormed STEM Clearly correct ans w er Plausible di s t r a c t o r s O p t i ons +/- feedback 67

65. I. Guidelines for Writing: MC Ensure similarity among alternatives with regard to: grammatical structure length mode of expression Grammatical errors provide unintentional clues to the answer When in doubt, students will select the longest alternative as the correct answer

66. Example: Length of Alternatives Neurotics are more likely than psychotics to be dangerous to society have delusional symptoms be dangerous to themselves *d. have insight into their own inappropriate behavior but nevertheless feel rather helpless in terms of dealing with their difficulties

67. Rules for Writing MCQs (Haladyna & Downing (1989) BALANCE THE KEY this refers to where you place the correct answer in the response choices. You should vary where you place the correct answer.

68. Use negatively stated stems sparingly and when using negatives such as NOT , underline or bold the print. Use none of the above and all of the above sparingly, and when you do use them, don't always make them the right answer. Only one option should be correct or clearly best.

69. All options should be homogenous and nearly equal in length. The stem (question) should contain only one main idea. Keep all options either singular or plural. Have four or five responses per stem (question).

70. When using incomplete statements place the blank space at the end of the stem versus the beginning. When possible organize the responses. Reduce wordiness. Keep the stem simple. When writing distracters, think of incorrect responses that students might make. (plausible)

71. USE SIMPLE TERMINOLOGY S p e l l ou t acrony m s an d / o r def i ne difficult – to – understand terms.

72. Additional Tips for Creating MCQs Limit the number of option Don’t lift phrases directly from the text or lecture Make sure questions are not culturally biased or contain sexist terminology or which have cultural bias

73. Write the correct answer before creating the distracters Put responses vertically below the stem Avoid unnecessary repetition in the response options

74. Revisit the questions you have formulated in the B-D-A instructional framework and make modifications or adjustment based from your learnings. 81

75. LET’S APPLY 82 Literacy In s t ruct i onal Framework Questions Reasons Skill/s De v e loped Before During After

76. 80