Module 10 Knowledge Test_revised in health

Knowledge

1. Prepare and Use Knowledge Assessments

2. Introduction Why is knowledge assessed? What problems did you have with tests as a student? As a faculty member? 2

3. Objectives Select methods for assessing students’ knowledge Prepare knowledge assessments Develop questions for objective written examinations Administer and score knowledge assessments Use assessment results to improve performance 3

4. Knowledge Assessments Formative T o help guide students toward meeting the learning objectives Summative T o ensure that students finish the course with the required knowledge 4 How is knowledge assessed?

5. Think-pair-share What method of knowledge assessment method have you used? How do create the items? 5

6. Item type Selected Response Learners select correct answer Highly objective Highly reliable Easy to score Difficult to test clinical reasoning Difficult to write items Constructed Response Learners tell/ write what they know Less objective Less reliable Difficult to score Recall dependent Test few objectives Easy to write items 6

7. Key Points in Preparing Knowledge Assessments Identify the learning objectives Use simple and clear language & grammar Provide clear directions for each type of item Develop answer keys Arrange test items by subject matter, type of question, or level of difficulty Review the assembled test 7

8. Preparing a Test Bank Work in a group to develop questions. Sort questions based on subject, category, level of knowledge, or type of test item. Store the questions on a computer or cards. Ensure that only authorized persons have access to the question bank. 8

9. True/False - Advantages Can be written short and clear Can quickly test many objectives 9

10. Disadvantages High chance of guessing (50 %) Relatively difficult to construct Encourage memorization of rote fact May disadvantage the knowledgeable Correct false answer may not mean fact is known Low reliability 10

11. Tips for Writing Use clear, concise, and understandable language Avoid negatively stated items & use statements that are either completely true or false Avoid words that frequently identify a statement as true or false (mostly, often..) Make true statements equivalent in length and number to false statements Avoid negatives when possible Provide clear directions 11

12. Multiple Choice Questions (MCQ): Advantages Low guessing score Easily analyzed using statistics Can test MOST levels of Bloom’s Taxonomy Easy to score Highly reliable 12

13. Disadvantages More challenging to write, especially at higher levels Probably can not test synthesis Test-wise students can raise guessing score 13

14. Multiple Choice Elements 14 When is the risk of transmission of HIV from the mother to the child GREATEST? A. During pregnancy B. During labor & delivery C. During breast feeding Stem Distracter 1 Key Distracter 2

15. Tell us from your experience things that you don’t want to see in MCQ? 15

16. Tips for Writing Stem Relate items to learning objectives Write items for different levels of learning Establish the problem clearly Provide a complete statement Keep stem as short as possible Word the stem positively Write simple stems that state the problem and avoid using negative terms 16

17. Tips for Writing Distracters #1 Consider the number of distracters Write plausible but not trivial options Place options in logical order Write mutually exclusive options Use technical jargon from other parts of the course 17

18. Tips for Writing Distracters #2 Link options to each other Avoid “all of the above” Use “none of the above” sparingly Avoid options like A and B but not C… Avoid long options & keep similar in length Avoid patterns in keyed response 18

19. 19 Sample MCQ Direction Each of the items in this test is followed by four possible responses. For each item, choose ONE correct answer. Indicate your answer by circling the letter of your choice. Each correct response is worth one point.

20. MCQ: Testing Higher Level of Knowledge Use context-dependent items Provide data that requires Application of facts and principles Analysis and interpretation of data Evaluation of two or more plans Use of the management process 20

21. Context-Dependent Items Based on a real life case scenarios Help to assess the students’ ability to: Think critically Make correct decisions and Solve/ manage problems 21

22. Context-Dependent Items Tips for Writing Organize the context clearly Keep data reasonably short Consider using photos, diagrams, graphs, videos Provide sufficient information for analysis Questions should focus on the underlying thought process 22

23. Matching Questions Can be difficult to construct High basement score depending on how it is written May be confusing to some students 23

24. Matching Questions tips Consists of two lists Premise Response Imperfect matching items are more objective Tips: F ocus on one subject K eep statements brief A rrange in logical order 24

25. Short-Answer Question Tips Make the questions clear and easy to understand Prepare a structure marking sheet Include questions that have possible multiple responses 25

26. Test Administration Time allowed How to select and record answers Scoring system to be used Physical environment Remain in the room 26

27. Scoring the Knowledge Assessment Answers can be marked directly onto the test or onto a separate answer sheet Use an answer key (or a computer with scanner) to score the tests See the module for more information on scoring knowledge assessments 27

28. Using Assessment Results Formative assessments help students decide what content areas they need to spend more time studying Summative assessments determine if students are meeting the learning objectives. 28

29. Helping Students Learn Instruct them to review the material related to the questions they missed Give them an opportunity to ask questions about any test item When many students had trouble with the same question T he teaching methods or materials did not adequately address that learning objective OR T he question needs to be rewritten 29

30. Item Shells A (patient description) is unable to (functional disability) . Which of the following is most likely to have been injured? A ( patient description ) has ( symptoms, signs, or specific disease ) and is being treated with ( drug or drug class ). The drug acts by inhibiting which of the following ( functions, processes )? 30

31. Patient Vignettes Age, Gender (e.g. A 45-year-old man) Site of Care (e.g. comes to the emergency department) Presenting Complaint (e.g. because of a headache) Duration (e.g. that has continued for 2 days). Patient History (with family history?) Physical Findings 31

32. Item Modeling Design new items from existing items by altering: Patient age or sex Setting (Hospital to Homebirth, OPD to Ward) Type of Case (AP to IP) Item format 32

33. Summary What questions do you have about developing, administering and scoring knowledge assessments? How can we assist faculty members in developing, administering and scoring knowledge assessments to support implementation of curricula? 33

34. 34

35. Activity Individually construct – 30 min 1 knowledge, 1 comprehension and 1 context dependent item (application or problem-solving ) MCQ 2 True/False and one matching item Item critique within group – 10 min One selected for large group presentation and discussion - 10 min 35