adt Principles of Assessment Education.pptx

Education

1. Dr. Aditi Chaudhuri MD, PGDGM, FAIMER Fellow Associate Professor, Community Medicine IHFW, Kolkata Principles of Assessment

2. Describe Outline Identify Purpose Describe Principles Enumerate Types Demonstrate Aligning with objectives - of A ssessment Will be able to …….

3. Basic terminologies Assessment: Purported action to collect information about students’ performance Evaluation: Action for collecting information on whether the educational objectives were achieved In assessment, we measure objectively , interpret subjectively & form a judgment based on norm/ criteria Test: Generally refers to written instrument Task: Generally use to assess skills/ behavior

4. The word ‘assess’ comes from the Latin verb ‘ assidere ’ meaning ‘ to sit with ’. Assessment What is Assessment? We do ‘with’ and ‘for’ students Not ‘to ’ students

5. The conventional answer : To categorize them as ‘ Pass’ or ‘Fail’ ?? Why do we assess students in general?

6. Why do we need to assess medical students? Training for 007: License to KILL We are certifying students fit to deal with human lives

7. Main Purpose of assessment : Enhancing Student Learning Developing Competence

8. Purpose of assessment Assessment gives feedback to the To see that the Educational objectives/ desired l ea r n ing ou t c om e s are achieved/not To adjust their teaching strategies Teachers

9. Purpose of assessment Assessment gives feedback to the To assess their learning To improve their pe rf o rm anc e St u de n ts

10. Most common question a lecturer hears from students ? Students are preoccupied with what constitutes the assessment in their chosen field “Will it be included in the Examination?” Assessment determines student approaches & direction to learning

11. What are the different types of assessment ?

12. Criterion Referenced Testing (CRT): Based on pre-decided criteria Norm Referenced Testing (NRT) Based on ranking of the students Types of assessment Formative : Involves ongoing informal observations throughout the term Summative : Final assessment Internal : (CIA)

13. T y p e s o f assessment Formative Assessment F ee d b ac k t o t h e s t u d e n ts during teaching learning process S u mm a t i v e A ss e ss m e n t E n d o f t e rm e x a m i n a t i o n t a k i n g i n t o co n s i d e r a ti o n t h e k n o w l e d g e & s k ills a c qu i r e d d u r i n g t he e n t i r e co u r s e e . g . , 1 st ,2 nd P r o f e s s i o n al exam I n t e r n a l A ss e ss m e n t F r e q u e n t o b s e r v a t i o n o f t he s t u d e n t by t he t e ac h er t o k n ow t he a c q u i s iti o n o f p r e d e t e rm i n e d d e s i r a b l e b e h a v i o r s , d e a ls mo r e w ith t h o s e a b iliti e s w h i c h c an n o t be t e s t e d in a s u mm a t i v e e v al u a ti o n

14. The Garden Analogy If we think of our STUDENTS as plants … Summative assessment of the plants is the process of simply measuring them. It might be interesting to compare and analyze measurements but, in themselves, these do not affect the growth of the plants. Formative assessment , on the other hand, is the equivalent of watering the plants appropriate to their needs - directly affecting their growth.

15. V a li d ity Does the assessment task assess what you want to assess ? Validity ensures that assessment tasks and the assessment criteria effectively measure the student’s attainment of the intended learning outcomes at an appropriate level . You want to assess the skill of palpation of liver: -- Giving a question to write the steps… is it a valid assessment tool?

16. Reliability Means that different assessors using the same assessment criteria and marking scheme would arrive at the same results . - Students ranked better in NEET, do they do better in MBBS? - Same answer script assessed to two teachers in the same department - Same answer script assessed at two different times

17. Objectivity: The extent to which independent and competent examiner agree on what constitute an appropriate answer for each of the element of the measuring tool Educational Impact: The overall impact of assessment on students, health system & on Community.

18. Fairness ………….. ??????

19. Ali gnm e n t Educational Objective/Desired learning outcomes Assessment Teacher perspective: Student perspective : Assessment Learning activities Teaching activities Outcomes

20. Take Home Message Define the desired learning outcomes (DLO) Choosing teaching activity that leads to DLO Assessing students on the desired learning outcomes

21. Alignment through Blueprinting Planning Assessment of all domains with requisite weightage to content

22. SIO Task Cognitive Domain Psychomotor Domain Affective Instrument Enumerate causes of U-5 under-nutrition List the factors associated with under-five under-nutrition Recall SAQ List the indicators of Under-nutrition List the indicators used to measure under-nutrition Recall SAQ Interpret the growth chart of an under-five Interpretation of plotted weight according to color zone Recall, Interpreta-tion , Problem Solving SAQ Assess the nutritional status of an under-5 child Calculate age in months Measure weight Plotting in Growth Chart Interpretation Problem Solving Imitation & control OSCE/ Structured checklist Counsel the mother Name the persons who supervise RNTCP at TU level Receive Respond OSCE/ structured checklist Formative assessment of 7 th Sem Students in Community Medicine

23. Techniques of Assessment

24. Assessment Tools Written Examination -Essay/SAQ/MEQ - Selected-response assessment MCQ Matching T r ue / f a l se Oral : Viva Voce Practical: Case/ OSPE/ OSCE

25. Essay Type Questions & Short Answer Questions.

26. Exercise : 2 min Frame one question University Examination Any speciality by group

27. 1. Essay questions Supply type 2. Short answer questions 3. Multiple choice questions S e l e c t i o n type

28. Essay Type questions

29. Most widely used form Student has freedom, within the subject context, to determine the nature and scope of the answer. Also known as free response questions Useful for evaluating higher domains of learning and comprehension

30. Examples Write an essay on Maternal Mortality in India Discuss the Polio Eradication in India .

31. Advantages/Disadvantages of Essay type questions

32. A d v a n t a g e s Easy to set Tests students ability towards Understanding Written self-expression Organization/Planning an answer C larity of ideas

33. Disadvantages Time consuming High degree of subjectivity is involved in marking. The marking is unreliable With deliberate efforts examiner can build reliability into his marking of unstructured essay questions

34. 3. Low Validity as only a small range of content may be covered 4. Extraneous factors influence the marks awarded Hand writing Grammar Diagrams Under linings Use of pens I am a student I is an student I am a student I am a student I a m a st u dent

35. 5. Examiner may be biased towards a particular point of view. Inter-rater & Intra-rater difference in marks (Examiner tough/ lenient ; delightful/ morose ) 6 . Permit and occasionally encourage bluffing

36. Framing of Essay Questions Objective for assessment should be same as learning objective Example : Learning objective: Student should be able to enumerate 10 common toxic effects of a particular drug. Matching question: Enumerate the common toxic effects of this drug Therefore more precise and clearer the objectives , easier and better to formulate good essay questions

37. The question should clearly specify to the student as to what is expected of Him/her. Maternal Mortality Ratio One student current situation Another factors contributing Another strategies to reduce Technically all correct Student marked on different abilities still score equal marks

38. Structure the Essay with weightage (SEQ) Write an essay on Maternal Mortality Ratio covering the Definition (2) Factors contributing (3) Current situation (3) Strategies under national program for reduction (4) clearly specifies to the student as to what is expected of Him/her.

39. Example Write the management of hemolytic jaundice in aneonate . Write the management of tuberculosis in a pregnant female HIV patient. Phrase the question in a simple, clear & straight forward language Where necessary provide reference points

40. Sampling of a few answer script first before marking SLO should be the criteria, give consensus weightage Avoid bias against poor presentation bad handwriting spelling and grammar Ensure adequate time for scoring all the answers Conceal the identity of the student Mark essays question by question rather than student by student Always good if essay questions are marked by more than one examiner and then take average marks Assessing EQ

41. Modified Essay Questions(MEQs)

42. A problem solving type of question P roblem or case scenario given to students Q uestions are framed based on problem/ case Student has to apply what he has learnt in the context of given question.

43. Q . Mr. Sen admitted in Male Surgical Ward with diagnosis of thyrotoxicosis with the advice of thyroidectomy . Define thyrotoxicosis . (2) Write the common signs & symptoms of thyrotoxicosis . (3) Describe the special post-operative management of Mr. Sen following nursing process. (8) List the common complications of thyroidectomy . (2) 2 nd Year GNM Exam 2017

44. MEQs can assess ( as you wish ): R ecall C omprehension A nalysis S ynthesis & E valuation of knowledge Advantages: Can be used for higher order abilities More specific & less subjective Focus on Clinical Competence, Management skill Can help in finding specific area of weakness Disadvantages: - Penalized multiple times for one mistake - Preparation is resource & time intensive

45. Short answer questions

46. Framed to avoid bias due to literary presentation, style etc. Different than " short notes " Generally tests lower domains of learning. Used in situation where reliability of scoring & objectivity is preferred

47. Simplest forms of SAQs requires the students to write A word A Phrase Number Sentence To complete a statement or a diagram.

48. Types Completion type Unique / one best response type Open SAQs

49. 1. Completion type Simple questions framed for testing factual recall .

50. The microorganism responsible for swine flu is Body mass index = / Impaired Fasting Glucose is considered when Fasting blood glucose rises above ___ mg/dl (WHO criteria) . The dose of Vitamin A for prophylaxis in children above one year of age is IU

51. 2 . Unique or “one best answer” type SAQs

52. Examples Write the dose of Isoniazid (in mg) for a one year old child weighing 10 kg under daily DOTS Classify the following antibiotics as bactericidal or bacteriostatic Penicillin Chloramphenicol Cephalosporin Tetracycline Rifampicin

53. 3.Open SAQs Provide some option to the student regarding choice of correct answer

54. Examples Enumerate three side effects of streptomycin Enumerate three causes of massive spleenomegaly These two questions require the student to choose three options out of the many available

55. Framing of SAQs Simple language Wording of questions should be unambiguous Well defined task should be given so very little scope of subjective interpretation . Give appropriate reference whenever required Statement should ne positively worded Avoid unintentional clues like ‘a’, or ‘an ’ or ‘answer space’ which is too proportional Indicate mark to each question

56. Example Cholera is __________ A water-borne disease A communicable disease A disease by V. Choleri The question can be better framed as: The micro-organism causing Cholera is _______ Example (appropriate reference): Infant Mortality Rate of India according to SRS 2016 estimation is ______ per 1000 live births

57. Advantages Questions are easily set Easy to score Range of subject area covered is wide Marked fairly objectively and reliability is high The Longer the expected answer, more is the subjectivity

58. Disadvantages 3-4 times more time is required for answering a SAQ as compared to MCQs so less number of questions in test paper hence reliability less than MCQ paper Marking of Open SAQs involves some degree of subjectivity.

59. A few unique methods of assessment 360 ◦ / Multi-source Assessment: - Assessment by teachers, peers, subordinates, paramedics etc. Portfolio-based Assessment: - Against the pre-decided SIOs, how far they have achieved is to be documented periodically in portfolios Visual Analog Scale: Used for subjective assessment like pain, satisfaction etc. Rate your satisfaction with this session in a scale of 0 to 100, where 0= no satisfaction & 100= absolute satisfaction 100

60. C on cl u si o n For students : “Assessment is the curriculum ” Students will learn what they think they will be assessed on , not what is in the curriculum.

61. Assess m e n t