RIDE 2010 presentation - Ipsative assessment and motivation of distance learners

Research in Distance Education: impact on practice conference, 27 October 2010. Presentation in Assessment Strand by Dr Gwyneth Hughes, Institute of Education and Dr Megan Crawford, Oxford Brookes University.
More details at www.cde.london.ac.uk.

1. Ipsative assessment and motivation of distance learners Gwyneth Hughes and Megan Crawford

2. 2 Contents 1 Why does assessment need radical reform? 2 Ipsative assessment -what are the possible benefits? 3 Results of a study on ipsative feedback for distance learners 4 Recommendations

3. Why assessment reform? Problems with current assessment practices Dominance of summative assessment and grading over formative ( Gibbs, 2006 ) Assessment can be demotivating ( Broadfoot , 1996) Feedback is not helpful for students so they don’t act ( Lizzio & Wilson 2008 ) Suggestions for improving feedback practice More clarity over standards and criteria More dialogue Improved formative feedback ( Nicol & Macfarlane-Dick, 2006 ) 3

4. What if an obsession with criteria and standards is at the root of the problem? ‘ if we wish to discover the truth about an educational system, we must first look to its assessment … ’ ( Rowntree 1987) 4

5. Ipsative assessment as an alternative to criteria-referenced assessment 5

6. Personal best 6

7. 7 What is ipsative assessment ? An ipsative assessment compares current performance with a previous performance. Ipsative feedback Ipsative grades

8. 8 Possible benefits of ipsative assessment Improves the usability of feedback e.g. through closing feedback loops See handout Links up generic feedback across modules in a course Improves self-esteem by rewarding progress Ipsative grades might motivate learners to act on feedback

9. Possible use with distance learners Can aid motivation at a distance as part of supportive open learning. Written feedback is particularly important to students learning wholly at a distance- can help them to respond in revisions and following pieces of work. The tutor can take on a coaching role in a very direct way and help their students personal response to feedback which at a distance may be even more tied up with emotions, identity and feelings of self-worth than F2F. 9

10. Study on ipsative feedback and learner motivation MA in Applied Educational Leadership and Management (AELM) is delivered wholly online to a fixed cohort. Our study looked at three tutors on the two core modules, where tutors had had specific training in giving good formative feedback. Module had been commended by External Examiner for its high quality written feedback. 10

11. Tutor feedback profile 11 Criteria-related Task-orientated Criteria-related Generic Ipsative Task-orientated Ipsative Generic

12. Examples of feedback Typical comments comparing the work with assessment criteria and standards were: “Have a look at the Assessment Criteria P. 37 to 39 of the Course Handbook: 1) Criteria A: you show a good level of understanding of the key issues and have begun to present extracts from the relevant literature..” (Tutor B) “What then might have taken the assignment to the next grade level? … you might have engaged rather more in a critical analysis of material in this field.” (Tutor A) 12

13. Ipsative feedback: student views Feedback was perceived as vague lacking clarity Learners used grades and feedback to self-assess on making progress but some would like more help e.g. “I do think about my own progress but of course I am never entirely confident in my own judgment of this!” “ if someone would read my first and second assignment and examine my progress as a paper writer and give me general comments about how I did or didn’t improve and what I needed to focus on that will be very helpful.” 13

14. Tutor feedback profile 14 Criteria-related Task-orientated Criteria-related Generic Ipsative Task-orientated Ipsative Generic

15. Recommended profile 15 Criteria-related Task-orientated Criteria-related Generic Ipsative Task-orientated Ipsative Generic

16. Ipsative feedback: tutor views “…if one tutor supervises the whole process of an assignment from the draft to the final submission, then ipsative assessment can be included fairly easily” Concerns were raised about needing staff development and that a new assessment approach might be time consuming. 16

17. Ipsative grades: student views “I fully appreciate that my final grade will not be based on my efforts, motivation or progress but simply on the work I submit, and that students who seem to have made far less effort may end up with better final grades if their work is of a higher quality. That is the nature of academic qualifications and it should remain this way if the end qualification is to maintain its current status as a well-respected academic certificate.” 17

18. Ipsative grades: tutor views: “ It ( ipsative grading) is a radical proposal and…….It might motivate (lower achievers)…the only way I would see it working is if they get a grade for effort and for achievement. But I doubt that the effort would count towards the final mark. I haven’t had enough time to think about it.” 18

19. Recommendations Ipsative feedback could be introduced both to motivate students and to reduce obsession with grades and criteria Any ipsative grades would need careful explanation of the benefits Ipsative assessment would need to be included in the assessment regime and not provide extra work Assessors will probably need staff development and guidance 19

20. [email protected] m [email protected] 20

21. 21 References Biggs, J. 2003. Teaching for Quality Learning at University. Buckingham: Society for Research into Higher education and Open University Press. Broadfoot , P. 1996. Education, Assessment and Society. Buckingham: Open University Press. Carless, D. 2006. Differing perceptions in the feedback process. Studies in Higher Education. 3, no. 2: 219-233. Hughes, G. & Crawford, M. 2009. Challenging Higher Education: knowledge, policy and practice. Society for Research in Higher Education conference papers, Newport, Wales.   Nicol , D. & Macfarlane,-Dick, D. 2006. Formative assessment and self regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education 31 no. 2: 199-218. Institute of Education University of London 20 Bedford Way London WC1H 0AL Tel [phone] Fax [phone] Email [email protected] Web www.ioe.ac.uk