Authentic Assessment - USC and BU30 May 2024 public opt (1).pptx

Principles of good assessment and feedback apply across learning contexts, and this session will explore the ways in which we can consider teaching, learning and assessment in an era of generative AI. Drawing upon the ideas of Education 4.0, what is fit for purpose in the 21C Academy as we face huge pressures from the student body for more customisation and personalisation of education?

1. Authentic assessment in an age of Gen AI Anita Jones (SF) General manager CSALT Debbie Holley (PFHEA NTF) Professor of Learning Innovation Bournemouth University

2. Today we will start with making a paper airplane...and throwing it Make your paper airplane Decorate it Add you name to it....

3. The impact of GenAI – a worldwide issue

4. Future: the class of 2030

5. The class of 2030 A paradigm shift for the class of 2030 By the time today’s kindergartners enter the workforce, activities will substantially change across most occupations and will increasingly require the application of expertise and creative problem solving , as well as collaboration, management, and the development of people. McKinsey Whitepaper .

6. The Microsoft class of 2030 

7. Personalisation Re-imagining workspaces, learning spaces and educational spaces Feedback and feedforward Virtual coaching Micro credits Seamless flow in, through and out Universities and workspaces (University of the Future bringing Education 4.0 to life FICCI report 2018 )

8. Authentic assessment is a form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills Jon Mueller 2005 Mueller, J., 2005. The authentic assessment toolbox: enhancing student learning through online faculty development.  Journal of Online Learning and Teaching ,  1 (1), pp.1-7.

9. Beyond Blended JISC Beyond Blended Report

10. Bournemouth University Showcase bournemouth.ac.uk

11. Example: The virtual Library Student Led Project Planning Creating  Testing Using the virtual to motivate exploration of the real Access Hobbs and Holley full paper here:  

12. Godzilla Attack 2024: authentic learning for health professionals An 'alien attack' on Dorset Paramedic students practice a mass casualty evacuation exercise  Debrief to health mentors Filmed in 360 and scenarios recorded for our student mental health nurses, children's nurses, physiotherapists and occupational health students – embedded in our curriculum this year

13. 'Godzilla Attack' a mass casualty evacuation exercise  https://youtu.be/rmGwRzzUIFw?si=hFPPSdrNOgRm_uGT Example of student learning:   generates no metric footprint,  transformational learning taking place

14.  Augmented Reality: challenges and opportunities Living Manuscripts  Zoom filter:  you are your avatar https://www.bbc.co.uk/news/av/world-us-canada-56005428   www.muellerundschindler.com Click to add text

15. University of the Sunshine Coast usc.edu.au

16. TEQSA RFI and UniSC Action Plan on GenAI UniSC Position of AI, ethical and responsible use (L&T and R) Program Development Planning - working at program level The UniSC Context

17. Why Do It? We do not just want students to know the content of the disciplines when they graduate, we want them to be able to use the acquired knowledge and skills in the real world. Authentic assessments are direct measures Authentic assessments provide multiple paths to demonstration of learning Authentic assessments capture the constructive nature of learning

18. Think Authentic Assessment Students want to know that their work in university is preparing them for the workplace. What alternatives are there to the Essay and Exam? What type of assessment will get your students pumped about their chosen discipline/career?

19. Validity Does the assessment measure what it is supposed to measure? Think…

20. Reliability How consistent / fair /equity replicable is the assessment? Think…

21. Practicality How much effort is it to write and/or assess/mark? (Is it worth it?) Think…

22. As you design assignments, please consider: What learning outcome(s) is/are being assessed? Are the requirements clear? Is the word limit appropriate for the task? How easy would this task be to plagiarise ? How universally designed is the assessment? Think…

23. Keep Doing Encourage intrinsic motivation.  Highlight how the writing process helps students learn. Update academic integrity policies to make them explicit about the use of automated writing tools.  Ask students to affirm their submissions are their own work and not that of another person or of any automated system.  Analysis that draws on class discussion.  Analysis of recent events not in the training data for the system.  Assignments that articulate nuanced relationships between ideas.  Assign in-class writing as a supplement to or launching point for take-home assignments. Assign steps in the writing process and/or reflection on that process.  Hold individual conferences on student writing or ask students to submit audio/video reflections on their writing.  Test your own writing assignments.  Start Doing Stop Doing Requiring handwritten submissions Adopting surveillance tools Adopting systems trained to recognize specific student writing Teach students about text generators and “AI.”  Show students examples of inaccuracy, bias, logical, and stylistic problems in automated outputs.  Tell students that detectors for identifying “AI”-generated text exist and are improving.  Assign prompts that state-of-the-art systems such as ChatGPT are not good at.  Require verifiable sources and quotations.  Analysis of specifics from images, audio, or videos.  Analysis of longer texts Adapting College Writing for the Age of Large Language Models such as ChatGPT : Some Next Steps for Educators:  https://bit.ly/CAISteps Anna Mills & Lauren Goodlad , Jan 2023

24. EXAMPLES

25. Examples of Alternatives Reflections on the process At various times students turn in written descriptions of their research process. Problem solving approach What steps would be taken to solve a problem. Literature review Evaluative annotated bibliography. Science in the news Find evidence in literature for news release claims. Event initiated examples Evaluate a current event based on literature findings. Letter to the Editor Choose a newspaper/magazine and write an editorial.

26. Examples of Alternatives Design a Theme for a Conference Describe why this topic would be of interest to experts in this field. Diary Entry From a famous person in your field/patient with a disorder/legal client Evaluate thinking Have students discuss what they found and compare sources. Read and find facts Read an editorial, article, reflection and find facts to support it. Create a webpage Select a topic to do with course content. Biography Select a scholar/researcher in field and report on career, influences, major ideas, moods and trends in research program.

27. Examples of Alternatives Follow a piece of legislation through government processes What groups are lobbying for/against it and why? Who do you side with and why? Follow a current foreign policy issue as it develops Have students adopt the perspective of one of the various groups involved and predict the next move. Nominate someone for a Nobel prize Justify nomination. Adopt a persona Write journal entries, letters, commentaries from that person’s perspective Write a resume and cover letter (real or fictional) Find a relevant job posting in your field Write a presentation proposal Find a relevant scholarly or industry conference and write a proposal abstract. Write an exam Provide answers and justify rationale for questions asked.

28. Examples of Alternatives Write a review of a performance, a movie, a book, a journal article, a guest speaker lecture… Write a newspaper, magazine, Wikipedia, webpage story on a topic. Evaluate an article in a trade magazine (Psychology Today, Harvard Business Review) - Compared to findings in scholarly journals. Describe your dream job Have students research careers in the field and justify their choice of company, location, job, etc. Watch a movie or TV show Discuss how characters in different roles or procedures compare to national standards/how compare relationships to known models/theories Compare and contrast primary and secondary sources. Evaluate a website.

29. Examples of Alternatives Compare and contrast the state of knowledge on a topic in two different decades or eras. Conduct research but don’t write the final draft. Prepare for a hypothetical interview Have students do background research on a company or job offer and how they fit with job description. Compare and contrast the content, style, and audience of three different scholarly journals in a field. Compare and contrast a scholarly journal article with an article from a popular magazine. Prepare for an interview with a top figure in the field Justify questions, provide background notes. Compare and contrast the ways different disciplines deal with the same subject matter.

30. UniSC examples of mitigation Eg1 - Use of Cadmus for Task 2 and 3 – students work on the assignment in the platform, and educative approach, in advance, mitigates the ability to use AI. Eg 2 - Task 1 used a recorded Oral in response to a video scenario – students had to watch a video and then record themselves giving an oral response to the video scenario Eg 3 - Task 1 is a Reflective Journal based upon what students have observed and understand from course materials and tutorials etc Eg 4 - Task 2 Portfolio is a structured analysis based upon attendance at a fieldtrip

31. UniSC examples of embracing AI An example of an assessment tasks that relies on AI: Task 1 in Teaching Junior Secondary History Students asked to write an individual reflection on the process of marking and moderation and ability, strengths and weaknesses of generative AI to create students' responses in junior history and what implications this has for professional practice.

32. Follow up support:

33. UniSC Support Resources MyUniSC Intranet  Design and review assessment Mark and grade assessment Provide feedback on assessment USC Connect Knowledge Search terms Curriculum at USC Guidance on using rubrics Build a rubric in ten steps Writing Learning Outcomes Words for Assessment Assessment Product Types Grading Tool Options Course Outline Checklist 35 Canvas Resources How do I use the Gradebook? How do I get to SpeedGrader from an assignment, quiz, or graded discussion? How do I use SpeedGrader ?

34. Sector resources and follow up reading Transforming assessment Authentic assessment toolkit Tips to writing a strong rubric UNSW Learning Outcomes Guide How rubrics can inspire your students best work Phil Dawson resources Resources, including articles on the subject and samples of ChatGPT academic writing:  https://bit.ly/AIwritingEDU Adapting College Writing for the Age of Large Language Models such as ChatGPT : Some Next Steps for Educators:  https://bit.ly/CAISteps Espasa , Anna et al, ‘Does the Type of Feedback Channel Used in Online Learning Environments Matter? Students’ Perceptions and Impact on Learning’ (2022) 23(1) Active learning in higher education 49 Hattie, J., Crivelli , J., Van Gompel , K., West-Smith, P. and Wike, K., 2021. Feedback That Leads to Improvement in Student Essays: Testing the Hypothesis that “Where to Next” Feedback is Most Powerful. Frontiers in Education, 6. Long, Duri , & Magerko , Brian. (2020). What is AI Literacy? Competencies and Design Considerations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–16.  https://doi.org/10.1145/3313831.3376727 Bearman, M., Ajjawi , R., Boud , D., Tai, J. & Dawson, P. (2023). CRADLE Suggests... assessment and genAI . Centre for Research in Assessment and Digital Learning, Deakin University, Melbourne, Australia. doi: 10.6084/ m9.figshare.22494178

35. Debbie Holley - References and follow up reading Assessment future scenarios – outcome from codesign workshop: https://codesign.jiscinvolve.org/wp/2024/05/assessment-futures Beyond the blend (may 2024) https:// www.jisc.ac.uk /guides/beyond-blended-rethinking-curriculum-and-learning-design Exploring AI and assessment – avoid, outrun or embrace https://www.jisc.ac.uk/blog/exploring-ai-and-assessment-avoid-outrun-or-embrace