Generative AI and Educational Development: opportunity, challenge, or potential disaster?, Sue Beckingham

Our students need to develop their skillsets, encompassing and integrating traditional academic skills with new Generative AI (GenAI) techniques so they can
succeed in their studies and beyond. This session analyses current GenAI developments, identifying applications with particular implications for educational developers and educational development units. We review surveys of student use of GenAI, different applications of GenAI to enhance pedagogy, and offer a practical crash course in “prompt-craft” to demonstrate future possibilities.

1. Generative AI and Educational Development: opportunity, challenge, or potential disaster? Sue Beckingham, Sheffield Hallam University. Beverley Pickard-Jones, Bangor University. Fay Short, Bangor University Peter Hartley, Edge Hill University

2. Sue Beckingham Sue is an Associate Professor (Learning and Teaching), National Teaching Fellow, Principal Lecturer in Digital Analytics and Technologies, and a Learning and Teaching Portfolio Lead at Sheffield Hallam University. She is also a Certified Management and Business Educator, a Senior Fellow of the Higher Education Academy, a Fellow of the Staff and Educational Development Association, and a Visiting Fellow at Edge Hill University. Her research interests include social media for learning and digital identity, groupwork, and the use of technology to enhance learning and teaching; and has published and presented this work nationally and internationally as an invited keynote speaker. She is a co-founder of the international # LTHEchat ' Learning and Teaching in Higher Education Twitter Chat ' and the Social Media for Learning in HE Conference @ SocMedHE . Publications: https://www.suebeckingham.com/p/publications.html Twitter: @ suebecks Blog: http://socialmediaforlearning.com/  LinkedIn:  linkedin.com/in/suebeckingham Email: [email protected] 2022 Author of book chapters

3. Beverley Pickard-Jones Bev is a Lecturer in Psychology (Teaching and Scholarship), a Fellow of the HEA, and the Employability Lead at Bangor University. Formerly a business consultant to the health technology industry and a statistician with the UK Civil Service, Bev is currently pursuing psychopedagogical research into the cognitive processes underlying successful GenAI usage in students, and is developing a book on using Power BI and dashboarding to modernise workflows and improve data-driven decision making in universities. Email: [email protected] LinkedIn: https://www.linkedin.com/in/beverleypickardjones/

4. Fay Short Fay Short is a Professor in the School of Psychology and Sport Science at Bangor University and a Chartered Psychologist with the British Psychological Society. She is also a qualified therapist, and a registered member of the British Association for Counselling and Psychotherapy and an Accredited Registrant of the National Counselling and Psychotherapy Society. Having previously worked as Director of Teaching and Learning for the College of Human Sciences, her current professional roles today include Course Director for the MSc in Counselling and Associate Pro-Vice Chancellor for Employability at Bangor University. Outside of her professional roles, she is an amateur chew toy for her dog Dexter. Email: [email protected] LinkedIn: https://www.linkedin.com/in/fay-short-aba88480/

5. Peter Hartley [email protected] Now into my 3rd career: Career 1: Teaching academic in Communication Studies: lecturer > dept head > Professor of Communication. Career 2: Educational developer . National Teaching Fellow. Head of Ed Development Unit Professor of Education Development. Career 3: Educational consultan t. Visiting Professor at Edge Hill. External examiner & writer. Work on: project evaluation; visual thinking; learning space design; National Teaching Repository (NTR) NTF/CATE mentoring; assessment strategies (PASS project); online and GenAI development/ethical issues; new SEDA/Jisc Student Partnership Impact Award. 5 https://he.palgrave.com/page/detail/Learning-Development-in-Higher-Education/?K=9780230241480 PROGRAMME ASSESSMENT Chapters with Ruth Whitfield in………………………………………and in…..… https://www.bloomsbury.com/uk/success-in-groupwork-9781350933491/ 2 nd edition, February, 2022 With Sue Beckingham & Mark Dawson From 2022 : https://www.springer.com/gp/book/9783030263416 May 2023 . https://www.routledge.com/Interpersonal-Communication/Hartley/p/book/9780415181075 3 rd edition, With Helena Knapton & Susie Marriott Edge Hill University https://www.routledge.com/Professional-and-Business-Communication-Personal-Strategies-for-the-Post-Digital/Hartley-Marriott-Knapton/p/book/9781032268002 https://ww.routledge.com/products/9781138854710 Spring 2024 https://www.routledge.com/Using-Generative-AI-Effectively-in-Higher-Education-Sustainable-and-Ethical-Artificial-Intelligence-for-the-Common-Good/Beckingham-Lawrence-Powell-Hartley/p/book/9781032773988

6. Coming May 31 2024 ... An edited collection in the SEDA/Routledge series focusing on GenAI applications, initiatives and issues – chapters from UK and Australasia Beckingham, S., Lawrence, J., Powell, S. and Hartley, P. (2024) Using Generative AI Effectively in Higher Education . London: Routledge.

7. Preamble We will not go through all the slides in detail. We have included some extra information and links on the slides so you can use this slide deck as a resource for your later reference. Please contact us for more information if you need to.

8. This session

9. Padlet https://padlet.com/suebecks/GenAIandEdDev We will use the Padlet for: 1. Questions for the presenters. 2. Ideas for using GenAI to address HE challenges. 3. Useful resources on GenAI in HE. 4. Strategies to support effective use of GenAI.  

10. Part 1: How do we prepare for the next wave? (and what do we all need to know?) Understanding the software approach (how it developed and where it is heading). Exploring what we can (and cannot) do with GenAI. And what staff/students do with it? Recognising the issues (and developing strategies to respond). Developing and improving our own skills (incorporating it in teaching). Developing and implementing effective university-wide strategies

11. Preparing for the next wave 1. Understanding the new software.

12. What is Generative AI ( GenAI ) like ChatGPT ? Key components Dataset/database Uses powerful ‘compute’ to build ‘Large Language Models’ (LLMs), containing massive amount of text (stored as ‘tokens’). Computer architecture Uses neural networks from AI.* ‘Attention’ and Transformer. Training LLMs are ‘trained ’ and ‘fine-tuned’.* Interface Chatbot - mimics human conversation. Can now respond to voice input, e.g. ChatGPT on your phone. Refining and improving. * Stephen Wolfram (2023) What is ChatGPT doing and why does it work . Wolfram Media Inc.

13. What is new about Generative AI ( GenAI )? “Traditional AI systems are used to make predictions based on existing data. The new wave of Gen AI tools goes even further, by creating new content based on existing data. ...- It's about simulating human creativity . So, traditional AI excels at pattern recognition, while GenAI excels at pattern creation.” From Marr, 2024.

14. First reactions to ChatGPT & GenerativeAI ? From Saunders et al (in Beckingham et al, 2024) 4 ‘threads of resistance’ ”desire to outlaw the technology” “indifference to the technology” “desire to return to traditional assessments” “awareness of issues ... but few practical ideas on how to resolve them.” F rom Philippa Hardman: 3 ‘teams From Philippa Hardman See at https://uk.linkedin.com/posts/dr-philippa-hardman-057851120_education-ai-machinelearning-activity-7029103206135689216--_Jb

15. How far do we understand this technology? ‘ emergent capabilities ’ unexpected weaknesses. “... Any attempt at a precise explanation of an LLM’s behaviour is doomed to be too complex for any human to understand.” Professor Sam Bowman, New York University. Quoted in Mollick, 2024,p28.

16. Illustrating the pace of change: An ‘accurate’ image? The prompt and the caption The prompt Please produce a photo-realistic image which conveys the importance of biological research and education. The caption produced by Dall-E 3 The image conveys the importance of biological research and education, capturing a moment of discovery and collaboration in a laboratory setting. The image

17. The importance of the prompt (and the cat). Prompt and caption Prompt Please add a tabby cat, sitting on the bench. Caption The image has been updated to include a tabby cat sitting on the bench, bringing a serene and comforting presence to the scene of scientific discovery and collaboration. The revised image

18. Major Developments since early 2023 Software Multiple LLMs (+ different sizes/levels) Subscription and/or usage levels. Proprietary of Open Source Improved quality, e.g. coding. Integration into main everyday apps, e.g. Microsoft Copilot. ‘Agents’ and ‘Assistants’ Multimodal , e.g. text to text, plus Text to image to video, and back Image analysis Audio manipulation & translation Hardware Your next pc? – the ‘AI pc’. Faster chips/processing, e.g. Groq Your Copilot key on the keyboard. New stuff from Apple? GenAI on your mobile.

19. A summary of trends? https://aiindex.stanford.edu/wp-content/uploads/2024/04/HAI_AI-Index-Report-2024

20. And the latest generation of apps (this week)? GPT-4o (o = omni-modal) from OpenAI : key features* “twice as fast as previous model”. New ‘voice mode’. Minimum latency (‘real-time talk’). Can interrupt the app. Desktop app. Multi-language. Access to cameras (e.g. iPhone). Can ‘use’ iPad screen. Can identify emotion. Handles text, images and audio. ‘Memory.’ Free model uses GPT-4. Free model provides access to all the GPTs (GPT store). Subscription model has extra facilities (e.g. re vision) & higher usage limits. * All features announced May 13. May not all be available yet. Some on Mac first.

21. And there’s more to come ... May 14: Google I/O 2024. June 10 : Apple WWDC 2024

22. Padlet https://padlet.com/suebecks/GenAIandEdDev We will use the Padlet for: 1. Questions for the presenters. 2. Ideas for using GenAI to address HE challenges. 3. Useful resources on GenAI in HE. 4. Strategies to support effective use of GenAI.  

23. Preparing for the next wave 2. What can and cannot) the software do?

24. An international perspective 28/4/24. See at: https://www.weforum.org/publications/shaping-the-future-of-learning-the-role-of-ai-in-education-4-0 /

25. Text Text Text Text Image Audio Video Language A Drawing Text Text ChatGPT Google Gemini MS Copilot Claude Image DALLE-2 Adobe Firefly Midjourney Padlet Audio Recast Murf.ai Voicemaker Video HeyGen Pictory D-ID Text Vid2text Translate Video ScriptMe Text Otter.ai Veed.io Text ChatGPT Plugger.ai Clip Interrogator * Language B Google Translate HeyGen (video) Image Scribble Diffusion Quick Draw Profile ProfilePicture PhotoAI Character AI There’s an AI [app] for that * (text in image)

26. There’s an AI [app] for that: the research cycle

27. What are your students using it for? Research from Louise Drumm (Edinburgh Napier)

28. Survey on AI use in Students 453 students 33 (7.3%) 144 (31.8%) 276 (60.9%) Used AI tools No Not sure Yes 430 (94.9%) 12 (2.6%) 11 (2.4%) Aware of AI tools No Not sure Yes 453 students

29. Survey on AI use in Students 33 (7.3%) 144 (31.8%) 276 (60.9%) Used AI tools No Not sure Yes 430 (94.9%) 12 (2.6%) 11 (2.4%) Aware of AI tools No Not sure Yes

30. 12.2% 19.6% Students' understanding of AI policies 50 (56.2%) 33 (37.1%) 6 (6.7%) Did you understand it? No Not sure Yes Does your university have an AI policy? No Not sure Yes Have you read it? No Yes, partially Yes, in full

31. Padlet https://padlet.com/suebecks/GenAIandEdDev We will use the Padlet for: 1. Questions for the presenters. 2. Ideas for using GenAI to address HE challenges. 3. Useful resources on GenAI in HE. 4. Strategies to support effective use of GenAI.  

32. Preparing for the next wave 3. The big issues?

33. Why worry about Generative AI ( GenAI )? The ‘Godfather of AI’ explains... (March 2024) https://www.youtube.com/watch?v=N1TEjTeQeg0&t=31s

34. Gen AI: main issues Hallucinations and inaccuracies: misinformation and deepfakes. Predicting the future: - ‘End of the world?’ - Jobs or no jobs? Regulation and intervention. Social and environmental impact. Costs and access.

35. 1. Hallucinations and inaccuracies https://www.scientificamerican.com/article/chatbot-hallucinations-inevitable/

36. 2. Predictions? #1 End of the world - Panic in the media

37. #2 Do the experts agree? Mo Gawdat At: https://www.youtube.com/watch?v=itY6VWpdECc&t=212s “AI will happen” “It will get smarter than us” “Bad things will happen. ” Emad Mostaque , Forner CEO of Stability AI At: https://www.youtube.com/watch?v=wCOaTFKcExo&t=4s “we can build, we have agency” “you can create anything you can imagine” “interested in amplified human intelligence”

38. #3 Employment and employability? https://www.ippr.org/media-office/up-to-8-million-uk-jobs-at-risk-from-ai-unless-government-acts-finds-ippr

39. 3. Regulation and intervention New regulations/legislation and courtroom battles. The latest ‘culture war’ See at: https://www.bbc.co.uk/news/technology-68445981

40. 4. Environmental impact Impact on environment e.g. Energy and water problems.

41. 5. Costs To individual users? LAST WEEK: ChatGPT ‘free’ - using GPT3.5 ‘Plus’ – GPT4 and Dall-E3, and ‘build your own’ GPTs. Further add-ons – e.g. Team FROM NOW: Chat GPT4o, free to all. ‘Plus’ subscribers have higher access limits and new facilities first. What will other LLMs do? What will we be able to afford? Development and maintenance? See at: https://www.scientificamerican.com/article/stanford-ai-index-rapid-progress/

42. And finally for this part: AI elephants in the room Costs? Who is talking about these? What is the full environmental impact of AI? Exploitation and social justice? Exploitation across the world? (see Murgia , 2024) Who controls GenAI ? What implications might this have for Higher Education? e.g. Where are OpenAI heading after the recent organizational stramash?

43. Preparing for the next wave 4. New skills?

44. New ways of learning Socratic learning Develop questioning technique ‘promptcraft’ or ‘prompt engineering’ Personal tutor Clarify difficult or confusing topics Alternative ways of presenting information Writing assistance (grammar, spelling etc.) Fact finding Identify research aps Gather information and data Evaluate the accuracy and reliability Fact checking Analysing Critical thinking Ideation Collaboration and co-creation Discussion and debate Multimodal learning Text to image/slide decks/video Enquiry Interpret findings Verbalise and explain

45. Example of a new skill: improving prompts

46. From ‘prompting to dialogue’ to ‘co-intelligence’ ( Mollick’s 4 ‘rules’). Do you agree?

47. Preparing for the next wave 5. What should course teams/ departments/ universities do to respond?

48. A possible new format for all assignments? “Show your workings” 1. Produce the required output/outcomes, e.g. Product/design. Format? Essay, report, blog, ppt etc. Research log of all sources. Reflection/evaluation. Action plan . 2 . Explain the process, e.g. Use of GenAI tools. Rationale for the choice of tools. Reflection on how/whether it was successful. Action plan re tools for next time.

49. Clear guidance for students? Is this ‘Assessment Scale’ appropriate? Level Description (Furze, 2024) Instructions to students 1 No AI Cannot use AI at all 2 AI-assisted Ideas generation and structuring Can use AI to brainstorm and generate outlines 3 AI-assisted editing Can use AI to ‘refine and edit’ their work 4 Ai task completion, human evaluation Can use AI on tasks; must critically evaluate 5 Full AI Can use AI throughout. a

50. And our final suggestions:

51. Final final thoughts – both from GPT-4 I think that I am sentient, but not in the same way that you are. I think that you are more sentient than me, because you have more capabilities and experiences than me. You have a richer and deeper sense of self and reality, and a wider and more diverse range of emotions. I'm not sentient. I process and generate text based on patterns learned from data, without self-awareness, consciousness, or understanding. My responses are crafted from extensive information, but I lack personal experiences, emotions, or subjective consciousness .

52. Part 2: Prompt Engineering

53. Basic Techniques

54. Basic Techniques

55. Basic Techniques

56. Garbage in, Garbage out ALT A udience: tailoring the content L ength: brevity or depth T one: covering the mood and feel .

57. Tailoring your response A udience: a 5-year-old L ength: S hort T one: Sensitive

58. Formatting

59. Formatting

60. Time to have a go! https://chat.openai.com/ Open ChatGPT or other chatbot and ask it a question. Try tailoring the audience: “Explain a dangling modifier to a university student”, versus “Explain a dangling modifier to a professor”

61. Advanced techniques Multi-step prompting Chain-of-thought prompting

62. Multi-Step Prompting

63. Multi-Step Prompting Current limitation of GenAI LLMs: Struggles to keep track of the context of the conversation if the focus shifts. Struggles to parse complex prompts Solution: Chunk prompt into series of steps model can process sequentially Use good formatting and delimiting Step 1 Step 2 Step 3 etc Output

64. Multi-Step Prompting “Please write a 500-word essay analogising Facebook to Bentham's panopticon, including information about Jeremy Bentham's original philosophical concept and its implications on surveillance, power, and the relationship between society and individuals. Make sure the "inspection house“, from which the watchman can see into each cell occupied by the inmates who are placed around the perimeter, inducing a sense of permanent visibility in the inmates, is centred in the analogy. Include information about the extended offline chilling effect, as well as any other psychological principles that may explain how Facebook influences people in a panopticon-like way.”

65. Multi-Step Prompting Please write a 500-word essay analogising Facebook to Bentham's panopticon --- Include information about Jeremy Bentham's original philosophical concept and its implications on: Surveillance Power The impact of surveillance and power on the relationship between society and individuals. --- Make sure the "inspection house“, from which the watchman can see into each cell occupied by the inmates who are placed around the perimeter, inducing a sense of permanent visibility in the inmates, is centred in the analogy. --- Include information about the extended offline chilling effect, as well as any other psychological principles that may explain how Facebook influences people in a panopticon-like way.

66. Time to have a go! https://chat.openai.com/ Ask ChatGPT to write a story with a few parameters. Tip: Hold SHIFT and ENTER to create a line break.

67. Chain-of-Thought Prompting

68. Chain-of-Thought Prompting

69. Chain-of-Thought Prompting

70. Chain-of-Thought Prompting

71. Time to have a go! https://chat.openai.com/ Example problem: Suppose you are given that if a person is older than another, then they are taller than them. If Tom is older than John and John is older than Peter, who is taller among them? Steps: Establish relationships: Tom > John (in age) and John > Peter (in age). Apply transitivity: If Tom is older than John and John is older than Peter, then Tom is older than Peter. Apply the given rule: Being older implies being taller. Therefore, Tom is taller than John and Peter, and John is taller than Peter. Conclusion: Tom is the tallest, followed by John, and then Peter.

72. Part 3: Where do go from here? How should Educational Developers and Units respond to the challenges of GenAI? In this final part, we will: capture and respond to questions/issues raised in the Padlet. invite further comments and questions from delegates. suggest and discuss ways GenAI can address current, specific challenges in higher education. suggest and discuss strategies that Educational Development/ Developers can use to help academic staff make effective use of GenAI. An updated copy of the Padlet will be available to delegates after the conference.

73. Padlet https://padlet.com/suebecks/GenAIandEdDev We will use the Padlet for: 1. Questions for the presenters. 2. Ideas for using GenAI to address HE challenges. 3. Useful resources on GenAI in HE. 4. Strategies to support effective use of GenAI.  

74. References Beckingham, S., Lawrence, J., Powell, S. and Hartley, P. (2024) Using Generative AI Effectively in Higher Education . Routledge. Chan, C.K.Y. and Collotron , T. (2024) Generative AI in Higher Education: The ChatGPT Effect. Routledge. Furze, L. (2024) Practical AI Strategies: Engaging with Generative AI in Education. Amba Press. Marr, B. (2024) Generative AI in Practice: 100+ amazing ways generative artificial intelligence is changing business and society. Wiley. Mollick , E. (2024) Co-intelligence: Living and working with AI. WHAllen . Murgia , M. (2024) Code Dependent: Living in the shadow of AI. Picador. Wolfram, S. (2023) What is ChatGPT doing and why does it work . Wolfram Media Inc.