effectiveness of enhanced question-based tutoring

effectiveness of enhanced question-based tutoring

1. Effectiveness of the Technology-enhanced Question-based Tutoring in the Context of IT Learning Qualifying Examination Przemyslaw ( Pshemek ) Pawluk February 28, 2024 Advisory committee Supervisor: Judi McCuaig Co-supervisor: Agnieszka (Aga) Palalas Members: Shoshanah Jacobs Denis Nikitenko

2. Agenda Motivation Computational Thinking and Problem Solving Teaching Actions and Behaviours Tutoring Proposal Tutoring Behaviours Taxonomy Augmentations Work status and Plans Summary 2 P.Pawluk

3. A teacher is never a giver of truth – he is a guide, a pointer to the truth that each student must find for himself. ⎯ Bruce Lee ⎯ 3 P.Pawluk

4. Motivation 4 P.Pawluk

5. Problem description Role of Computational Thinking (CT) and Problem-Solving Skills (PSS) Understanding a problem and defining an algorithmic solutions Necessary skills in IT and in industry overall Large classes are not an ideal environment for development of CT and PSS [1,2] 5 P.Pawluk

6. Teaching CT and PSS Classroom teaching limitations Large student/teacher ratio limits teachers’ ability to effectively teach CT and PSS in personalized way. Tutoring CT & PSS Providing personalized learning More time for learner exploration and active learning Tutor’s role is to guide Many tutors lack training and experience. [1,7] 6 P.Pawluk

7. Peer tutors Peer tutors lack experience as educators and are not experts in the domain. Limited training is provided to them by institutions that hire tutors. Lack of experience and training impacts tutors’ effectiveness and repertoire of teaching actions they can offer. [16, 25] P.Pawluk 7

8. Guides vs. Tellers Guides Helps learner to identify objectives. Use behaviours and actions helping learner to achieve their objectives. Avoids giving readymade answers and encourages active learning. [3, 4, 7, 8] Tellers Assume goals and impose them on learner. Focus more on the content than on learner. Often give out solutions and readymade answers. [3, 5, 6] P.Pawluk 8

9. Research Statement Technology-augmented, question-based tutoring will increase the rate of guiding behaviours shown by tutors without compromising the effectiveness of the tutoring. 9 P.Pawluk

10. Research Questions Do tutors exhibit more guiding behaviours when using the technology-augmentation than they do without? Does question-based tutoring change the perceptions and reactions of tutors and tutees to the tutoring session? 10 P.Pawluk

11. Just-in-time Support for Tutors Providing tutors with just-in-time support that would them to generate ideas such as: Questions to ask Analogies to use Action plans Utilize the power of generative AI models 11 P.Pawluk

12. Teaching Actions & Behaviours 12 P.Pawluk

13. Teaching Behaviours Teaching behaviour encompasses the overall conduct, demeanour, and professional attitude of an educator. Teaching Behaviours are aspirational and describe the intent of the educator. Teaching actions on the other hand, refer to measurable, specific, deliberate steps or strategies taken by educators. P.Pawluk 13

14. Teaching Actions Teaching action refers to a specific, deliberate steps or strategies taken by educators to facilitate learning. They are planned and are a part of instructional methods or pedagogical approaches. They include activities like lecturing, guiding discussions, providing feedback, and designing learning activities. Actions are often directly related to instructional goals and are chosen based on their effectiveness in achieving desired educational outcomes. [15] 14 P.Pawluk

15. Tutoring 15 P.Pawluk

16. Tutoring Tutoring refers to the act of giving private educational instruction or guidance to an individual or a small group. Often happens outside of the standard classroom setting and is tailored to the specific needs of the individual or group. Tutors can help with a wide range of academic subjects, test preparation, and skill development. 16 P.Pawluk

17. P.Pawluk 17 Classroom Teaching Tutoring

18. Types of Tutors and Tutoring P.Pawluk 18

19. Types of Tutors and Tutoring P.Pawluk 19

20. Tutoring Tutoring is a vital element of the higher education Tutoring is commonly used to support learners outside of the classroom setting Role of the tutor Providing personalized learning Providing ad hoc help Building relationship with learner 20 P.Pawluk

21. Benefits of Tutoring Students who received tutoring performed better in their classes than those who did not receive tutoring.[9] Tutors can also help students develop better study habits and strategies that can enhance their overall academic success.[26] Tutoring allows for the pace of learning to be adjusted to suit the student [9]. 21 P.Pawluk

22. Proposal 22 P.Pawluk

23. Research Statement Technology-augmented, question-based tutoring will increase the rate of guiding behaviours shown by tutors without compromising the effectiveness of the tutoring. 23 P.Pawluk

24. Research Questions Do tutors exhibit more guiding behaviours when using the technology-augmentation than they do without? Does question-based tutoring change the perceptions and reactions of tutors and tutees to the tutoring session? 24 P.Pawluk

25. Research Plan P.Pawluk 25

26. Literature review P.Pawluk 26

27. Bloom’s Taxonomy Existing Models and Taxonomies P.Pawluk 27 Revised Bloom’s Taxonomy Flander’s Interactions Analysis Joyce & Weil’s Taxonomy Danielson’s Framework SOLO Taxonomy Fink’s Taxonomy Joyce’s Models of Teaching Kounin’s Model Ohlson’s Repertoire of Teaching Actions

28. Research contribution: Tutoring Behaviours Taxonomy (TBT) New taxonomy that focusses on tutoring and tutor’s behaviours and actions. Taxonomy unifies behaviours focussed on teaching (including feedback and engagement building) with behaviours that are social and organizational. 28 P.Pawluk

29. Tutoring Behaviours Taxonomy P.Pawluk 29

30. Telling and Guiding Action in TBT P.Pawluk 30

31. Experiments P.Pawluk 31

32. Exploratory experiment Goal: to form a base-line for further experimentation and test the taxonomy Population: Students (tutors and learners) from GBC School of Computer Technology who use MobiHelp tutoring services Data collected: transcripts of tutoring sessions and questioners When: Summer 2024 P.Pawluk 32

33. Main experiment Goal: to test the augmentations and their impact on tutoring Population: Students (tutors and learners) from GBC School of Computer Technology who use MobiHelp tutoring services Data collected: transcripts of tutoring sessions and questioners When: Fall 2024 / Winter 2025 P.Pawluk 33

34. Augmentations P.Pawluk 34

35. Research contribution: Augmentations Question Generator: Summarizing the problem and generating questions that may be used to elicit the actionable plan by the tutee. Analogy Generator: Explaining the problem with analogy or example generated based on the problem description. Action Plan Generator: Defining actionable steps that can be used twofold, to identify the gaps in understanding and to direct the learner’s actions. P.Pawluk 35

36. Work status and Plans Task Status Analysis of the tutoring process from the tutor’s perspective Partially done Experiments Planned, REB done, To be completed in Summer, Fall 2024 and Winter 2025 New taxonomy of tutoring behaviours First draft completed, to be revised in Fall 2024 Data analysis methods To be completed by Fall 2024 Augmentations for the tutor Work started, To be completed by Fall 2024 36 P.Pawluk

37. Thank you! Contact: Przemyslaw Pawluk [email protected] P.Pawluk 37

38. Bibliography Aho , Alfred V. (January 2011).  "Computation and computational thinking" . Ubiquity . 2011 (January). doi : 10.1145/1922681.1922682 Granvold , Donald K. (1997). "Cognitive-Behavioral Therapy with Adults". In Brandell , Jerrold R. (ed.).  Theory and Practice in Clinical Social Work . Simon and Schuster. pp.  189 .  ISBN   978-0-684-82765-0 . A. King, “From sage on the stage to guide on the side,” College teaching, vol. 41, no. 1, pp. 30–35, 1993. J. B. Biggs and K. F. Collis, Evaluating the quality of learning: The SOLO taxonomy (Structure of the Observed Learning Outcome). Academic Press, 2014 . D. Bligh, “What’s the use of lectures?” Journal of Geography in Higher Education, vol. 9, no. 1, pp. 105–106, 1985. M. Prince, “Does active learning work? a review of the research,” Journal of engineering education, vol. 93, no. 3, pp. 223–231, 2004. C. Bonwell and J. A. Eison , “Creating excitement in the classroom,” Washington, DC: George Washington University, School of Education and Human Development, 1991. L. W. Anderson, D. R. Krathwohl, P. W. Airasian , K. A. Cruikshank, R. E. Mayer, P. R. Pintrich , J. Raths, and M. C. Wittrock, “A taxonomy for learning, teaching, and assessing: A revision of bloom’s taxonomy of educational objectives, abridged edition- esri ,” 2023. 38 P.Pawluk

39. Bibliography P. A. Cohen, J. A. Kulik, and C.-L. C. Kulik, “Educational outcomes of tutoring: A meta-analysis of findings,” American educational research journal, vol. 19, no. 2, pp. 237–248, 1982. K. VanLehn , “The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems,” Educational psychologist, vol. 46, no. 4, pp. 197–221, 2011. G. Belgiovine , F. Rea, J. Zenzeri , and A. Sciutti , “Towards effective robot tutoring for skills acquisition.” K. Kechagias , “Teaching and assessing soft skills,” 2011. M. R. Lepper, M. F. Drake, and T. O’Donnell-Johnson, “Scaffolding techniques of expert human tutors.” 1997. R. D. Roscoe and M. T. Chi, “Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions,” Review of educational research, vol. 77, no. 4, pp. 534–574, 2007. L. Shulman, “Knowledge and teaching: Foundations of the new reform,” Harvard educational review, vol. 57, no. 1, pp. 1–23, 1987 39 P.Pawluk

40. Bibliography B. S. Bloom, “The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring,” Educational researcher, vol. 13, no. 6, pp. 4–16, 1984 B. S. Bloom, D. R. Krathwohl, B. B. Masia et al., “Bloom taxonomy of educational objectives,” in Allyn and Bacon. Pearson Education London, 1984 N. A. Flanders, “Analyzing teaching behavior.” 1970 J. S. Kounin , “Discipline and group management in classrooms.” 1970 C. Danielson, The framework for teaching evaluation instrument., 2013. J. B. Biggs and K. F. Collis, Evaluating the quality of learning: The SOLO taxonomy (Structure of the Observed Learning Outcome). Academic Press, 2014. L. D. Fink, “The power of course design to increase student engagement and learning,” Peer Review, vol. 9, no. 1, pp. 13–17, 2007 B. R. Joyce, M. Weil, and E. Calhoun, Models of teaching, 7th ed. Boston: Pearson, Allyn and Bacon, 2004. S. Ohlsson, “Some principles of intelligent tutoring,” Artificial Intelligence and Education: Learning environments and tutoring systems, vol. 1, p. 203, 1987. 40 P.Pawluk

41. Bibliography M. T. Chi, S. A. Siler, H. Jeong , T. Yamauchi, and R. G. Hausmann, “Learning from human tutoring,” Cognitive science, vol. 25, no. 4, pp. 471–533, 2001. B. P. Woolf, Building intelligent interactive tutors: Student-centered strategies for revolutionizing e-learning. Morgan Kaufmann, 2010. J. Hattie, Visible learning: A synthesis of over 800 meta-analyses relating to achievement. routledge , 2008 41 P.Pawluk