Open and Critical Perspectives on AI in Education

Presentation made at the 24th International Conference on Education Research (ICER) at Seoul National University, South Korea.

1. Open and Critical Perspectives on
AI in Education
The 24th International
Conference on Education
Research (ICER)
11-12 July 2024
DR. ROBERT FARROW
INSTITUTE OF EDUCATIONAL TECHNOLGY
THE OPEN UNIVERSITY (UK)
Artwork: Visual Thinkery CC BY

2. About Me

3. 3
Philosopher & Educational
Technologist
Senior Research Fellow
Research Programme LeadLearning in an Open World
Co-DirectorGlobal OER Graduate Network
Co-Editor, Journal of Interactive Media in Education
Mentor for Asia-Europe Foundation,
UKRI Future Leaders, UNESCO, SPARC
http://philosopher1978.wordpress.com/
BA (Hons) MA PhD Pg.CHEPSFHEA MAODEDr.Robert Farrow

4. The Global OER Graduate Network supports
researchers in open education
The aims of the GO-GN are to:
•raise the profile of research into open education
•offer support for those conducting doctoral
research in this area
•promote equity and inclusion in the field of open
education research
•develop openness as a process of research
http://go-gn.net

6. Summary: 4 arguments
for why we need open
and critical approaches
to AI in education

7. Openness and AI in
Education

8. The LighthillDebate on Artificial Intelligence
https://www.youtube.com/watch?v=03p2CADwGF8

9. The LighthillDebate on Artificial Intelligence
https://www.youtube.com/watch?v=03p2CADwGF8
Professor John McCarthy: Let's
see. Excuse me. I invented the
term artificial intelligence. I
invented it because we had to
do something when we were
trying to get money for a
summer study in 1956, and I
had a previous bad experience.

10. We don’t need AI… but

11. 11
Personalised Instruction Intelligent Tutoring SystemsGames & Simulations
Chatbot MentoringAdaptive Learning Automated Assessment

12. 12
https://www.iesalc.unesco.org/wp
-
content/uploads/2023/04/ChatGPT
-and
-
Artificial
-Intelligence
-in-higher
-education
-
Quick
-Start
-guide_EN_FINAL.pdf

13. Open Education
Widening
Participation
Open
Access
Open
Educational Resources
Open
Educational Practices
Open
SourceMOOCs
Equity,
Diversity, Inclusion
Social
Justice
De-
colonisation
Critical
Pedagogy

15. https://www.nytimes.com/2024/05/17/business/what-is-openwashing-ai.html

16. •No definitions for ‘open’ AI exist which leaves interpretative
space
•Many want to keep AI ‘closed’ and non-transparent for
reasons of commercial interest and/or safety but keep the
‘open’ branding and its positive connotation
•AI Systems are not democratic as barriers to entry are
enormous (computing power, data curation, moderation) even
if the source code is ‘openly licensed’
•AI systems don’t conform to our notions of open source
coding, and training data is also required for reproducibility
•Most AI systems are trained without regard for intellectual
property rights
•Algorithmic bias has been shown to a real, harmful aspect of
AI systems

17. What would an
authentically open AIED
be like?

18. 1. Socio-Technical
Perspective Over
Solutionism

19. 19

20. 20
https://twitter.com/debarghya_das/status/1636544140069711872

21. 21
AIED: OPPORTUITIES FOR AN OPEN UNIVERSITY
https://anatomyof.ai/

22. 22https://www.bloomberg.com/news/articles/2024-07-02/google-s-emissions-
shot-up-48-over-five-years-due-to-ai

23. 23
https://www.theverge.com/2024/7/3/24191405/meta-anpd-stop-training-ai-on-
brazilian-facebook-instagram-data

24. https://arstechnica.com/tech-policy/2024/07/ai-trains-on-kids-photos-even-when-
parents-use-strict-privacy-settings/

25. 25
https://www.wired.com/story/ai-tools-are-secretly-
training-on-real-childrens-faces/

26. 26
https://www.bloomberg.com/opinion/articles/2024
-04
-
03/the
-humans
-behind
-amazon
-s-just
-walk
-out
-
technology
-are
-all-over
-ai

29. Summary (1): Consider
the comprehensive
environmental and
socio-economic impacts
of AI, beyond just its
usefulness within
specific systems

30. 2. Explicability and
Transparency for Open
AIED

31. 31

32. 32

33. 33
https://www.ibm.com/topics/explainable-ai

34. 34
AI4PEOPLE ETHICAL FRAMEWORK
The AI4People initiative synthesizes 47 sets of guidelines to four traditional ethical
principles and proposes one new AI-specific principle (Floridi& Cowls, 2019).
Furthermore, appropriate governance measures need to be put in place so that it is
always possible to identify a human being who takes responsibility for what an algorithm
has done or recommended.
Greater transparency and explicability is a route to critical reflection upon the application
of algorithms in education (XAIED) and in social life more generally.
There are good arguments for making XAIED the default expectation for AIED.

35. 35
The closest thing to
openness in AIED is
‘explicability’ -but this
concept is typically
overstretched. We need
language that is not just
‘explicable’ from an
expert or technical
standpoint, but
explainable and
interpretable to a range of
stakeholders including
learners. This also
threatens to disrupt
traditional pedagogies
and introduce new forms
of expertise (Farrow,
2023).

36. 36
This typology proposed by Markus et al. (2021) distinguishes interpretability which is
human readable and fidelitywhich is the accurate, technical description of what happens
in the ‘black box’. (See also Khosraviet al., 2022)
Should we expect learners to understand these processes and the effects for their
learning?
For the general stakeholders lacking expert knowledge such transparency presumably
has limited value without a trusted broker who can interpret on their behalf.
Explainability
Fidelity (Accurate
description of tasks)
Interpretability
(Human
comprehensibility)
Clarity (rationale)Parsimony
(conciseness)
Completeness
(input-output
reporting)
Soundness (truthful
to task model)

37. 37
AIED: OPPORTUITIES FOR AN OPEN UNIVERSITY
https://doi.org/10.1080/17439884.2023.2185630

38. Summary (2):
Transparency is a
necessary but insufficient
component for open
AIED; explicability is
required but insufficiently
problematized

39. 3. Rethinking Learning
Materials and OER

42. https://news.unm.edu/news/transforming-education-at-unm-culls-launches-innovative-oer-programs-with-ai-integration-and-financial-
support

43. 43

45. 45
Currently, GPTs act to transform copyrighted content, ultimately making
it available to the public domain -there is no legal basis for copyrighting the
content that is produced by a GPT (Brittain, 2023).This means that anything
that is produced by such a machine is assumed public domain (and cannot be
openly licensed).This can be understood as an interesting subversion of
copyright, or as “the greatest art heist in history” (Centerfor Artistic Inquiry and
Reporting, 2023).

47. 47

48. 48

49. 49
Adobe

50. 50
OpenAI’sCTO refers to the
use of ‘publicly available
data’ in training ChatGPT
https://twitter.com/TechBurrit
oUno/status/1768363023192
768799

51. 51
https://www.theverge.com/2024/6/28/24188391/microsoft-ai-suleyman-social-contract-freeware

53. Summary (3): We need
to protect The
Commons and
intellectual property
rights

54. 4. Inclusive, Dialogic,
Multi-Perspectival
Research

55. 55
ARTIFICIAL INTELLIGENCE IN EDUCATION: BIAS
Algorithmic bias has been the focus of
many critiques of AI (e.g. Baker & Hawn,
2021; Birhane et al., 2022; Noble, 2018;
Samuel, 2021; Wachter, 2022; Zuboff,
2019).
CC BY NC SALeo Reynolds https://www.flickr.com/photos/lwr/2222227513
Algorithmic bias has been the
focus of many critiques of AI
(e.g. Baker & Hawn, 2021;
Birhaneet al., 2022; Noble,
2018; Samuel, 2021; Wachter,
2022; Zuboff, 2019).

56. 56
https://educationaltechnologyjournal.springeropen.com/articles/10.1186/s41239-023-00436-z
Meta-synthesis of 66 reviews:
•65% of low to medium quality –need for greater rigour
•Low/inconsistent profile of ethical considerations
•Identifies need for collaboration in development of AI applications,
curriculum, research
and evidence synthesis
•Preponderance of research in N. America
•Calls for collaboration to address language bias, provide contextual
knowledge, building context-transcendent perspectives

57. Openness IS criticality

58. Openness IS criticality
Cathedral & Bazaar (Raymond, 1999)

59. https://doi.org/10.1080/17439884.2016.1113991

60. Summary (4): Open
practices provide
authentic routes to
much needed inclusion
& critique of AIED
oversight and
development

61. Conclusion

64. 64
The EU AI Act
introduces
restrictions for
foundational AI
models but with some
exemptions for open
source models.
Transparency in AI
training is
encouraged by the
Act.
Open Source
training models have
been criticised for
lacking some of the
guardrails and
protocols of
foundational models.
Rights holders must
be notified when
materials are used for
training, and provided
with a right of
withdrawal.

65. 65

66. THANK YOU
philosopher1978.wordpress.com
[email protected]
@philosopher1978

67. 67
REFERENCES
ARL/CNI AI Scenarios: AI-Influenced Futures. Washington, DC, and West Chester, PA: Association of
Research Libraries, Coalition for Networked Information, and Stratus Inc., June 2024.
https://doi.org/10.29242/report.aiscenarios2024.
Baker, R. S., & Hawn, A. (2021, March 1). Algorithmic Bias in Education.
https://doi.org/10.35542/osf.io/pbmvz
Birhane, A., E. Ruane, T. Laurent, M. S. Brown, J. Flowers, A. Ventresque, and C. L. Dancy (2022). The
Forgotten Margins of AI Ethics. FAccT‘22: Proceedings of the 2021 ACM Conference on Fairness,
Accountability, and Transparency (Forthcoming). https://doi.org/10.1145/3531146.3533157
Bond, M., Khosravi, H., De Laat, M. et al. (2024). A meta systematic review of artificial intelligence in
higher education: a call for increased ethics, collaboration, and rigour. International Journal of
Educational Technology in Higher Education 21, 4. https://doi.org/10.1186/s41239-023-00436-z
Brittain, B. (2023). AI-generated art cannot receive copyrights, US court says. Reuters (August 21st).
https://www.reuters.com/legal/ai-generated-art-cannot-receive-copyrights-us-court-says-2023-08-21/
Centerfor Artistic Inquiry and Reporting (2023). Restrict AI Illustration from Publishing: An Open Letter.
https://artisticinquiry.org/AI-Open-Letter
Crawford, K. (2021). The Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence.
Yale University Press. doi:10.2307/j.ctv1ghv45t.
Doctorow, C. (2023). The Internet Con: How to Seize the Means of Computation. Verso.
Farrow, R. (2015). Open education and critical pedagogy. Learning, Media and Technology, 42(2), 130–
146. https://doi.org/10.1080/17439884.2016.1113991

68. 68
REFERENCES
AI AND OPEN EDUCATION
Farrow, R. (2023) The possibilities and limits of XAI in education: a socio-technical perspective.
Learning, Media and Technology, 48:2, 266-279. https://doi.org/10.1080/17439884.2023.2185630
Khosravi, H., S. Buckingham Shum, G. Chen, C. Conati, Y.-S. Tsai, J. Kay, S. Knight, R. Martinez-
Maldonado, S. Sadiq, and D. Gašević. (2022). Explainable Artificial Intelligence in Education. Computers
and Education: Artificial Intelligence 3,https://doi.org/10.1016/j.caeai.2022.100074
Noble, S. U. (2018). Algorithms of Oppression. NYU Press.
Raymond, Eric S. (1999). The Cathedral and the Bazaar: Musings on Linux and Open Source by an
Accidental Revolutionary. O'Reilly Media. ISBN 1-56592-724-9.
Samuel, S. (2021, September 18). AI’s Islamophobia problem. Vox.https://www.vox.com/future-
perfect/22672414/ai-artificial-intelligence-gpt-3-bias-muslim.
Stacey, P. (2023). AI From an Open Perspective
https://paulstacey.global/blog/ai-from-an-open-perspective
Wachter, S. (2022, February 15). The Theory of Artificial Immutability: Protecting Algorithmic Groups
under Anti-Discrimination Law. Tulane Law Review (forthcoming). https://ssrn.com/abstract=4099100,
Zuboff, S. (2019). The Age of Surveillance Capitalism. Public Affairs Books.

69. 69
Instructionby FazrianZahrawani(Noun Project)
Online Tutorby JuicyFish(Noun Project)
VRby MahmureAlp(Noun Project)
Mentorby Satria Arnata(Noun Project)
Adaptive Learningby Suprihatin(Noun Project)
Automated processby kliwir art(Noun project)
Cathedralby luca fruzza (Noun Project)
Tent by Sebastian Ilves (Noun Project)
#openaied
ADDITIONAL IMAGE CREDITS