An Overview of Artificial Intelligence (AI) adoption in Education

mcubric 161 views 30 slides Jul 14, 2024
Slide 1
Slide 1 of 30
Slide 1
1
Slide 2
2
Slide 3
3
Slide 4
4
Slide 5
5
Slide 6
6
Slide 7
7
Slide 8
8
Slide 9
9
Slide 10
10
Slide 11
11
Slide 12
12
Slide 13
13
Slide 14
14
Slide 15
15
Slide 16
16
Slide 17
17
Slide 18
18
Slide 19
19
Slide 20
20
Slide 21
21
Slide 22
22
Slide 23
23
Slide 24
24
Slide 25
25
Slide 26
26
Slide 27
27
Slide 28
28
Slide 29
29
Slide 30
30

About This Presentation

Presented at ECEL 2020 conference, Berlin, 29/10/20 → 30/10/20

Abstract: According to the latest AI index report*, the number of AI research papers has grown more than sevenfold in the last 20 years, reaching over 600,000 (Scopus indexed) publications in 2020. To transfer the results of this mass...


Slide Content

An Overview of AI Adoption in Education
An Overview of AI Adoption in Education
Marija Cubric, University of Hertfordshire &
ValentinaNejkovic, University of Niš
ECEL 2020, 29th October 2020
29/10/2020 M.Cubric & V.Nejkovic1

Presentation
Outline
Background
Methodology
Findings
Recommendations
Conclusions
29/10/2020M.Cubric & V.Nejkovic 2

AI Definitions
–Baker and Smith (2019) provide a broad definition of AI:
“Computers which perform cognitive tasks, usually associated
with human minds, particularly learning and problem-solving”
–Artificial Intelligence (AI) is a general term used to describe
computing technologies, which perform functions that aim to
reproduce or surpass the capabilities of human mind such as
reasoning, learning, adaptation, sensory cognition, creativity,
planning, optimisation, prediction and knowledge extraction from
diverse data.
29/10/2020M.Cubric & V.Nejkovic 3

Tertiary study
(“umbrella study” or
“review of reviews”)Primary studies
Secondary studies
Tertiary studiesTS
S1
P11P12...
S2
P21...
...
29/10/2020M.Cubric & V.Nejkovic 4

Motivation
29/10/2020 M.Cubric & V.Nejkovic5
600K+ academic papers in the area of AI (Shoham
et al. 2019)
Increased number of systematic literature reviews
(SLRs) on AI in different domains
No attempts to review the results of these SLRs in
a systematic way
No SLRs that bring together different AI
technologies, educational levels …

The aim
To review the main findings existing systematic
literature reviews on AI adoption in education and to outline a roadmap for further research in this area.
29/10/2020M.Cubric & V.Nejkovic 6

Methodology
Systematic literature review (SLR) is a type of a literature review that
follows a specific review protocol and quality procedures to select
relevant (primary) studies, extract and analyse the relevant information
from the studies to answer specific research questions.
A tertiary study compiles the evidence from other SLRs, using them as
primary studies for further analysis. This type of review is also known as
‘umbrella study’, ‘overview of systematic reviews’, ‘systematic review of
systematic reviews’ and ‘meta-review’.
This study is based on the guidelines from Kitchenhamand Charters
(2004), Budgens et al (2015) and other examples of tertiary studies such
as Kitchenhamet al. (2010), and Hodaet al. (2017).
29/10/2020M.Cubric & V.Nejkovic 7

Research
Questions
How many SLRs on AI in EDU were published since the re-birth of AI (2000) to date (2020)?
What is the quality of the SLRs in AIEDU?
What research areas are being addressed in the SLRs on AI in EDU?
Which individuals, organizations, publicationsand countriesare most active in SLR-
based research in AIEDU?
What are characteristics of the SLRs on AIEDU? E.g. number of primary studies and
years covered, data analysis methods and tools used
Whatbenefits andchallenges for AI adoption in EDU are found in the SLRs?
What recommendations the SLRS make forpractice andresearch in AIEDU?
29/10/2020M.Cubric & V.Nejkovic 8
RQ1
RQ2
RQ3
RQ4
RQ5
RQ6
RQ7

Search process
–Time of search: July 2020
–Databases:
vScopus
vWeb of Science
vIEEE and
vEBSCO databases
vSearchareas: title, abstract,
keywords
vPublication year: 2000+
vLanguage: English
29/10/2020 9
("systematic review" OR "systematic literature
review" OR "systematic map" OR "scoping review" OR "scoping
study" OR "mapping study" OR "mapping
review" OR "meta*analysis")
AND
("artificial intelligence"OR"machine learning"OR"neural
network"OR"ANN"OR"natural language
processing"OR"NLP"OR"*supervised learning"OR"deep
learning"OR"expert system"OR"data mining"OR“robot* OR
SAR" OR “chat*bot” OR"intelligent agent" OR "conversational
agent" OR "automated tutor" OR "learning analytics" OR "adapt*
testing" OR "adapt* system" OR "automat* feedback" OR
"automat*grad*" OR "automat*scor*" OR recommender OR “recommend*
system")
AND
(“education”OR college* OR undergrad* OR graduate OR
postgrad*OR “university”OR“school”OR“teach*” OR
“classroom”OR“training”OR “work*based learning” OR “lifelong
learning”)

Selection
criteria
Published after
01/01/2000.
Peer-reviewed publications
Language: English
AI, EDM, Robot-tutors,
Recommenders …
Secondary research
EDU: assessment,
feedback, personalisation
29/10/2020M.Cubric & V.Nejkovic 10
Only abstract available
Non-systematic literature
reviews
Repeated entries
False positives
Robotic/AI education
Building robots

Selection
process
29/10/2020M.Cubric & V.Nejkovic 11
1859 records identified
through search of electronic
databases
SCOPUS 1159
WoS 602
EBSCO 32
IEEE 66
355 duplicates removed
from electronic databases
1504
abstracts
and titles
screened
1446 excluded papers
applying exclusion
criteria
1036 are not EDU papers
14 are not SLR papers
83 are not AI papers
313: AI is not a primary
focus
58 potential includes 1 could not be retrieved
33 full papers
retrieved and
screened
24 are excluded after full
paper reading

Data
extraction
Bibliographic information such as: 1) citation,2) first author
affiliation,3) title, 4) abstract, 5) publication year,6)
authorkeywords, 7) indexkeywords, 8) publication type, 9)
source title.
SLR quality related information: 1)type of review, 2)number
of primary studies, 3)online databases, 4)years covered,
5)SLR guidelines, 6)search string (only AI related portion),
7)data analysis method, 8)data analysis tool
Research questions related information: 1)research
questions, 2) type of AI 3) EDU level 4) EDU discipline, 5) EDU
area 6) significant findings, 7)benefits of AI adoption, 8)
challenges of AI adoption, and 9)recommendations.
29/10/2020M.Cubric & V.Nejkovic 12
Data analysis: Descriptive Stats + Thematic analysis(CCM) performed on data extracted from the papers

Findings
29/10/2020M.Cubric & V.Nejkovic 13

RQ1:
Selected SLRs
published between
2014 and 2020 &
78.79% in 2019 and
2020
29/10/2020M.Cubric & V.Nejkovic 14
024681012141618
2014
2017
2016
2018
2020
2019
Number of SLRs per year

RQ2: Quality
assessment
Average score
2.65 (66.25%)
29/10/2020M.Cubric & V.Nejkovic 15
Quality criteria* :
Q1Are the review’s inclusion and exclusion criteria described and
appropriate?
Q2Is the literature search likely to have covered all relevant
studies?
Q3Did the reviewers assess the quality/validity of the included
studies?
Q4Were the basic data/studies adequately described?
Scoring: 2 (yes), 1(partially), 0 (no)
02468101214
1
2
2.5
3
3.5
4
Distribution of the quality score
*York University, Centre for Reviews and Dissemination (CDR) Database of
Abstracts of Reviews of Effects (DARE) criteria quoted inKithchenamet al. 2010

RQ4: Individuals,
organizations,
publications and
countries are
most active in
SLR-based
research in
AIEDU
29/10/2020M.Cubric & V.Nejkovic 16
Brazil*Ecuador
United States
Australia
UK
Spain
South Africa
China
Countries most active in SLR-based research in AIEDU
*sTAR (State of the Art through Systematic Review)) , an open source free
Computing Department of the Federal University of São Carlos in Brasil.

RQ3: Research
questions and AI
types covered in
the SLRs on AIED:
Focus on applications and
techniques RQs (50.48%)
and
Data Mining and Machine
Learning applications in
EDU (57.58%)
29/10/2020M.Cubric & V.Nejkovic 17
05101520253035
Stakeholders perspectives
Accessibility/Inclusivity
Practical guidelines
Evolution of publications
Ethical implications/concerns
Gaps/Open issues
Development of protocol/framework/solutions
Evaluation methods
Pedagogy
Challenges/Limitations/Risks/Concerns
Benefits/Opportunities
Quality
Impact
Techniques/Algorithms/Tools/Methods
Applications
Research Questions (RQ)
02468101214
Natural Language Processing (NLP)
AI (general)
Recommendation Systems
Educational Robotics
Machine Learning (ML)
Educational Data Mining
AI Types

RQ3:
Educational
applications, levels
and disciplines
covered in the
SLRs on AIEDU
29/10/2020M.Cubric & V.Nejkovic 18
012345678910
Serious games
Evalutaion of online learning resources
Assessment and evaluation
Educational Data Mining
Not specified
E-learning Recommender Systems
Adaptive systems and personalisation
Educational robotics
Profiling and prediction
EDU applications
05101520
Pre-tertiary education
Tertiary education
Not specified
EDU level
051015202530
Languages
STEM
Not specified
EDU Subject Area

RQ5: Review
protocolsadopt
ed in theSLRs
on AIEDU
–Number of primary studies
–Range: 8-256
–Median=38.5
–Years covered
–IQR: 2003-2018 :
–Median: 2006
–Data analysis method :
–Not reported (n=19. 57.56%)
–Qualitative (n=10, 30.37%)
–Quantitative (n=2, 6.06%)
–Mixed (n=1, 3.03%)
–Most cited protocols :
–Kitchenham and Charters, (2007)
–Kitchenham, et al. (2009),
–Moher et al. (2009).
29/10/2020M.Cubric & V.Nejkovic 19

RQ6.1: Benefitsof AI adoption in EDU
(not reported n=13,39.39%)
29/10/2020M.Cubric & V.Nejkovic 20
EFFICIEN
CY
PROVEN
METHOD
S
INCREASED
ACCURACY
RETENTION
TEAC
HING
PROC
ESSSATISFACTION
PROFILING
PREDICTION
Institutional Benefits
ASSESSMENT&FEE
DBACK
MOTIVATION
ENGAGE
MENT
FAIRNESS
PERFORMAN
CE
PERSONALIS
ATION
WELL-
BEING
MISC
ADAP
TATIO
NITS
LEARNING
PROCESS
L&T BEnefits

RQ6.1:
Benefitsof AI
adoption in
EDU
29/10/2020 M.Cubric & V.Nejkovic21
Learning & Teaching
•ASSESSMENT&FEEDBACK, MOTIVATION,
ENGAGEMENT, FAIRNESS, PERFORMANCE,
PERSONALISATION, WELL-BEING, ADAPTATION,
ITS, LEARNING PROCESS
Institutional
•PREDICTION, PROFILING, SATISFACTION, TEACHING
PROCESS, RETENTION, EFFICIENCY, ACCURACY
Not reported
•N=13 (39.39%)

RQ6.2: Challengesof AI adoption in EDU
(not reported N=12, 36.36%)
29/10/2020M.Cubric & V.Nejkovic 22
LEARNIN
G
PROCE…
NEGATIVE
FEELINGS
ASSESSM
ENT
NO
EFFECTS
ENGAGE
MENT
STAFF
WORKLO
AD
PEDAGOGYACCESS
RECOM
MENDAT
IONS
LEARNIN
G GAINS
SCOPE
COMP
LEXITY
SKILLSCOST
L&T Challenges
ETHICSDATA
QUALITY
PROFI
LING
EXPLANATI
ONS
EVAL
UA…
DES…
RES…
OVE
RF…
PED
A…
GENERA…
Research Challenges

RQ6.2:
Challengesof
AI adoption in
EDU
29/10/2020M.Cubric & V.Nejkovic 23
Research
•ETHICS, DATA, QUALITY, PROFILING, EXPLANATIONS,
EVALUATION, DESIGN , RESEARCH METHODS, OVERFITTING,
PEDAGOGY, GENERALISIBILITY
Learning & Teaching
•COST, ASSESSMENT, STAFF WORKLOAD, SCOPE,
COMPLEXITY , NEGATIVE FEELINGS, NO EFFECTS,
PEDAGOGY, RECOMMENDATIONS, ENGAGEMENT, SKILLS,
LEARNING GAINS, ACCESS, LEARNING PROCESSES
Not reported
•N=12 (36.36%)

RQ7: Recommendations
(Not included n=5, 15.15%)
M.Cubric & V.Nejkovic 29/10/202024
NEW APPLICATIONS
PERSONALISATION
AI TRAININGSRL
INCLUSII
VITY
GBL
IMPLI
CIT …
STAN
DA…
SCAL
ABI…ITS
DECIS
ION …
FLEXI
BILI…
ETCHICS
TRANSPARE
NCY
PARTICIPAT
ORY
DEVELOP…
PRACTICE
DATA
EMPIRICAL
EVIDENCE
TECHNOLOGY
ACCEPTANCE
TECHNIQUES
RESEACRH
DESIGN
LEARNER
MODEL
S/H
PERCEPTION
S
SOCIAL and
ETHICAL
CONSIDERA…
EVALUATIO
N
DISCIP
LINARY
DIVE…
CRITI
CALLI
TY
RESEARCH

Recommendations,
Limitations and
Conclusions
29/10/2020M.Cubric & V.Nejkovic 25
–More focus on
–pedagogy& social implications of AI ED
–assessment and feedback applications
–specific educational levelsand subject areas
–protocol compliance
–reporting of benefits, challenges, practice as well as research recommendations
–Limitations–overlap in primary studies related to the same topic
–quality of the original primary studies considered in the SLRs
–the research questions not addressed in the SLRs
–lack of other similar studies makes it difficult to assess the progress made in the field
–Conclusions
–Despite the limitation of the research, this study provides a timely identification and categorisation of some important findings from 33 SLRs (directly) and 1489 primary studies (indirectly), on the AI in EDU and it helps in raising awareness on the potential benefits and challenges associated with the use of AI in Education

References
–Budgen, D., Brereton, P., Drummond, S., & Williams, N. (2018). Reporting systematic
reviews: Some lessons from a tertiary study. Information and Software Technology, 95, 62-
74.
–Kitchenham, B., Brereton, O. P., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009).
Systematic literature reviews in software engineering–a systematic literature review.
Information and software technology, 51(1), 7-15.
–Kitchenham, B.A. & Charters, S. (2007) Procedures for Performing Systematic Literature
Review in Software Engineering, EBSE Technical Reportversion 2.3, EBSE-2007-01,
Software Eng. Group.
29/10/2020M.Cubric & V.Nejkovic 26

List of primary
sources
–Almeida, T.O.,Jose Francisco De Netto, M. (2019). Adaptation content in robotic systems: A systematic mapping study,
Proceedings - Frontiers in Education Conference, 4 March 2019, Article number865853948th
–Torrezao da Costa, N., Pereira Junior, C. X., Dias Araujo, R. and Aparecida Fernandes, M. (2019). "Application of AI Planning
in the Context of e-Learning,"2019 IEEE 19th International Conference on Advanced Learning Technologies (ICALT), Maceió,
Brazil, 2019, pp. 57-59, doi: 10.1109/ICALT.2019.00021.
–Araka, E., Maina, E., Gitonga, R.et al.(2020). Research trends in measurement and intervention tools for self-regulated
learning for e-learning environments—systematic review (2008–2018).RPTEL15,6.
–Zawacki-Richter, O., Marín, V.I., Bond, M.et al.(2019). Systematic review of research on artificial intelligence applications
in higher education – where are the educators?.Int J Educ Technol High Educ16,39.
–Kai Siang Chan, Nabil Zary (2019). Applications and Challenges of Implementing Artificial Intelligence in Medical
Education: Integrative Review, JMIR Med Educ. 2019 Jun 15;5(1):e13930. etal
–Alexander Winkler-Schwartzetal. (2019). Artificial Intelligence in Medical Education: Best Practices Using Machine
Learning to Assess Surgical Expertise in Virtual Reality Simulation, J Surg Educ. Nov-Dec 2019;76(6):1681-1690.
–Xiaoming Zhai, Yue Yin, James W. Pellegrino, Kevin C. Haudek & Lehong Shi(2020).Applying machine learning in science
assessment: a systematic review,Studies in Science Education,56:1,111-151
–Tamada, M.M, de Magalhães Netto, J.F. and de Lima, D.P.R. (2019). "Predicting and Reducing Dropout in Virtual Learning
using Machine Learning Techniques: A Systematic Review,"2019 IEEE Frontiers in Education Conference (FIE), Covington,
KY, USA, 2019, pp. 1-9, doi: 10.1109/FIE43999.2019.9028545.
–Chary M, Parikh S, Manini AF, Boyer EW, Radeos M. A (2018). Review of Natural Language Processing in Medical
Education. West J Emerg Med. 2019 Jan;20(1):78-86. doi: 10.5811/westjem.2018.11.39725. Epub 2018 Dec 12. PMID:
30643605; PMCID: PMC6324711.
–Tarus J.K., Niu Z. (2017) A Survey of Learner and Researcher Related Challenges in E-learning Recommender Systems. In:
Uden L., Liberona D., Liu Y. (eds) Learning Technology for Education Challenges. LTEC 2017. Communications in Computer
and Information Science, vol 734. Springer, Cham.
29/10/2020M.Cubric & V.Nejkovic 27

List of primary
sources
–Rivera A.C., Tapia-Leon M., Lujan-Mora S. (2018). Recommendation Systems in Education: A Systematic Mapping Study.
In: Rocha Á., Guarda T. (eds) Proceedings of the International Conference on Information Technology & Systems (ICITS
2018). ICITS 2018. Advances in Intelligent Systems and Computing, vol 721. Springer, Cham.
–Pinho, P.C.R. et al. (2019). "Developments in Educational Recommendation Systems: a systematic review," 2019 IEEE
Frontiers in Education Conference (FIE), Covington, KY, USA, 2019, pp. 1-7, doi: 10.1109/FIE43999.2019.9028466.
–Murad,D.F., Heryadi, Y., Wijanarko,B.D., Isa, S.M. and Budiharto,W. (2018). "Recommendation System for Smart LMS
Using Machine Learning: A Literature Review,"2018 International Conference on Computing, Engineering, and Design
(ICCED), Bangkok, Thailand, 2018, pp. 113-118, doi: 10.1109/ICCED.2018.00031
–Ravyse, W.S., Seugnet Blignaut, A., Leendertz, V.et al.(2017). Success factors for serious games to enhance learning: a
systematic review.Virtual Reality21,31–58.
–Alenezi, H.S.,Faisal, M.H. (2020). Utilizing crowdsourcing and machine learning in education: Literature review,
Education and Information TechnologiesVolume 25, Issue 4, 1 July 2020, Pages 2971-2986
–Smakman M., Konijn E.A. (2020). Robot Tutors: Welcome or Ethically Questionable?. In: Merdan M., Lepuschitz W.,
Koppensteiner G., Balogh R., Obdržálek D. (eds) Robotics in Education. RiE 2019. Advances in Intelligent Systems and
Computing, vol 1023. Springer, Cham. https://doi.org/10.1007/978-3-030-26945- 34
–Papadopoulos,I., Lazzarino, R., Miah, S., Weaver, T., Thomas, B., Koulouglioti, C. (2020). A systematic review of the
literature regarding socially assistive robots in pre-tertiary education, Computers & Education, Volume 155, 2020, 103924
– Hein, M., & Nathan-Roberts, D. (2018). Socially Interactive Robots Can Teach Young Students Language Skills; a
Systematic Review. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 62(1), 1083–1087.
https://doi.org/10.1177/1541931218621249
29/10/2020M.Cubric & V.Nejkovic 28

List of primary
sources
–Lai Poh Emily Toh, Albert Causo, Pei-Wen Tzuo, I-Ming Chen, & Song Huat Yeo. (2016). A Review on the Use of Robots in
Education and Young Children.Journal of Educational Technology & Society,19(2), 148-163.
–Mite-Baidal K., Delgado-Vera C., Solís-Avilés E., Espinoza A.H., Ortiz-Zambrano J., Varela-Tapia E. (2018) Sentiment
Analysis in Education Domain: A Systematic Literature Review. In: Valencia-García R., Alcaraz-Mármol G., Del Cioppo-
Morstadt J., Vera-Lucio N., Bucaram-Leverone M. (eds) Technologies and Innovation. CITI 2018. Communications in
Computer and Information Science, vol 883. Springer, Cham.
–S. M. Muthukrishnan, N. B. M. Yasin and M. Govindasamy (2018). "Big data framework for students' academic
performance prediction: A systematic literature review,"2018 IEEE Symposium on Computer Applications & Industrial
Electronics (ISCAIE), Penang, 2018, pp. 376-382.
–David Otoo-Arthur, Terence Van Zyl, Terence Van Zyl (2019). A Systematic Review on Big Data Analytics Frameworks for
Higher Education - Tools and Algorithms. Proceedings of the 2019 2nd International Conference on E-Business,
Information Management and Computer Science, August 2019,Article No.: 15Pages 1–9
–Abu Saa, A., Al-Emran, M. & Shaalan, K. (2019). Factors Affecting Students’ Performance in Higher Education: A
Systematic Review of Predictive Data Mining Techniques.Tech Know Learn24,567–598Shin, D., Shim, J. A Systematic
Review on Data Mining for Mathematics and Science Education.Int J of Sci and Math Educ(2020).
–Cristina Alonso-Fernández, Antonio Calvo-Morata, Manuel Freire, Iván Martínez-Ortiz, Baltasar Fernández-Manjón (2019).
Applications of data science to game learning analytics data: A systematic literature review, Computers & Education,
Volume 141, 2019, 103612.
–Du, X.,Yang, J.,Hung, J.-L.andShelton, B.(2020), "Educational data mining: a systematic review of research and
emerging trends",Information Discovery and Delivery, Vol. 48 No. 4, pp. 225-236.
29/10/2020M.Cubric & V.Nejkovic 29

List of primary
sources
–Abid,A., Kallel, I. and Ben Ayed,M. (2016). "Teamwork construction in E-learning system: A systematic literature review,"
2016 15th International Conference on Information Technology Based Higher Education and Training (ITHET), Istanbul,
2016, pp. 1-7, doi: 10.1109/ITHET.2016.7760756.
–Hellas, A. etal. (2018). Predicting academic performance: a systematic literature review, ITiCSE 2018 Companion:
Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science
Education, July 2018, Pages 175–199
–Agrusti, F., Bonavolontà, G., & Mezzini, M. (2019). University Dropout Prediction through Educational Data Mining
Techniques: A Systematic Review.Journal of E-Learning and Knowledge Society,15(3), 161-182.
–Liz-Domínguez, M.; Caeiro-Rodríguez, M.; Llamas-Nistal, M.; Mikic-Fonte, F.A. (2019). Systematic Literature Review of
Predictive Analysis Tools in Higher Education.Appl. Sci.,9, 5569.
–Salal, Yass & Kumar, Mukesh. (2019). Systematic Review of Predicting Student's Performance in Academics.
–Hernández-Blanco, A., Herrera-Flores, B., Tomás,D., Navarro-Colorado,B. (2019). "A Systematic Review of Deep Learning
Approaches to Educational Data Mining",Complexity,vol.2019,Article ID1306039.
–Papamitsiou, Z., Economides, A.A. (2014). Learning Analytics and Educational Data Mining in Practice: A Systematic
Literature Review of Empirical Evidence, Educational Technology & Society, Vol. 17, No. 4, Review Articles in Educational
Technology (October 2014), pp. 49-64
29/10/2020M.Cubric & V.Nejkovic 30