2025-03-03-Data-related-Ethics Issues in Technologies for Professional Learning.pdf

How could modern LA research address data-related ethics issues in informal and situated professional learning? I will identify in this talk three relevant insights based on field studies around workplace LA interventions: Firstly, in informal and situated learning, data isn’t just about the learners. Secondly, the affordances of manual and automatic data tracking for learning are very different, with manual tracking allowing a high degree of learner control over data. Thirdly, learning is not necessarily a shared goal in workplaces. These can be translated into seeing a potential for systems endowed with sufficient natural-language-processing capability (now seemingly at our fingertips with LLMs), and socio-technical design and scenario-based data collection analysis as design and research methods.

1. 1
Data-relatedethicsissuesin technologiesforprofessional learning
W I S S E N
◼T E C H N I K
◼L E I D E N S C H A F T
uwww.tugraz.at
Data-relatedEthicsIssusin
Technologies forProfessional Learning
March 3, 2025 -Professional Learning Analytics: repurposing workplace data for
professional learning and development @LAK 2025
Viktoria Pammer-Schindler, Institute ofHuman-CentredComputing

2. 3
Data-relatedethicsissuesin technologiesforprofessional learning
3
HCI and EdTEch
Designinginteractivesystems
froma socio-technicaland learningperspective
Viktoria Pammer-Schindler, Institute ofHuman-CentredComputing1
5
1
1
1
2
1
3
1
4
Fesslet al. (2017) @IEEE TLT –
In-appreflectionguidance…
Disch et al. (2022) @ACM CHI. Design
Space forKnowledge Construction …
Wolfbauer et al. (2022) @IEEE
TLT. A Scriptfor
ConversationalRefl. Guidance
…
Rivera-Pelayo et al.
(2017) @ACM toCHI–
Moodtrackingin a call
center…
Barreiroset al. (2019) @Int. J. Hum-Comp.
Int. PlantingtheSeed ofPositive Human-
IoTInteraction.1
5
1
1
1
2
1
3
1
4
Pammer-Schindler & Rosé
(2021) @ijAIED. Data-
Related Ethics Issues …Initiation and Ideation
Analytical and
Feasibility
Implementation and
Execution
Generatingbusinessmodelideas.
Describethebusinessmodelideaand
showthepotentialandrelevancefor
investingtimeandresourcestofurther
elaboratetheidea.
Gate: Commitment to provide business
resources for further elaborating BM idea
Gate: Decision to implement
business model in a (sub-) market
Prototyping and
Validation
Analyticallyanalyzeandevaluatethe
businesscase.Thisstagerequires
resourcesforthebusinessmodelteam
andsupplementaryfunctions
Testingthebusinessmodelidea
(assumptions,hypothesis)through
successfulMVP prototypesand
customerinteractions.
Rolloutthebusinessmodelinatleasta
sub-marketandscalethebusiness
model.
Controllingofthebusinessmodeland
ensuringitssustainability.
▪Business Model Canvas
▪Value Proposition Canvas
▪Data Map
▪Data Product Canvas
▪Data Service Cards
▪Business Model Canvas
▪SWOT, PESTEL Analysis
▪Focus groups, customer or expert
Interviews
▪Market and competitor analysis
▪Customer surveys / MVP field tests
▪Business case calculation
▪Business Model Risk Evaluation
▪Scenario Planning / Technique
▪BM Implementation Plan /
Roadmap
▪Metrics (e.g., customer
satisfaction, market share,
ROI, margin, …)
-Fit with strategic roadmap of
company (strategic fit)
-Strategic importance
Technical gap, technicalcomplexity
Requireddatasources
-Idea evaluation criteria
-Novelty of problem
-Novelty of solution
SupportingAVL strategy, required
competencies
-Possession of core competencies
anddynamic capabilities
-Benchmark with competitors
-Alignment with market trends
-Potential customer demand
-Rough cost and revenue
estimation
-Technological Complexity
-Technological Effort
-Commitmentof stakeholders and
key partners
-Successful prototyping and
customer interaction, -Results from
customer surveys or field tests
-Viable Business Case Calculation
(ROI,Financial Plan)
-Technical proof of concept
Tools
Criteria
Information
from tools
inform the
decision
Goals
Gate: Decision to test business
model sketch with a PoC prototype
Idea Description
▪Decide in what direction you want
to go with a data-driven innovation
▪Analyze your current situation (data
sources, customer pains, ..)
▪Describe your business model ideaActivities
Business Model Design &
Evaluated customer demand
MVP Implemented business model
Outcome
▪Implement and rollout the business
model
▪Continuously monitor the success
of the business model
▪Continuously adopt the business
model to a changing environment
▪Check technical feasibility
▪develop a minimum viable product
▪Create a financial model
▪Perform a risk evaluation
▪Test your MVP with customer
interactions
▪Create a business model design
▪Identify and test hypothesis in your
business model design
▪Evaluate your business model
analytically
Fruhwirth& Pammer-Schindler
(2023) @eBled. Towardsa
ProcessModel forDDBM.
Outstanding Paper
Disch et al. (2023)
@IntJ Hum-CompInt.
UsingKnowledge
Constr. Theory to
Evaluate…
White collarworkEducation Manufacturing,
electrical+ metal
engineering
Healthsector
Innovation + open
science
Mirzababaei& Pammer-Schindler (2022)
@EC-TEL. Best papernomination
Mirzababei& Pammer-Schindler (2023)
@IEEE TLT. Learning Engineering withLLMs
Fjadljevicet al.,
2020 @ACM LAK
–Slow isGood…
Bangerl, David, Disch &
Pammer-Schindler (2025)
@ACM CHI. CreAItive
Collaboration…
Automation Assistance
Scaffolding
Learning

3. 4
Data-relatedethicsissuesin technologiesforprofessional learning
4
Viktoria Pammer-Schindler, Institute ofHuman-CentredComputing1
5
1
1
1
2
1
3
1
4
Fesslet al. (2017) @IEEE TLT –
In-appreflectionguidance…
Rivera-Pelayo et al. (2017) @ACM toCHI–
Moodtrackingin a callcenter…
Fesslet al., 2012 @EC-TEL. Mood
Tracking in Virtual Meetings.1
5
1
1
1
2
1
3
1
4
Pammeret al., 2015 @EC-TEL.
The Value ofSelf-Tracking …
Pammer&Bratic,2034 @CHI
LBW. Surprise, surprise, …-

4. 5
Data-relatedethicsissuesin technologiesforprofessional learning
5
1)Data and UI representrelevant aspectsof
workplace/lifeactivities
2)Learning relatestoongoingprofessional/private
lifeexperience
Viktoria Pammer-Schindler, Institute of Human-Centred Computing
Data + Reflectionprompts = Learning
Pammer-Schindler & Prilla(2021) @IwC. The Reflection
Object

5. 6
Data-relatedethicsissuesin technologiesforprofessional learning
6
Learning analytics„works“ in formal educationsettings–whatare
additional/different challengesin professional learning?
General challengesin LA
▪Data
▪Analysis
▪Interpretation
▪Action
Additional challenges
(focus: informal professional learning)
▪Data:Sensitivityand confidentialityofdata
▪Analysis: Contextualisationofdata
▪Analysis: Findingtime and spacetoreflect
on dataforlearning
▪Interpretation: Whatcanreflectionachieve
–not everythingisa learningopportunity
▪Action:Someinsightsmaybeout ofscope
toimplement
Viktoria Pammer-Schindler, Institute of Human-Centred Computing

6. 7
Data-relatedethicsissuesin technologiesforprofessional learning
7
Data:Sensitivityand confidentialityofdata
Pammer-Schindler & Rosé, 2021 @ijAIED
▪Data-related ethics issues in field studies
▪Ways forward: How might such issues be addressed by modern
AI and data-based research?
Data are the foundation of modern AI
➢addressing data-related ethics issues will be central to making LA and
AIED work for informal and situated professional learning
Viktoria Pammer-Schindler, Institute of Human-Centred Computing

7. 8
Data-relatedethicsissuesin technologiesforprofessional learning
8
Multiple casesstudy, secondaryanalysis
Cases 1,2: Reflectionon time managementbasedon
activityloggingdata(Pammeret al., 2015; Fesslet al., 2017)
Case X: Cancelleddue toconcernsaroundworkplace
surveillanceratherthansupport forlearning
Cases 3,4: Reflectionon self-trackedmooddata(Fesslet al.,
2012; Rivera-Pelayo et al., 2017)
Primary RQs arounddesign and effectofinteractiondesign
incldatavis. + reflectionprompts.
SecondaryRQ: Whatethicsissuesappearedin field
studies?
Viktoria Pammer-Schindler, Institute of Human-Centred Computing1
5
1
1
1
2
1
3
1
4
Fesslet al. (2017) @IEEE TLT –
In-appreflectionguidance…
Rivera-Pelayo et al. (2017) @ACM toCHI–
Moodtrackingin a callcenter…
Fesslet al., 2012 @EC-TEL. Mood
Tracking in Virtual Meetings.1
5
1
1
1
2
1
3
1
4
Pammeret al., 2015 @EC-TEL.
The Value ofSelf-Tracking …
Pammer&Bratic,2034 @CHI
LBW. Surprise, surprise, …-

8. 9
Data-relatedethicsissuesin technologiesforprofessional learning
9
Issuesaroundactivity-log baseddataforreflection
▪Sensitive dataw.r.t. individual users
▪Activityon multiple devices
-> challengesarounddataintegration
▪Sensitive dataw.r.t. others
▪Issuesre-appearedin informal earlydesign activitiesin thehealthsector,
wheredatarelevant formedicalprofessionals‘ learningconcernspatients.
▪Confidentialbusiness/organisational data
▪Review ofautomaticallytrackedsensordataischallenging
Viktoria Pammer-Schindler, Institute of Human-Centred Computing1
5
1
1
1
2
1
3
1
4

9. 10
Data-relatedethicsissuesin technologiesforprofessional learning
10
Issuesaroundmoodself-tracking forreflection
Supports social awarenessand provides
side-channelcommunicationin collaboration
and teamwork
▪Show usernames?
+Makesdataactionable, high interestin moodofothers
−Changesthenatureof„self-tracking“
▪Self-tracking (moodstmt+ contextualnote)
giveshigh usercontrol
➢Noconcernsabouttouchingon privacyofothers, orconfidentiality.
Viktoria Pammer-Schindler, Institute of Human-Centred Computing

10. 11
Data-relatedethicsissuesin technologiesforprofessional learning
11
Summary: Key themes
(Pammer-Schindler & Rosé, 2021 @ijAIED–Data-related ethics issues in technologies for informal professional learning)
Ch1: Data for learning is not only about the learner (cf. also Pammer-Schindler & Prilla, 2021)
Ch2: Manual tracking may be a conduit for user control
▪In additiontofacilitatinglearningbystimulatingengagement.
▪Butsensitivityand confidentialityarestill issues!
Ch3: Learning isn‘ta priori a sharedgoalofall stakeholders
▪Re-contextualisationofdatamaybecritical
▪interpretation
▪ownership, confidentiality: usingdataloggedforperformancetrackingforlearning
▪workplacesurveillanceasbarriertolearning
Viktoria Pammer-Schindler, Institute of Human-Centred Computing

11. 12
Data-relatedethicsissuesin technologiesforprofessional learning
12
Waysforward: How might such issues be addressed by
modern AI and data-based research?
W1: Manual notesin naturallanguageaskeydataforLA
▪High usercontrol
▪Loggingasa reflectionintervention
▪New NLP capabilities(e.g., LLMs) support LA ofsuch data.
W2: Socio-technicaldesign processes
▪~ human-centreddesign; includingidentificationofconcretedatatobeusedorgeneratedin
design process.
W3: Scenario-baseddatacollectionin labs
▪Developasa communityrichscenariosofworkplacelearning
▪Thatcanbereplicatedand usedtostructurelab experiments
Viktoria Pammer-Schindler, Institute of Human-Centred Computing
ongoing
ongoing
???

12. 13
Data-relatedethicsissuesin technologiesforprofessional learning
13
Ongoing: Manual notesin natural
languageaskeydataforLA
Viktoria Pammer-Schindler, Institute of Human-Centred Computing
▪Transferfromtrainingtopracticeischallengingfor
professionals
▪Planning+reflectingontransferhelps
professionals
➢This iswhatthechatbotdoesforlearners.
▪Keepingtrackofwhathappenswithinformation
fromthetraininginpracticeishardforteachersin
MOOCsettings.
➢Gettingsuch dataiswhatthechatbotdoesfor
teachers.
➢Interactionlogandcontentanalysischatbotdata
canfeedintoLA-basedsystems.

13. 14
Data-relatedethicsissuesin technologiesforprofessional learning
14
References
Fesslet al., 2012. Mood Tracking in Virtual Meetings. Proceedings of the 7th European Conference on Technology-Enhanced Learning
(ECTEL 2012), 2012.
Fesslet al., 2017. In-app Reflection Guidance: Lessons Learned across Four Field Trials at the Workplace. IEEE Transactions on Learning
Technologies, Vol 10/4, pp 488-501,2017. DOI: https://doi.org/10.1109/TLT.2017.2708097
Pammer& Bratic, 2013. Surprise, Surprise: Activity Log Based Time Analytics for Time Management. 2013 ACM SIGCHI Conference on
Human Factors in Computing Systems, CHI ’13, Paris, France, April 27 -May 2. DOI: https://doi.org/10.1145/2468356.2468395
Pammeret al., 2015. The Value of Self-Tracking and the Added Value of Coaching in the Case of Improving Time Management. In: Design
for Teaching and Learning in a Networked World, Proceedings of the10th European Conference on Technology Enhanced Learning
(ECTEL 2015), pp.467-472, 2015. DOI: https://doi.org/10.1007/978-3-319-24258-3_41
Pammer-Schindler, 2019. Designing Data-Driven and Adaptive Technologies for Reflective Learning in the Workplace. Habilitation thesis,
Graz University of Technology.
Pammer-Schindler&Prilla,2021. The reflection object: An activity-theory informed concept for designing for reflection. Interacting with
Computers. DOI: https://doi.org/10.1093/iwc/iwab027
Pammer-Schindler&Rosé, 2021. Data-Related Ethics Issues in Technologies for Informal Professional Learning. International Journal of
Artificial Intelligence in Education. DOI: https://doi.org/10.1007/s40593-021-00259-x
Rivera-Pelayoetal.,2017.Introducing Mood Self-Tracking at Work: Empirical Insights from Call Centers. ACM Trans. Comput.-Hum.
Interact., ACM, 2017, 24, 3:1-3:28. DOI: https://doi.org/10.1145/3014058
Viktoria Pammer-Schindler, Institute of Human-Centred Computing