Use Cases and Course Review - Lecture 8 - Human-Computer Interaction (1023841ANR)

This lecture forms part of a Human-Computer Interaction course given at the Vrije Universiteit Brussel.

1. 2 December 2005
Human-Computer Interaction
Use Cases and Course Review
Prof. Beat Signer
Department of Computer Science
Vrije Universiteit Brussel
beatsigner.com

2. Beat Signer -Department of Computer Science [email protected] 2December15, 2023
HCI Research Contributions Revisited
▪Empirical contributions
▪Artefact contributions (systems research)
▪Methodological contributions
▪Theoretical contributions
▪Dataset contributions
▪Literature survey contributions
▪Opinion contributions

3. Beat Signer -Department of Computer Science [email protected] 3December15, 2023

4. Beat Signer -Department of Computer Science [email protected] 4December15, 2023
Prof. Dr. Beat Signer
Cross-Media Technology, Interac-
tive Paper, Data Physicalisation
Dr. Audrey Sanctorum
User-defined XDI and IoT Inter-
action, Human-AI Interaction
CISA
Human-Machine &
Human-Information
Interaction
Information
Systems &
Management
Information
Visualisation
& Navigation
WEB & INFORMATION
SYSTEMS ENGINEERING
Maxim Van de Wynckel
Hybrid Positioning, Implicit
Human-Computer Interaction
Yoshi Malaise
Technology-enhanced Learning,
Content-driven Presentations
CROSS-MEDIA INFORMATION SPACES
AND ARCHITECTURES (CISA)

5. Beat Signer -Department of Computer Science [email protected] 5December15, 2023
WEB & INFORMATION
SYSTEMS ENGINEERING
CISA
Human-Machine &
Human-Information
Interaction
Information
Systems &
Management
Information
Visualisation
& Navigation
CROSS-MEDIA INFORMATION SPACES
AND ARCHITECTURES (CISA)
Ekene Attoh
IoT Middleware, Context-aware
Computing, Implicit HCI
Isaac Valadez
Knowledge Physicalisation and
Augmentation, Tangible UIs
Xuyao Zhang
Extensible Platform for Dynamic
Data Physicalisation
Ingela Rossing
Dynamic Data Physicalisation
Framework and Guidelines

6. Beat Signer -Department of Computer Science [email protected] 6December15, 2023
WEB & INFORMATION
SYSTEMS ENGINEERING
CISA
Human-Machine &
Human-Information
Interaction
Information
Systems &
Management
Information
Visualisation
& Navigation
CROSS-MEDIA INFORMATION SPACES
AND ARCHITECTURES (CISA)
Arun Sojan
Physicalisation for Digital Twins
Piet Van Der Paelt
Julia-based Framework for
Simulation and Optimisation
Evan Cole
Technology-enhanced Learning,
Study Lenses

7. Beat Signer -Department of Computer Science [email protected] 7December15, 2023
Interactive Paper
Global Information Systems Group
ETH Zurich

8. Beat Signer -Department of Computer Science [email protected] 8December15, 2023
Visual and Interaction Design
▪Visualisationof interactive
areas and functionality
▪design patterns
▪Interaction design
▪online versus offline input
processing
▪lack of modal dialogues
▪multimodal interaction
▪Findings might be appli-
cable to traditional GUIs
▪e.g. cross-application widget
interaction
Global Information Systems Group, ETH Zurich

9. Beat Signer -Department of Computer Science [email protected] 9December15, 2023
iGestureFramework
▪iGestureWorkbench
▪create/test gesture sets and
algorithms
▪Different modalities
▪digital pen, tablet PC,
mouse, Wii remote, …
▪multimodal gestures
▪Open source
▪http://www.igesture.org

10. Beat Signer -Department of Computer Science [email protected] 10December15, 2023
01001000100
0101001100010010100010010
0100100010010001000100001001000
100010001100001001000110010101010110
1100010010100010010010010100010101011001
101001100001100 101010100
0010100 100 110101010101
00100 01010101000
1001 1111111111
111 0000000010
0 11111111
0101001011
010101010
111010101
001010010110
Fluid Cross-Media Information Spaces

11. Beat Signer -Department of Computer Science [email protected] 11December15, 2023
RSL Hypermedia Metamodel
Global Information Systems Group, ETH Zurich

12. Beat Signer -Department of Computer Science [email protected] 12December15, 2023
What is Wrong with Slideware?
▪Simulation of physical slides
▪limited spacedue to the slide concept
▪linear navigationfrom slide to slide
▪difficult to reuse contentand embed rich media types
▪MindXpres addresses these issues
▪unlimited canvas with zoomable user interface
▪non-linear navigation and associative linking
▪content-based approachwith automatic visualisation

13. Beat Signer -Department of Computer Science [email protected] 13December15, 2023
Presentation Solutions
▪PaperPoint
▪interactivePowerPoint paper
handouts
▪non-linear navigationand
annotations
▪MindXpres
▪extensible presentation
platform
▪content-driven approach
▪non-linear navigationvia
zoomable user interface
▪associativelinking
▪rich media plug-ins

14. Beat Signer -Department of Computer Science [email protected] 14December15, 2023
Interactive Source Code Plug-in
Paul Mestereaga

15. Beat Signer -Department of Computer Science [email protected] 15December15, 2023
MindXpres Video

16. Beat Signer -Department of Computer Science [email protected] 16December15, 2023
Personal Information Management (PIM)
▪Keeping, organisingand
re-findinginformation
▪digital and physical
▪Study of human-
information interaction
▪files, piles, mixtures, …
▪OC2 PIM model
▪based on RSL hypermedia
metamodel
▪Cross-Media PIM system
▪explicitas well as implicit
associationsbetween entities

17. Beat Signer -Department of Computer Science [email protected] 17December15, 2023
PimVisDocument View
Audrey Sanctorum

18. Beat Signer -Department of Computer Science [email protected] 18December15, 2023
PimVisSetup

19. Beat Signer -Department of Computer Science [email protected] 19December15, 2023
Tangible Holograms (TangHo)
Timothy J. Curtin

20. Beat Signer -Department of Computer Science [email protected] 20December15, 2023
Collaborative TangHoInteraction

21. Beat Signer -Department of Computer Science [email protected] 21December15, 2023
TangHoPrototype
▪Lego Mindstorms-based
6DOF arm prototype
▪motors can be locked or set to
float mode
-useful for bidirectional I/O
▪replaceableend effectors for different
types of non-visual feedback
▪Challenges
▪inverse kinematics (final system
is body mounted)
-use robotics toolbox (MATLAB)
▪limited HoloLens hand tracking
▪3D printing of final arm
Timothy J. Curtin

22. Beat Signer -Department of Computer Science [email protected] 22December15, 2023
TangHoInteractions

23. Beat Signer -Department of Computer Science [email protected] 23December15, 2023
Real-Time Cross-Media Data Exploration

24. Beat Signer -Department of Computer Science [email protected] 24December15, 2023
Dynamic Data Physicalisation
▪Understanding the design space
▪how do users map data to physicalisations
▪Data physicalisation design guidelines
▪what are the just-perceptual differences that
a physical variable can convey?
▪Dynamic data physicalisation framework
▪data physicalisation grammar
▪data processing with device-independent output
▪software drivers for different physicalisations (e.g.TangHo)
▪New application domains
Jacques Bertin, 1967

25. Beat Signer -Department of Computer Science [email protected] 25December15, 2023
Dynamic Data Physicalisation Framework
...

26. Beat Signer -Department of Computer Science [email protected] 26December15, 2023
SpeeG2: Multimodal Text InputUser
Speech recognition
(Microsoft SAPI 5.4)
Skeletal tracking
(Microsoft Kinect)
5
4
2
3
SpeeG2 GUI
6
1
Sven De Kock

27. Beat Signer -Department of Computer Science [email protected] 27December15, 2023
Video: SpeeG2 Performance (21 WPM)

28. Beat Signer -Department of Computer Science [email protected] 28December15, 2023
Course Summary
1.Introduction
▪HCI and Interaction Design
▪history of human-computer interaction
-analogue computers, oNLineSystem (NLS) and desktop computers
▪“evolution” of interfaces
▪interface types
-command-based, WIMP, pen, touch, speech, gesture, VR, AR, multimodal, …
2.HCI and Interaction Design
▪what to design
-who (user), how (task) and where (context)
▪usability goals
-effectiveness, efficiency, safety, utility, learnability, memorability
▪user experience (UX) goals

29. Beat Signer -Department of Computer Science [email protected] 29December15, 2023
Course Summary …
HCI and Interaction Design …
▪Double Diamond Design Process
▪user-centred design principles
▪Interaction Design Lifecycle Model
-establishing requirements, designing alternatives, prototyping and evaluating
3.Requirements Analysis and Prototyping
▪types of requirements
-functional, data, environmental, user characteristics, usability, user experience
▪requirements gathering techniques
-interviews, focus groups, surveys, direct and indirect observation, …
▪task description
-scenarios and use cases
▪prototyping
-horizontal vs.vertical, throwaway vs.evolutionary

30. Beat Signer -Department of Computer Science [email protected] 30December15, 2023
Course Summary …
4.Human Perception and Cognition
▪human senses and modalities
▪model of visual perception processing
▪anatomy of the human eye
-lens, pupil, retina (rods and cones), …
▪brightness perception and illusions
▪visual field and peripheral vision
▪perception
-biased by past, present and future
-designing for biased perception
▪Gestalt principles
-law of proximity, law of similarity, law of continuity, law of closure, …
▪long-term memory and immediate memory

31. Beat Signer -Department of Computer Science [email protected] 31December15, 2023
Course Summary …
5.Design Guidelines and Models
▪Shneiderman’seight golden rules of interfaces design
▪Nielsen’s ten usability heuristics for UI design
▪visibility, simplicity and mapping
▪guidelines for using colours
▪inattentional/change blindness
▪affordances
▪Fitts’s law

32. Beat Signer -Department of Computer Science [email protected] 32December15, 2023
Course Summary …
6.Evaluation Methods
▪why, what, where and when?
▪DECIDE evaluation framework
-goals, questions, methods, practical issues, ethical issues, evaluate/present
▪usability testing
▪experimental design
-between subjects, within-subjects and pair-wise design
▪UEQ+ user experience questionnaire
▪inspection
-heuristic evaluation and walkthroughs
▪analytics
-Google analytics, A/B testing, …
▪predictive models
-GOMS model, keystroke level model and Fitts’s law

33. Beat Signer -Department of Computer Science [email protected] 33December15, 2023
Course Summary …
7.HCI Research Methods
▪seven types of HCI research contributions
-empirical contributions, artefact contributions (systems research),
methodological contributions, theoretical contributions, dataset contributions,
literature survey contributions, opinion contributions
▪descriptive, relational and experimental research
-statistical significance (e.g. t tests)
-correlation does not imply a causal relationship
▪surveys, diaries and interviews
▪grounded theory (inductive research)
▪ethnography
▪automatic data collection
-measuring the human and online data
▪human computation

34. Beat Signer -Department of Computer Science [email protected] 34December15, 2023
Course Summary …
8.Use Cases and Course Review
▪use cases for different HCI research contributions

35. Beat Signer -Department of Computer Science [email protected] 35December15, 2023
Exam
▪Written closed book exam
▪covers content of lecturesand exercises
▪Date
▪January 9, 13:00–16:00
▪Overall grade = writtenexam (50%) + assignment (50%)
▪for the assignment students have some flexibility in distributing
the grades (±2 points)
-send us an email by January 7, 2024 with all group members in CC if
you want to distribute some points
▪note that the grade for the writtenexam as well as for the assign-
menthave to be 8/20 or higher in order to pass the exam!
▪Submission of the assignment via Canvas
▪deadline: December 23, 24:00(Brussels timezone)

36. Beat Signer -Department of Computer Science [email protected] 36December15, 2023
Exam …
▪The exam will cover all the content presented in the
lecturesas well as any additional information from
the exercise sessions
▪includes the videos shown in some of the lectures
▪Make sure that you understand the basic conceptsfirst
▪however, we might ask questions at any level of detail to evaluate
your knowledge
▪Applying the theoretical knowledge to concrete use
cases
▪e.g. given a description of a specific problem in a given domain
-decide about requirements gathering techniques
-define different types of requirements

37. Beat Signer -Department of Computer Science [email protected] 37December15, 2023
Exam …
▪Applying the theoretical knowledge to concrete
use cases …
▪e.g. given a description/pictures of a specific UI
-identify potential issues based on heuristics, Gestalt principles, Fitts’s law and
other guidelines
▪Describing (and potentially applying) certain concepts
▪Interaction Design Lifecyle Model
▪Double Diamond Design Process
▪steps of the DECIDE evaluation framework
▪usability goals
▪…

38. Beat Signer -Department of Computer Science [email protected] 38December15, 2023
Other Related Courses
▪Specialisation: Data Management and Analytics(DAMA)
▪Information Visualisation(MA)
▪representation of data (encoding of value/relation)
▪presentation of data (visualisation techniques)
▪interaction
▪dashboards and case studies
▪Next Generation User Interfaces(MA)
▪multimodal interaction
-theoretical concepts, fusion and fission, ...
▪interactive tabletops and surfaces
▪pen-based interaction
▪gesture-based interaction
▪tangible interaction and mixed reality

39. Beat Signer -Department of Computer Science [email protected] 39December15, 2023
Other Related Courses ...
▪Advanced Topics in Big Data(MA)
▪seminar about recent developments in Big Data
▪scalable data management and big data analytics
▪graph processing and distributed query processing
▪human-data interaction and human-in-the loop data processing
▪information visualisation and data physicalisation
▪Bachelor's and a Master's thesis

40. Beat Signer -Department of Computer Science [email protected] 40December15, 2023
References
▪ B. Signer, Fundamental Concepts for
InteractivePaperand Cross-Media Information
Spaces, Second Edition, ISBN 978-3-8370-2713-6,
August2017
▪J.O. Wobbrockand J.A. Kientz, Research Contributions
in Human-Computer Interaction, Interactions 23(3),
April 2016
▪https://doi.org/10.1145/2907069
▪B. Signer, Towards Cross-Media Information Spaces and
Architectures, Proceedings of RCIS 2019, Brussels,
Belgium, May 2019
▪https://beatsigner.com/publications/signer_RCIS2019.pdf

41. Beat Signer -Department of Computer Science [email protected] 41December15, 2023
References ...
▪B. Signer and T.J. Curtin, Tangible Holograms:
Towards Mobile Physical Augmentation of Virtual
Objects, Technical Report WISE Lab, WISE-2017-01,
March 2017
▪https://beatsigner.com/publications/signer_arXiv2017.pdf
▪B. Signer, P. Ebrahimi, T.J. Curtin and Ahmed K.A.
Abdullah, Towards a Framework for Dynamic Data
Physicalisation, International Workshop Toward a Design
Language for Data Physicalisation, Berlin, Germany,
October 2018
▪https://beatsigner.com/publications/signer_DataPhys2018.pdf

42. Beat Signer -Department of Computer Science [email protected] 42December15, 2023
References ...
▪L. Hosteand B. Signer, SpeeG2: A Speech-and
Gesture-based Interface for Efficient Controller-free Text
Entry, Proceedings of ICMI 2013, 15th International
Conference on Multimodal Interaction, Sydney, Australia,
December 2013
▪https://beatsigner.com/publications/hoste_ICMI2013.pdf
▪E.I. Reuss, B. Signer and M.C. Norrie, PowerPoint
Multimedia Presentations in Computer Science
Education: What do Users Need?, Proceedings of USAB
2008, 4th Symposium on Usability & HCI for Education
and Work, Graz, Austria, November 2008
▪https://beatsigner.com/publications/reuss_USAB2008.pdf

43. Beat Signer -Department of Computer Science [email protected] 43December15, 2023
References ...
▪R. Roels and B. Signer, MindXpres: An Extensible
Content-driven Cross-Media Presentation Platform,
Proceedings of WISE 2014, 15th International
Conference on Web Information System Engineering,
Thessaloniki, Greece, October, 2014
▪https://beatsigner.com/publications/roels_WISE2014.pdf
▪R. Roels, P. Mestereagaand B. Signer, An Interactive
Source Code Visualisation Plug-in for the MindXpres
Presentation Platform, Communications in Computer
and Information Science (CCIS), 583, 2016
▪https://beatsigner.com/publications/roels_CCIS2016.pdf

44. Beat Signer -Department of Computer Science [email protected] 44December15, 2023
References ...
▪Y. Malaise and B. Signer, Personalised Learning
Environments Based on Knowledge Graphs and the
Zone of Proximal Development, Proceedings of
CSEDU 2022, 14th International Conference on
Computer Supported Education, April 2022
▪https://beatsigner.com/publications/malaise_CSEDU2022.pdf
▪M.C. Norrie, B. Signer, M. Grossniklaus, R. Belotti,
C. Decurtinsand N. Weibel, Context-aware Platform for
Mobile Data Management, Wireless Networks (WINET),
13(6), Springer, December 2007
▪https://beatsigner.com/publications/norrie_WINET2007.pdf

45. Beat Signer -Department of Computer Science [email protected] 45December15, 2023
References ...
▪B. Signer, U. Kurmann and M.C. Norrie, iGesture:
A General Gesture Recognition Framework, Proceedings
of ICDAR 2007, 9th International Conference on
Document Analysis and Recognition, Curitiba, Brazil,
September 2007
▪https://beatsigner.com/publications/signer_ICDAR2007.pdf
▪S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring
and Re-finding Documents in Cross-Media Information
Spaces, Proceedings of AVI 2016, International Working
Conference on Advanced Visual Interfaces, Bari, Italy, June
2016
▪https://beatsigner.com/publications/trullemans_AVI2016.pdf

46. Beat Signer -Department of Computer Science [email protected] 46December15, 2023
References ...
▪MindXpresProject
▪https://mindxpres.com
▪Dynamic Data Physicalisation and Tangible Holograms
▪https://wise.vub.ac.be/topic/dynamic-data-physicalisation
▪https://wise.vub.ac.be/topic/tangible-holograms-tangho
▪iGesture
▪http://igesture.org
▪OpenHPS
▪https://openhps.org

47. 2 December 2005
The End
Good Luck with the Exam!