AI for HCI – could this be a better title if I’d asked ChatGPT

Seminar in Pisa, Italy, 11th June 2024

https://www.alandix.com/academic/talks/Pisa-AI4HCI-2024/

AI has entered into all aspects of life. Sometimes this is hidden, below the surface of the devices and applications we use; sometimes much more explicit in interactions with devices and user interfaces. In this talk I'll explore some of the ways in which AI can be used to enhance existing interactions and also how we can effectively design user interfaces for A-rich systems. In addition I can be used by UX designers and AI developers need better ways to interact with their tools and systems. Perhaps more fundamental is not the direct effects of AI, but the ways in which it is fundamentally changing the society and world in which we live.

1. AI for HCI could this be a better title if I’d asked ChatGPT Alan Dix https:// alandix.com / aibook / @ alanjohndix HCI for AI AI for HCI

2. today I am not talking about … qualitative–quantitative reasoning deep digitality and digital thinking next generation UX tools long tail of small data physicality now digital light walking round Wales virtual crackers and slow time digital humanities and community heritage modeling dreams, regret and the emergence of self

3. 2 nd edition coming soon plus AI for HCI AI for Social Justice plugs

4. it takes two to tango a synergistic approach to human-machine decision making https://tango- horizon.eu /

5. p ersonal interest some examples gender and ethnic bias in ML … 1992 aQtive – dot com intelligent interfaces GAs for submarine design

6. AI and User Interaction HCI for AI Rich Systems Intelligent Interfaces Ethics Social Justice Governance H uman like C omputing Interfaces for AI developers Big Data and Evaluation

7. HCI for AI Rich Systems Intelligent Interfaces Ethics Social Justice Governance H uman like C omputing Interfaces for AI developers Big Data and Evaluation Intelligent Interfaces intelligent interfaces appropriate intelligence AI & error

8. HCI for AI Rich Systems Intelligent Interfaces Ethics Social Justice Governance H uman like C omputing Interfaces for AI developers Big Data and Evaluation HCI for AI Rich Systems autonomous vehicles smart houses using ChatGPT

9. HCI for AI Rich Systems Intelligent Interfaces Ethics Social Justice Governance H uman like C omputing Interfaces for AI developers Big Data and Evaluation Big Data and Evaluation A-B testing layout design surrogate users heuristics & guidelines

10. HCI for AI Rich Systems Intelligent Interfaces Ethics Social Justice Governance H uman like C omputing Interfaces for AI developers Big Data and Evaluation Interfaces for AI developers XAI visualization hidden values

11. HCI for AI Rich Systems Intelligent Interfaces Ethics Social Justice Governance H uman like C omputing Interfaces for AI developers Big Data and Evaluation AI helping UI UI helping AI front-end user-facing back-end developer-facing

12. HCI for AI Rich Systems Intelligent Interfaces Ethics Social Justice Governance H uman like C omputing Interfaces for AI developers Big Data and Evaluation Intelligent Interfaces HCI for AI Rich Systems synergistic interactions hybrid human–AI

13. hybrid human–AI HCI for AI Rich Systems Intelligent Interfaces Ethics Social Justice Governance H uman like C omputing Interfaces for AI developers Big Data and Evaluation synergistic interactions hybrid AI working with UX designers hybrid AI working with AI developers adapting AI for synergy adapting UI for synergy

14. HCI for AI Rich Systems Intelligent Interfaces Ethics Social Justice Governance H uman like C omputing Interfaces for AI developers Big Data and Evaluation Intelligent Interfaces HCI for AI Rich Systems synergistic interactions focus

15. is AI different? the three Cs: c omplexity (un) c ertainty c o-adaptation

16. touchstone phrases alien intelligence not as we know it appropriate intelligence fitting within interaction framework optimise the human –AI system, not just the AI sufficient reason explanations we would accept from a human being critical for AI explainability

17. styles of collaboration Servant – we tell AI what to do (explicitly) Master – AI tells us what to do, e.g. Uber Symbiosis – using complementary abilities “Man-Computer Symbiosis” J. C. R. Licklider, March 1960 N.B. boundaries may be blurred – e.g. maps interfaces, recommendations

18. interacting with AI from adaptive websites to semi-autonomous cars ... think about abilities (function allocation) rote vs creative pattern finding vs interpretation think about handover physical handover regaining situational awareness not at the last moment!

19. design for interacting with AI some heuristics ... deterministic ground knowing what may change appropriate intelligence when it all goes wrong incidental interaction beyond the current task

20. deterministic ground

21. deterministic ground ... for menus David Mary Florence Adrian Andy Brian Charlotte Colin David Devina . . . Message adaptive area most popular/likely at top quick when it gets it right deterministic ground alphabetic below still easy to scan when it isn’t

22. Adrian Andy Brian Charlotte Colin David Devina Elizabeth Florence George . . . Message deterministic ground ... for menus … another option adaptive aspect highlight most popular/likely items deterministic ground overall order always alphabetic

23. appropriate intelligence

24. rules of standard AI interfaces it should be right as often as possible when it is right it should be good good for demos look how clever it is!

25. rules of appropriate intelligence it should be right as often as possible when it is right it should be good when it isn’t right ... it shouldn’t mess you up what makes a system really work! ✗ ✔

26. onCue intelligent ‘context sensitive’ toolbar sits at side of the screen watches clipboard for cut/copy suggests useful things to do with copied date

27. 20 21 22 23 25 24 20 17 7 7 3 7 the dancing histograms very useful a ing out some of the textile sites yo x's page at http:// www.hiraeth.com / onCue (dot com days!) intelligent ‘context sensitive’ toolbar histograms user selects text and copies it to clipboard icons slowly fade in

28. incidental interaction

29. beyond intention 31 car courtesy lights turn on when doors unlocked/open turned off after time period when engine turned on incidentally the lights come on driver's purpose is to get into the car

30. beyond intention 32 shopping cart goods in shopping cart analysed e.g. Amazon books used to build knowledge about books people who like X also like Y used to give you suggestions “you might like to look at …”, “special offer …” incidentally shown related titles my purpose to buy a book

31. beyond intention 33 onCue ‘intelligent’ toolbar appropriate intelligence make it good when it works don’t make it hard of it doesn’t analyses clipboard contents suggests things to do with it incidentally alternative things to do user's purpose to copy text elsewhere

32. beyond intention 34 continuum of purpose press light switch intentional expected walk into room expecting lights to switch on incidental walk into room … unbeknown to you … air conditioning increases tip e-reader to turn page explicit implicit

33. two tasks primary task current focus of attention may directly include AI secondary task supported by sensing may be same or different user same or different time

34. synergy cooperation and adaptation adapting AI for human interaction adapting user interfaces for AI AI

35. adapting AI for interaction full automation – best/safe human in the loop – suggest and inform

36. In Concert

37. collating notices several newspaper notices for each concert semi-automated grouping: – liberal suggestions – conservative warnings

38. adapting interaction for AI ‘best’ UI – short term gain epistemic interaction – long term gains

39. temperature setting buttons 20° 17° current temperature target temperature “currently heating” icon

40. “I want it cooler” button “I want it warmer” button

41. 20° 17° more information for adaptation

42. epistemic action wondering what’s inside? just peek

43. document results – option 1 – scroll scroll down to see more results

44. document results – option 2 – accordion press ‘+’ to expand interesting items

45. which option? – epistemic interaction … scroll accordion A-B user testing scroll a little better accordion gives more data for AI adaptation

46. design for interacting with AI deterministic ground helping users know what may or may not adapt appropriate intelligence tuning AI to offer human alternatives and fail well incidental interaction sensing one taks to help another epistemic interaction choosing user interaction that is informative for ML

47. big lessons primary and secondary tasks good AI+HCI not about developing the most accurate AI but creating most effective and enjoyable overall human-technical system

48. AI for HCI HCI for AI Alan Dix https:// alandix.com / aibook /