Applied Distributed Cognition using foundation models and biosensing

In this talk I lay out a multidisciplinary research agenda that unifies two main lines of basic research—biosensing/neurosensing and foundation model-based AI—as core building blocks for developing applied distributed cognition. We can now take the frameworks cognitive science has used to explain how humans think with our environments and invert them to design environments that think with us, that can anticipate our future cognitive states and compensate for our cognitive limitations (and, of course, be applied for darker purposes). I will highlight several projects that explore practical aspects of this opportunity, from robotics to decision-making, and describe how it can be seen as a continuation of the original cybernetic vision of the 1940s that undergirds so much of today’s technological research progress.

1. Applied Distributed Cognition using foundation models and biosensing (a research agenda) Mike Kuniavsky UC Davis May 30, 2024

2. About me

3. Applied Distributed Cognition – May 30, 2024 3 The human experience of new technology

4. Applied Distributed Cognition – May 30, 2024 4 The human experience of new technology

5. Applied Distributed Cognition – May 30, 2024 5 The human experience of new technology

6. Applied Distributed Cognition – May 30, 2024 6

7. Applied Distributed Cognition – May 30, 2024 7

8. Applied Distributed Cognition – May 30, 2024 8

9. Applied Distributed Cognition – May 30, 2024 9

10. Applied Distributed Cognition – May 30, 2024 10 Accenture Labs

11. Applied Distributed Cognition – May 30, 2024 11

12. A little Cybernetics The May 1942 Macy Foundation Conference on Cerebral Inhibition, and how it changed our world

13. 1946-1953: the Macy conferences on Cybernetics Margaret Mead, anthropologist Arturo Rosenblueth , physiology Gregory Bateson, anthropologist Warren McCulloch, neuropsychiatry Walter Pitts, mathematics John von Neumann, mathematics Norbert Wiener, mathematics Joseph Licklider , psychology Claude Shannon, engineering Harold Abramson, medicine Applied Distributed Cognition – May 30, 2024 13

14. Applied Distributed Cognition – May 30, 2024 14 Cybernetics saw biology and electronics as two examples of the same thing

15. Applied Distributed Cognition – May 30, 2024 15

16. Human computer integration: Preemptive Action Applied Distributed Cognition – May 30, 2024 16 Source: Kasahara, Shunichi, Jun Nishida, and Pedro Lopes. “Preemptive Action: Accelerating Human Reaction Using Electrical Muscle Stimulation Without Compromising Agency.” (2019)

17. Human computer integration: HINDSIGHT Applied Distributed Cognition – May 30, 2024 17 Source: Schoop , Eldon, James Smith, and Bjoern Hartmann. “ HindSight : Enhancing Spatial Awareness by Sonifying Detected Objects in Real-Time 360-Degree Video.” (2018)

18. Applied Distributed cognition Enacting an explanatory framework

19. Applied Distributed Cognition – May 30, 2024 19 Distributed cognition

20. Distributed Cognition: a short inaccurate history 1970s: Social Cognition 1980s: Actor Network Theory 1980s: Minsky’s The Society of Mind 1990s: Nass and Reeves’ The Media Equation 1990s: Hutchins coins “Distributed Cognition” 2000s: Clark’s Extended and Embodied cognition etc

21. Applied Distributed Cognition Applied Distributed Cognition – May 30, 2024 21

22. Applied Distributed Cognition: BACKGROUND Applied Distributed Cognition – May 30, 2024

23. new BIOsensors to identify cognitive states Apple (published 2023) Meta (published 2023)

24. A person + AI = a distributed cognition team with complementary roles

25. Foundation models n-to-n reasoning, where n can be human or machine cognition

26. FOUNDATION MODELS Applied Distributed Cognition – May 30, 2024 26 Source: Bommasani , Rishi, Drew A. Hudson, Ehsan Adeli , Russ Altman, Simran Arora, Sydney von Arx , Michael S. Bernstein, et al. “On the Opportunities and Risks of Foundation Models.” (2021)

27. FOUNDATION MODELS TO anticipate needs and actions Applied Distributed Cognition – May 30, 2024 27 Source: Bommasani , Rishi, Drew A. Hudson, Ehsan Adeli , Russ Altman, Simran Arora, Sydney von Arx , Michael S. Bernstein, et al. “On the Opportunities and Risks of Foundation Models.” (2021)