Social Interaction in Immersive Environments.pdf

We are witnessing a remarkable transformation in the way technology shapes our social lives. What originated with the Internet, social media, and smartphones is now evolving into immersive technologies. While immersive technologies began as single-user experiences, they are now transforming into rich, multi-user environments, creating new opportunities for social interaction, embodiment, and shared experiences.

In this keynote, we'll explore how immersive environments influence human connection, the challenges of embodiment, ways of interacting between digital and physical realms in location-based VR, and new possibilities these spaces open up for learning and social science research.

1. SocialInteraction in
Immersive Environments
Matthias Wölfel

2. iXperienceLab

3. Social Interaction
Designing a social interaction in
immersive environments brings
its own complexities compared
to single-user applications.
Examples
§co-located or remote
§remote interaction or agent-mediated
§level of intimacy (e.g. proxemics)
§appearance

4. VR Campus
§social-VR application for lectures
§in-house development in Unity3D
§General Data Protection
Regulation compliant
§hosted on own cloud services
§PDF and OBJ interfaces
§personalized avatars via Ready
Player Me
§60 Meta Quest 2 are provided to
the students to be used remotely

5. Tobeabletocheck appearancein VR

6. VR Campus
Spawn Room
Basketball Court
LecturePlatform

8. Do youseeaddedvaluein theVR Campus?
AddedValue:
§practicalexperience
§fun
§interactiveexchange
§playfulcomponents
§alternative tofrontal lecture
yesno

9. Do youseeaddedvaluein theVR Campus?
Lack ofAddedValue:
§lack ofmeaningfulnessof
PowerPoint in immersive VR
§lack ofabilitytotakenotes
§cybersickness
yesno

10. SocialPerceptionofAvatars
withouttracking
face and gaze
tracking

11. SocialPerceptionofAvatars
1 2
TN
NT
1 2
Interview
Group
withfacial
tracking
withoutfacial
tracking
Study Procedure
Questionaire

12. SocialPerceptionofAvatars
Interview
Group
withfacial
tracking
withoutfacial
tracking
Likability
p-value
Likert scale
(1 to7; 7 better)4.75
0.9681
TN
NT
6.31
5.69
2
6.00
6.00
0.0230.750
0.538
0.334

13. SocialPerceptionofAvatars
Interview
Group
withfacial
tracking
withoutfacial
tracking
Uncanniness
p-value
Likert scale
(1 to7; 7 better)4.75
0.9681
TN
NT
2.63
2.38
2
2.63
2.25
0.4830.274
1.000
0.648

14. SocialPerceptionofAvatars
Interview
Group
withfacial
tracking
withoutfacial
tracking
Humanlikeness
p-value
Likert scale
(1 to7; 7 better)4.75
0.9681
TN
NT
5.88
5.06
2
5.56
5.56
0.0340.814
0.451
0.149

15. SocialPerceptionofAvatars
Interview
Group
withfacial
tracking
withoutfacial
tracking
Self-Similarity
p-value
Likert scale
(1 to7; 7 better)4.75
0.9681
TN
NT
4.56
4.75
2
4.88
4.69
0.6360.608
0.336
0.907

16. SocialPerceptionofAvatars
Interview
Group
withfacial
tracking
withoutfacial
tracking
Knowledgeable
p-value
Likert scale
(1 to7; 7 better)4.75
0.9681
TN
NT
5.50
5.38
2
5.50
5.56
0.7950.968
0.984
0.721

17. Mediation ofIntangibleCultural Heritage

18. Agent-MediatedCommunication
§VR offers a variety of representation of embodied agents
§customized agents depending on situation or narrative
§interaction concepts possible that only work in virtual world
Disembodied HumanAnthropomorphic
Object
Object

19. Agent-MediatedCommunication
§Presence is not influenced by
the type of visual appearance
of the agent.
§Attractiveness is partly
influenced by the type of
visual appearance of the agent.
§Sense of agency is not
influenced by the type of
visual appearance of the agent.

20. CalmComputing
The idea of calm computing is rather
than focusing on computing to
augment and bring relevant
information in a more natural and
integrated way.
Imitate sitting postures by anthropo-
morphic objects to improve sitting
habits.

21. What Type of Object?
Desk ObjectsAbstract ObjectsHerbal Objects

22. Opinion of the Users
What is your preferred object type?
herbal objectsdesk objectsabstract objects

23. Opinion of the Users
Would you prefer a physical object?
yesundecidedno

24. Feedback on the Seating Position
Detect seating position with sensor and transfer to plant.

25. KineticSpace
A tool to provide easy and intuitive access to gesture
recognition technology:
§record movements
§classify movements automatically
§analyze movements automatically
[...] What is it good for? Well, it can read a gesture from one person and
register it on another and you can train it to register tiny movements and,
potentially, allow for full motion control of your PC. Minority Report it isn't,
but that future is getting closer and closer.

27. Calm Computing
design the object which is able to reflect user habits

28. Setup

29. 0
1
2
3
4
5
6
7
8
9
10
1. Quartal2. Quartal
§A situation is established, which
corresponds to a possible real use of the
application.
§The user is invited to sit in front of the
computer or laptop and to research any
topic in the online encyclopedia
Wikipedia.
§The user has been informed that the
virtual plant reacts to his/her actions, but
no details have been given.
Experimental Setup

30. Opinion of the Users
Do you think you can improve your seating position using this application?
strongly disagreedisagreeundecidedagreestrongly agree

31. Opinion of the Users
Could you focus on your task without getting distracted by the application?
strongly disagreedisagreeundecidedagreestrongly agree

32. Opinion of the Users
Could you get the information provided by the orchid?
strongly disagreedisagreeundecidedagreestrongly agree

33. UncannyValley
1
2
3
4
5
6
7
123456789101112131415
beforevalleyin valleyafter valley

34. How embodied agents should respond to nonverbal behavior

35. Services
EmbodiedAgent
Middleware
Rasa Core
(dialogsystem)
KineticSpace
(gesturerecognition)
Deepface
(identification)
Unity
(3D-agent)
Whisper
(speechrecognition)
Neural-HMM
(speechsynthesis)
Input
Output

36. Social Signals
Social SignalTyp BehaviourCaptureTransmit
nodding gesturepositiv■■
waveinggesturepositiv■■
lateral head tilt posturepositiv■■
looking at person gazepositiv□■
turning to person proxemicspositiv□■
approaching personproxemicspositiv■■
following personproxemicspositiv■■
hands behind the back postureneutral■■
arms neutralpostureneutral■■
arms crossedposturenegativ■■
looking awaygazenegativ□■
turning away proxemicsnegativ□■
flinchproxemicsnegativ■■

37. Interaction on 180°screen

38. PerceivedSympathy
1
2
3
4
5
6
7
ImitationConsentRejectionRandom

39. PerceivedSocialCompetence
1
2
3
4
5
6
7
ImitationConsentRejectionRandom

40. Immersive Learning
Immersive Learning is a learning method where learners are
embedded in a virtual world.
Goals of learning with VR:
§learning effectiveness
§cost & availability
§realistic simulation
§practicing dangerous situations
§fun while learning

41. Tell meand I will forget,
showmeand I mayremember;
involvemeand I will understand.
“Konfuzius, Chinesischer Philosopher

42. WouldConfucius haveusedimmersive learningapplications?

43. DimensionsofReality
physical
reality
mixed
reality
virtual
reality
mental
reality

44. Mental Model
mental model
A mental model is a simplified model of the environment.
Only the relevant aspects of the environment (physical reality) are
represented in the mental model.
physical
reality
mixed
reality
virtual
reality
mental
reality

45. Mental Model
forms generatesalignsaligns
mental model
physical
reality
mixed
reality
virtual
reality
mental
reality

46. Mental Model
forms generatesalignsaligns
mental model
physical
reality
mixed
reality
virtual
reality
mental
reality
suspensionofdisbelief

47. Star-Wars-Effect
The Star-Wars-Effect(also naïvephysics) describesthatviewers
havefalseexpectationsofphysicalevents.

48. G
A
P
Therearestill
many
differences!

49. In the near future, neither tactile perception
nor self-perception can be established without
considerable effort.
Technological Differences
Examples:
§mismatch between sensory information can cause
cyber sickness
§lack of physical replication reduces physical constrains
and weight representation
§use of hand-hold controllers instead of free-hand
interaction

50. Conceptual Differences
Strategies that are not possible in reality, but in
virtual environments include superhuman skills
such as:
§teleportation,
§flying, or
§manipulation of objects outside arm reach
Interaction in virtual environments requires
strategies that are either not needed or not
possible in reality.

51. Didactical Differences
Examples
§the superimposition of
information in real space,
§changes of perspective through
body ownership illusion
Mixed reality enables the implementation of didactic concepts
that cannot be realized in other forms of analog or computer-
supported learning using a 2D monitor.

52. Link betweentheWorlds
mental model
of the virtual application
mental model
of the applicationmental model
of the real application
applicationdigital twin
Application
forms
deduces
apply
applyadapts
apply

53. Internal and External Validity
Examples
§Skills may improve within a VR learning task, but skills may worsen in a real
task.
§Behaviors may differ because VR is not perceived as real or harmful.
We cannot assume that the knowledge and insights (for example, in
the social sciences) gained in the virtual world are directly applicable
to the real world.

54. Training and Post-Training Skills
learningin thevirtual
learningin reality
applyingin realityTime
Skills

55. Training and Post-Training Skills
learningin thevirtual
learningin reality
applyingin realityTime
Skills

56. Training and Post-Training Skills
learningin thevirtual
learningin reality
applyingin realityTime
Skills

57. User Acceptance and Involvement
Occurring Problems
§the leap into the unknown
§environmental blindness
§spectator blindness
§social isolation of the user
VR installations in semi-public space are
§in the willingness to use and
§in actual use
strongly influenced by social opinion and other social factors.

58. AudienceFunnelforImmersive VR
Passer-By
a personwhoisuninvolved,
not aware, ornot paying
attentiontotheinstallation

59. AudienceFunnelforImmersive VR
Spectator
a personwho
observesthe
installationand the
headsetuser

60. AudienceFunnelforImmersive VR
ImplicitlyInteractingUser
a personwhosebehaviorisnot
primarilyfocusedon interaction
withtheinstallationbut triggers
actionsunconsciously

61. AudienceFunnelforImmersive VR
ExplicitlyInteractingUser
a personwhoconsciously
performsactionsin orderto
interactwiththeinstallation

62. AudienceFunnelforImmersive VR
Headset User
a personwhowearsan
HMD and interactswithin
thevirtual environment
Companion
a personwho
interactswiththe
personusingan HMD

63. AudienceFunnelforImmersive VR
Follow upActingUser
a personwhocontinuesto
interactwiththeinstallationon-
site orperformsoff-site actions
relatedtotheinstallation

64. StagingVR in (Semi-)publicSpace

65. I usedtheinstallationbecause...
9
1616
20
43
0
5
10
15
20
25
30
35
40
45
50
I watched the
monitor
someone dragged
me into it
someone told me
about it
I observed the
ground projection
I have seen others
interacting

66. Book Recommendation
In our new book you will find
detailed information about
Immersive Virtual Reality.
SCAN ME

67. Questions?