Measuring Presence in VR: Sound Design in Autonomous Vehicle-Pedestrian Communication
I undertook an independent unit of study which investigated the use of sounds for communication between autonomous vehicles and pedestrians. VR was used to simulate a number of different scenarios relating to vehicle and pedestrian interaction. Two seperate papers were created with one focusing on sound whilst my one focused on presence.
Measuring Presence in VR: Sound Design in Autonomous Vehicle-Pedestrian Communication
I undertook an independent unit of study which investigated the use of sounds for communication between autonomous vehicles and pedestrians. VR was used to simulate a number of different scenarios relating to vehicle and pedestrian interaction. Two seperate papers were created with one focusing on sound whilst my one focused on presence.
Measuring Presence in VR: Sound Design in Autonomous Vehicle-Pedestrian Communication
I undertook an independent unit of study which investigated the use of sounds for communication between autonomous vehicles and pedestrians. VR was used to simulate a number of different scenarios relating to vehicle and pedestrian interaction. Two seperate papers were created with one focusing on sound whilst my one focused on presence.
Introduction
The purpose of this research was to gain potential insights into users' feelings within a virtual environment within VR that utilises a 360-degree video. During the research phase, we examined what factors lead to certain predictions of the autonomous vehicle. We also explored how sound influenced user's perceptions towards the scenario. At the time this research was part of an iterative process and future work can investigate how to create greater relationships with AVs and pedestrians through VR. Future studies can expand our design based on our findings to improve immersion as well as improving sound design. Within these two papers I worked alongside one other student whilst the other paper was created two other students however we worked on it individually. During the data analysis phase we worked together to code interviews and then I worked with my peer to write the final presence paper.
Literature Review
This study explores the concept of presence within VR, focusing on interactions between pedestrians and AVs. Presence is understood as the perception of being within an virtual environment through encompassing physical, social and self-dimensions. While prior research explored visual elements of the experience, this study will examine the crucial role of sound to enhance the user's immersion. Within this experience, it'll aid in studying pedestrian behaviour safety. This research aims to fill the gaps in our understanding of how audio can improve AVs and pedestrian interactions. Previous existing VR studies, as well as advanced VR technologies, will help us understand pedestrian's trust with AVs to a greater extent.
Methods
The research methods for this study aimed to measure the presence in VR in the context of communication between autonomous vehicles and pedestrians. The study included 48 participants, divided into a control group of 12 with no AV sounds whilst the 36 did have AV sounds This process then had quantitative and Qualitative methods for data collection and analysis which was conducted in Dovetail through coding. During the process there were post VR questionnaires asking about presence, and prediction accuracy seeing how their predictions and behaviour were accurate. There were also observations, user interviews and thematic analysis. The video was made with a 360-degree high res camera and spatial audio then integrated in Unreal Engine. The protocol was briefing, VR calibration and 9 scenarios with predicting the scenario, questionnaire and interview.
Introduction
The purpose of this research was to gain potential insights into users' feelings within a virtual environment within VR that utilises a 360-degree video. During the research phase, we examined what factors lead to certain predictions of the autonomous vehicle. We also explored how sound influenced user's perceptions towards the scenario. At the time this research was part of an iterative process and future work can investigate how to create greater relationships with AVs and pedestrians through VR. Future studies can expand our design based on our findings to improve immersion as well as improving sound design. Within these two papers I worked alongside one other student whilst the other paper was created two other students however we worked on it individually. During the data analysis phase we worked together to code interviews and then I worked with my peer to write the final presence paper.
Literature Review
This study explores the concept of presence within VR, focusing on interactions between pedestrians and AVs. Presence is understood as the perception of being within an virtual environment through encompassing physical, social and self-dimensions. While prior research explored visual elements of the experience, this study will examine the crucial role of sound to enhance the user's immersion. Within this experience, it'll aid in studying pedestrian behaviour safety. This research aims to fill the gaps in our understanding of how audio can improve AVs and pedestrian interactions. Previous existing VR studies, as well as advanced VR technologies, will help us understand pedestrian's trust with AVs to a greater extent.
Methods
The research methods for this study aimed to measure the presence in VR in the context of communication between autonomous vehicles and pedestrians. The study included 48 participants, divided into a control group of 12 with no AV sounds whilst the 36 did have AV sounds This process then had quantitative and Qualitative methods for data collection and analysis which was conducted in Dovetail through coding. During the process there were post VR questionnaires asking about presence, and prediction accuracy seeing how their predictions and behaviour were accurate. There were also observations, user interviews and thematic analysis. The video was made with a 360-degree high res camera and spatial audio then integrated in Unreal Engine. The protocol was briefing, VR calibration and 9 scenarios with predicting the scenario, questionnaire and interview.
Introduction
The purpose of this research was to gain potential insights into users' feelings within a virtual environment within VR that utilises a 360-degree video. During the research phase, we examined what factors lead to certain predictions of the autonomous vehicle. We also explored how sound influenced user's perceptions towards the scenario. At the time this research was part of an iterative process and future work can investigate how to create greater relationships with AVs and pedestrians through VR. Future studies can expand our design based on our findings to improve immersion as well as improving sound design. Within these two papers I worked alongside one other student whilst the other paper was created two other students however we worked on it individually. During the data analysis phase we worked together to code interviews and then I worked with my peer to write the final presence paper.
Literature Review
This study explores the concept of presence within VR, focusing on interactions between pedestrians and AVs. Presence is understood as the perception of being within an virtual environment through encompassing physical, social and self-dimensions. While prior research explored visual elements of the experience, this study will examine the crucial role of sound to enhance the user's immersion. Within this experience, it'll aid in studying pedestrian behaviour safety. This research aims to fill the gaps in our understanding of how audio can improve AVs and pedestrian interactions. Previous existing VR studies, as well as advanced VR technologies, will help us understand pedestrian's trust with AVs to a greater extent.
Methods
The research methods for this study aimed to measure the presence in VR in the context of communication between autonomous vehicles and pedestrians. The study included 48 participants, divided into a control group of 12 with no AV sounds whilst the 36 did have AV sounds This process then had quantitative and Qualitative methods for data collection and analysis which was conducted in Dovetail through coding. During the process there were post VR questionnaires asking about presence, and prediction accuracy seeing how their predictions and behaviour were accurate. There were also observations, user interviews and thematic analysis. The video was made with a 360-degree high res camera and spatial audio then integrated in Unreal Engine. The protocol was briefing, VR calibration and 9 scenarios with predicting the scenario, questionnaire and interview.
Results
Overall, realism in both visual and audio aspects was crucial for immersion, however, the lack of interactivity and repetitive elements of the experience lessened the sense of presence. There were 6 components to test the user's experience of the study, this included-
Environmental Realism
Realism: Realistic 360-degree videos enhanced the sense of presence, though some participants were aware it was fake.
Location Familiarity: Familiar locations increased location (Sydney University Campus), but not for everyone.
Interactive Depth: Lack of interaction reduced presence despite visual realism.
Virtual Embodiment
Physical Body Extension: Most felt their virtual body extended into VR but lacked physical interaction.
Limited Motor Interactivity: Inadequate hand and body movement reduced immersion.
Virtual Awareness: Awareness of using a headset varied among participants, impacting immersion.
Fear of Collision: Mixed responses to fear, indicating various levels of embodiment and realism perception.
Other Pedestrians
Social Presence: Low due to lack of interaction with virtual pedestrians.
Realistic Actors: Virtual realism of pedestrians was acknowledged, though it didn't enhance social interaction.
Emotional Response
Sounds: Sounds evoked emotional responses, with "avoid" sounds perceived as calming or aggressive, and "collide" sounds inducing fear.
Technological Discomfort
Headset Issues: Blurry visuals and low video quality slightly affected immersion.
Repetitive Scenarios: Short, repetitive scenarios and frequent UI interruptions broke immersion.
Audio Realism
Sound localisation: Both Spatial and non-spatial audio groups could localise and distinguish sounds, enhancing realism.
Ambiguous Sounds: Confusing sounds like planes disrupted immersion for some.
Discussion
This study highlights the crucial role of emotional responses, interactivity, and realistic environments in enhancing VR experiences. It reveals that a similar and distinct sound evokes stronger emotional and physical reactions. Immersive interactions and environments provide a deeper sense of presence. Challenges within this study included confusion from unclear background sounds and interruptions from the interface. Future improvements should focus on refining sound design, increasing interactivity and enhancing the realism of the environment to contribute to immersion. The main points from this study include-
Emotional Response to Sound Design
The sound design in VR triggers varied emotional reactions, influencing user perception and decisions.
Participants had mixed responses to the 'avoid' sound, feeling either relaxed or fearful, based on their real-world sound associations.
The 'collide' sounds, similar to emergency sirens, had higher prediction accuracy and evoked strong physical reactions.
Familiar sounds from the real world enhance the effectiveness of VR sound design.
Interactivity and Immersion in VR
VR immersion relies heavily on interactivity and realistic body representation.
Participants reported a reduced sense of presence due to the lack of full-body interaction and lifelike visual hands.
Responses to the feat of collision varied, highlighting the need for VR designs to accommodate diverse user experiences.
Realistic virtual body parts and environments are essential for deeper user immersion.
Real-World Based Virtual Environments
VR environments that replicate real-world locations increase users' physical presence and emotional connection.
Realistic 360-degree video recordings enhanced some of the users' physical presence.
Lack of interaction with virtual objects reduced immersion for some participants.
Realistic and interactive social actors are crucial for improving VR experiences.
The Role of Virtual Environment Sounds
Background sounds are vital for creating a realistic and immersive VR atmosphere.
Most participants perceived ambient sounds, although some focused solely on the main action
An unintended plane sound caused confusion, emphasizing the need for clear and distinct sound design
Future sound designs should ensure ambient noises are distinguishable and do not interfere with key VR sounds
Enhancing Virtual Experiences Through Technology
360-degree video quality and user interface design significantly impact VR immersion.
Frequent UI interruptions and scenario repetition detracted from the immersive experience.
Users still reported high levels of immersion despite some complaints about video quality.
Improvements in video quality and scenario diversity could further enhance VR experiences.
Conclusion
Ultimately this study investigated how presence and sound influence the interactions between pedestrians and AVs in VR. Whilst participating users felt immersed within the environment which was strengthened by realistic sound and video recordings. Those in the soundless felt immersed, however, those that had sound significantly heightened their immersion and emotional responses, especially in the potential collision scenario. However, constraints were the lack of social interaction and physical embodiment affected presence. Future research aims to improve aspects such as interactivity, social presence and audio quality to exemplify immersion.
Results
Overall, realism in both visual and audio aspects was crucial for immersion, however, the lack of interactivity and repetitive elements of the experience lessened the sense of presence. There were 6 components to test the user's experience of the study, this included-
Environmental Realism
Realism: Realistic 360-degree videos enhanced the sense of presence, though some participants were aware it was fake.
Location Familiarity: Familiar locations increased location (Sydney University Campus), but not for everyone.
Interactive Depth: Lack of interaction reduced presence despite visual realism.
Virtual Embodiment
Physical Body Extension: Most felt their virtual body extended into VR but lacked physical interaction.
Limited Motor Interactivity: Inadequate hand and body movement reduced immersion.
Virtual Awareness: Awareness of using a headset varied among participants, impacting immersion.
Fear of Collision: Mixed responses to fear, indicating various levels of embodiment and realism perception.
Other Pedestrians
Social Presence: Low due to lack of interaction with virtual pedestrians.
Realistic Actors: Virtual realism of pedestrians was acknowledged, though it didn't enhance social interaction.
Emotional Response
Sounds: Sounds evoked emotional responses, with "avoid" sounds perceived as calming or aggressive, and "collide" sounds inducing fear.
Technological Discomfort
Headset Issues: Blurry visuals and low video quality slightly affected immersion.
Repetitive Scenarios: Short, repetitive scenarios and frequent UI interruptions broke immersion.
Audio Realism
Sound localisation: Both Spatial and non-spatial audio groups could localise and distinguish sounds, enhancing realism.
Ambiguous Sounds: Confusing sounds like planes disrupted immersion for some.
Discussion
This study highlights the crucial role of emotional responses, interactivity, and realistic environments in enhancing VR experiences. It reveals that a similar and distinct sound evokes stronger emotional and physical reactions. Immersive interactions and environments provide a deeper sense of presence. Challenges within this study included confusion from unclear background sounds and interruptions from the interface. Future improvements should focus on refining sound design, increasing interactivity and enhancing the realism of the environment to contribute to immersion. The main points from this study include-
Emotional Response to Sound Design
The sound design in VR triggers varied emotional reactions, influencing user perception and decisions.
Participants had mixed responses to the 'avoid' sound, feeling either relaxed or fearful, based on their real-world sound associations.
The 'collide' sounds, similar to emergency sirens, had higher prediction accuracy and evoked strong physical reactions.
Familiar sounds from the real world enhance the effectiveness of VR sound design.
Interactivity and Immersion in VR
VR immersion relies heavily on interactivity and realistic body representation.
Participants reported a reduced sense of presence due to the lack of full-body interaction and lifelike visual hands.
Responses to the feat of collision varied, highlighting the need for VR designs to accommodate diverse user experiences.
Realistic virtual body parts and environments are essential for deeper user immersion.
Real-World Based Virtual Environments
VR environments that replicate real-world locations increase users' physical presence and emotional connection.
Realistic 360-degree video recordings enhanced some of the users' physical presence.
Lack of interaction with virtual objects reduced immersion for some participants.
Realistic and interactive social actors are crucial for improving VR experiences.
The Role of Virtual Environment Sounds
Background sounds are vital for creating a realistic and immersive VR atmosphere.
Most participants perceived ambient sounds, although some focused solely on the main action
An unintended plane sound caused confusion, emphasizing the need for clear and distinct sound design
Future sound designs should ensure ambient noises are distinguishable and do not interfere with key VR sounds
Enhancing Virtual Experiences Through Technology
360-degree video quality and user interface design significantly impact VR immersion.
Frequent UI interruptions and scenario repetition detracted from the immersive experience.
Users still reported high levels of immersion despite some complaints about video quality.
Improvements in video quality and scenario diversity could further enhance VR experiences.
Conclusion
Ultimately this study investigated how presence and sound influence the interactions between pedestrians and AVs in VR. Whilst participating users felt immersed within the environment which was strengthened by realistic sound and video recordings. Those in the soundless felt immersed, however, those that had sound significantly heightened their immersion and emotional responses, especially in the potential collision scenario. However, constraints were the lack of social interaction and physical embodiment affected presence. Future research aims to improve aspects such as interactivity, social presence and audio quality to exemplify immersion.
Results
Overall, realism in both visual and audio aspects was crucial for immersion, however, the lack of interactivity and repetitive elements of the experience lessened the sense of presence. There were 6 components to test the user's experience of the study, this included-
Environmental Realism
Realism: Realistic 360-degree videos enhanced the sense of presence, though some participants were aware it was fake.
Location Familiarity: Familiar locations increased location (Sydney University Campus), but not for everyone.
Interactive Depth: Lack of interaction reduced presence despite visual realism.
Virtual Embodiment
Physical Body Extension: Most felt their virtual body extended into VR but lacked physical interaction.
Limited Motor Interactivity: Inadequate hand and body movement reduced immersion.
Virtual Awareness: Awareness of using a headset varied among participants, impacting immersion.
Fear of Collision: Mixed responses to fear, indicating various levels of embodiment and realism perception.
Other Pedestrians
Social Presence: Low due to lack of interaction with virtual pedestrians.
Realistic Actors: Virtual realism of pedestrians was acknowledged, though it didn't enhance social interaction.
Emotional Response
Sounds: Sounds evoked emotional responses, with "avoid" sounds perceived as calming or aggressive, and "collide" sounds inducing fear.
Technological Discomfort
Headset Issues: Blurry visuals and low video quality slightly affected immersion.
Repetitive Scenarios: Short, repetitive scenarios and frequent UI interruptions broke immersion.
Audio Realism
Sound localisation: Both Spatial and non-spatial audio groups could localise and distinguish sounds, enhancing realism.
Ambiguous Sounds: Confusing sounds like planes disrupted immersion for some.
Discussion
This study highlights the crucial role of emotional responses, interactivity, and realistic environments in enhancing VR experiences. It reveals that a similar and distinct sound evokes stronger emotional and physical reactions. Immersive interactions and environments provide a deeper sense of presence. Challenges within this study included confusion from unclear background sounds and interruptions from the interface. Future improvements should focus on refining sound design, increasing interactivity and enhancing the realism of the environment to contribute to immersion. The main points from this study include-
Emotional Response to Sound Design
The sound design in VR triggers varied emotional reactions, influencing user perception and decisions.
Participants had mixed responses to the 'avoid' sound, feeling either relaxed or fearful, based on their real-world sound associations.
The 'collide' sounds, similar to emergency sirens, had higher prediction accuracy and evoked strong physical reactions.
Familiar sounds from the real world enhance the effectiveness of VR sound design.
Interactivity and Immersion in VR
VR immersion relies heavily on interactivity and realistic body representation.
Participants reported a reduced sense of presence due to the lack of full-body interaction and lifelike visual hands.
Responses to the feat of collision varied, highlighting the need for VR designs to accommodate diverse user experiences.
Realistic virtual body parts and environments are essential for deeper user immersion.
Real-World Based Virtual Environments
VR environments that replicate real-world locations increase users' physical presence and emotional connection.
Realistic 360-degree video recordings enhanced some of the users' physical presence.
Lack of interaction with virtual objects reduced immersion for some participants.
Realistic and interactive social actors are crucial for improving VR experiences.
The Role of Virtual Environment Sounds
Background sounds are vital for creating a realistic and immersive VR atmosphere.
Most participants perceived ambient sounds, although some focused solely on the main action
An unintended plane sound caused confusion, emphasizing the need for clear and distinct sound design
Future sound designs should ensure ambient noises are distinguishable and do not interfere with key VR sounds
Enhancing Virtual Experiences Through Technology
360-degree video quality and user interface design significantly impact VR immersion.
Frequent UI interruptions and scenario repetition detracted from the immersive experience.
Users still reported high levels of immersion despite some complaints about video quality.
Improvements in video quality and scenario diversity could further enhance VR experiences.
Conclusion
Ultimately this study investigated how presence and sound influence the interactions between pedestrians and AVs in VR. Whilst participating users felt immersed within the environment which was strengthened by realistic sound and video recordings. Those in the soundless felt immersed, however, those that had sound significantly heightened their immersion and emotional responses, especially in the potential collision scenario. However, constraints were the lack of social interaction and physical embodiment affected presence. Future research aims to improve aspects such as interactivity, social presence and audio quality to exemplify immersion.