Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments

Proceedings IEEE Virtual Reality 1999 (1999), pp. 222-228, doi: 10.1109/VR.1999.756955
Experiments
Tasks
Findings

What are the effects of tactile, olfactory, audio, and visual sensory cues in a virtual environment on presence and memory of the environment?

Variables and Constants

    Auditory Fidelity
  • audio (on/off) - various audio cues through the environment vs none
    Haptic Fidelity
  • haptics (on/off) - haptic cues through a real fan and heat lamp vs none
Independent Variables
Constants

Frame rate: 20 fps

System Info

Displays
Input Hardware

Alias Wavefront modeling package; headphones; oxygen mask and pump to deliver coffee smell; pump to supply fresh air and remove coffee smell; heat lamp; fan

Participant Info

All participants were undergraduate students at the Georgia Institute of Technology who were given partial course credit for the study. Participants had either no or limited (only one time) experience with virtual environments.

Total # Age Range Gender Balance
322 -

Participants were told they were evaluating a system allowing them to see an office in a virtual environment in order to decide on renting it. Users were moved around the office at preset times, but could look around freely. In each room, participants were told to look around the room.

Interaction and Environment

Interface

Participants were told before the task that the environment could present visual, auditory, tactile, and olfactory cues, but that they would not necessarily receive any or all of the cues. Participants wore a face mask to deliver smells and headphones to deliver sounds.

The environment was an office space with several rooms. For the auditory condition, sounds of a fan, a toilet flushing, a copier, and city noise were delivered to the user at the appropriate points in the walkthrough. The copier sounds were displayed at three volume levels, increasing as the user got closer to it. Other sounds were enabled and disabled at present points. For the haptic condition, a fan was blown onto the user when a virtual fan was encountered and a heat lamp was used in a sunny outdoor balcony area. In the olfactory condition, coffee smells were delivered via the face mask in the reception area where a coffee maker could be seen. Fresh air was delivered to remove the coffee smell after the user moved away.

Dimensionality Scale Density Visual Realism
2.5D Medium High High
Metrics

  • presence - 100 point rating of overall sense of presence; 13-question questionnaire on presence
  • accuracy - 9-question questionnaire on memory of spatial layout and object location
  1. There was a significant direct effect of audio (on/off) on presence for a interactive experience of the ve task.

    Providing auditory cues increased the reported sense of overall presence and the rating of presence from the questionnaire.

    Specificity: Somewhat general

  2. There was a significant direct effect of haptics (on/off) on presence for a interactive experience of the ve task.

    Providing tactile cues increased the reported sense of overall presence and the rating of presence from the questionnaire.

    Specificity: Somewhat general

  3. There was a significant direct effect of olfaction (on/off) on accuracy for a interactive experience of the ve task.

    Providing olfactory cues increased memory of object location.

    Specificity: Somewhat general

  4. There was a significant direct effect of haptics (on/off) on accuracy for a interactive experience of the ve task.

    Providing tactile cues increased memory of object location.

    Specificity: Somewhat general