Using haptic cues to aid nonvisual structure recognition

ACM Transactions on Applied Perception (2008), pp. 1-14, doi: 10.1145/1279920.1279922
Experiments
Tasks
Findings

Are audio or haptic cues better for object selection and recognition in a non-visual environment? Where is audio better than haptic and vice versa, and where is both better than one alone?

Variables

    Auditory Fidelity
  • audio (on/off) - different sounds representing differing objects for recognition are either present or not present.
    Haptic Fidelity
  • haptics (on/off) - haptic force cues and guidance for object selection and recognition are either present or not present.
Independent Variables
  • spatial node mixture - the amount of sphere and cube nodes in the environment changes, but the total nodes remains 4 throughout.
  • audio cues - sounds played representing different nodes (sphere or cube) are turned on and off.
  • haptic cues - haptic guidance to another node and sensing of surface structure is turned on and off.
System Info

Displays
None
Input Hardware

Participant Info

Right-handed, sighted participants that were each paid £5.

Total # Age Range Gender Balance
9 25 - 30

First, participants completed a practice session in which they could use both haptic and audio cues. Participants were asked to search for nodes in the environment, press the space bar immediately after identifying a node in order to time how long it took to identify the node, and announce what type of node it was. When participants had identi?ed all of the nodes, they were free to continue exploring the environment until they were con?dent of the spatial relationships between the nodes. They then exited the virtual environment and drew the overall structure on a piece of paper.

Interaction and Environment

Interface

Participants sat in front of the Omni, holding the stylus in their right hand, and pressed the space bar to start the session. In the experimental sessions, if haptic cues were available, the space bar would guide them to the left-most node, which they would then identify, speaking their answer out loud to the investigator. The investigator noted down whether the answer was correct. Pressing the space bar again would guide the stylus to the next node. If haptic cues were not available, participants had to independently search the environment.

Dimensionality Scale Density Visual Realism
3D Small Controlled variable Medium
Metrics

  • accuracy - observes comparison of spatial structure user drawings and keys, comparing relative distances (all should have been integer units apart) and angles (180, 90, or 45) between nodes for correctness.
  • errors - how many nodes were correctly identified based on type.
  • time - how fast each node was recognized.
  1. There was a significant interaction between audio (on/off), haptics (on/off), spatial node mixture, audio cues, and haptic cues on time for a comprehension of spatial information and travel - exploration task.

    In the condition where haptic and audio cues were available together, participants identi?ed both nodes and structures fastest, and identi?ed nodes more accurately, than in any of the other conditions. The advantage the cues provided was highly signi?cant

    Specificity: Somewhat specific

  2. There was a significant interaction between audio (on/off), haptics (on/off), audio cues, and haptic cues on errors for a comprehension of spatial information and object selection task.

    The percentage of structures correctly identi?ed stood at 75% for the haptic-only and haptic and audio conditions, compared to 28% and 25% for the no-cue and audio-only conditions

    Specificity: Somewhat specific

  3. There was a significant interaction between audio (on/off), haptics (on/off), spatial node mixture, audio cues, and haptic cues on errors for a comprehension of spatial information, object selection, and travel - exploration task.

    100% of participants rated audio cues as useful for recognizing nodes, compared to 78% rating haptic cues as useful. Of participants, 100% also rated haptic cues as useful for recognizing structure, compared to 44% rating audio cues as useful. The large p

    Specificity: Highly specific