Auditory Display
For Human Computer Interfacing

October 20, 2008

Copyright 1997-2008, Perry R. Cook, Princeton University


* Permission to make digital or hard copies of part or all
of this work for personal or classroom use is granted with
or without fee provided that copies are not made or dis-
tributed for profit or commercial advantage and that copies
bear this notice and full citation on the first page. To copy
otherwise,to republish, to post on services, or to redistribute
to lists, requires specific permission and/or a fee.


First, Some Terms: The Senses

I. Visual
    N: Vision
    V: Visualize, Image, View, Display

II. Auditory
    N: Hearing, Audition
    V: Hear, Listen, Play, Present, Sound

III. Haptic
    N: Kinesthesis + Taction
    V: Feel, Touch, Manipulate

IV. Olfactory
    N: Olfaction, Smell
    V: Smell, Sniff, ..

V. Gustatory
    Yummy! (Loaded with Metaphors)



Auditory Display Terms

I. Auralization
    Realistic "rendering" of sonic environments.

II. Auditory Icons

    Real World Sound Events as Signals

III. Earcons

    Abstract, Hierarchical Sonic Grammar

IV. Audification/Sonification

    Data --> Sound



I. Auralization

    Realistic "rendering" of sonic environments.
    Audio "Ray Tracing" from source(s)
    to listener(s), including effects
    of diffraction, diffusion, etc.

    (Or just another confused definition
    of Audification, Sonification, Scientific
    Auditory Display, etc.)


II. Auditory Icons

Use of recordings of real-world sounds to
signal events. Real-world sounds, if selected
correctly, can carry lots of intrinsic meaning
because of our experience with them.

Typical System: Gaver's Sonic Finder System

Examples: Glass Breaking for error, Rooster Crowing
for schedule alarms, "Yippee" for successful compile,
"NDope" for errors on compile, etc.



Problems with real-world sounds are that they don't
mean the same thing to all people (like icons too),
they can become tiresome, sometimes they take more
time to play than the information they carry, etc.


III. Earcons

More abstract (than auditory icons) sonic events.
These are Hierarchical, and can be concatenated
and mixed to build up complex meanings.

Examples:
The work of Meera Blattner (the inventor of the term Earcon)
and Stephen Brewster's page and thesis on Earcons.

Here's Perry's Earcon Designs for Tom Pirelli's Arial Home:

Dog coming in
Dog leaving
Lawyer arriving
Lawyer leaving
Lawyer on phone
Dog on phone

IV. Sonification

The mapping of data relationships to auditory relationships
for the purpose of communicating and/or comprehending
relations in the domain under study.

Some Existing Auditory Display Systems

CAITLIN: A Musical Program Auralisation Tool to
Assist Novice Programmers with Debugging
Department of Computer Studies, Loughborough University

ADSL: An Auditory Domain Specification Language
for Program Auralization
Syracuse University

LSL: A Specification Language for Program Auralization
Purdue University

Sonnet: Audio-Enhanced Monitoring and Debugging
IBM Watson

FAUST: A Framework for Algorithm Understanding
and Sonification Testing
Princeton

LISTEN: Sounding Uncertainty Visualization
MUSE: A Musical Data Sonification Toolkit
University of California Santa Cruz


V. Some Psychoacoustics

Human hearing is sensitive to (in rough order):
  • Frequency, Pitch
      50 Hz. to 4KHz
      0.5% changes perceived
  • Time
      0.2 events/second to 20 events/second
      4% changes perceived
  • Spatial Location
      360 degrees in the plane of our ears,
      above and below, and distance too.
      4-6 degrees, depending on nature of sound.
  • Intensity, Loudness
      100 dB (factor of 10,000,000,000)
      20% changes perceived
  • Timbre
      Defined as "everything which is not
      pitch or loudness"
      Impulsive vs. sustained, nasal, bright, ...
  • Voice Quality
      breathy, creaky, strained, ...

Web References:

Home Page of the International Community of Auditory Display
Specifically check the Annotated Bibliography Here.

Some Non-Web References Made from Dead Trees:

"Auditory Display: Sonification, Audification, and Auditory Interface"
Santa Fe Institute Studies in the Sciences of Complexity
Proc. Vol. XVIII
Reading, MA: Addison-Wesley, 1994.

"Multimedia Interface Design"
M. Blattner and R. Dannenberg eds.
Reading, MA: ACM Press/Addison-Wesley, 1992.

"Auditory User Interfaces"
T. V. Raman
Kluwer Academic Publishers, Boston, 1997.

Wanna Know How to Synthesize Sound?

Back to Princeton General Information Section