Last edited 27jan01 by gfrancis
The page was originally designed by
Find this document at

Homepage for the

Audible Sketchpad for the CAVE

Ours is an Research Experience for Undergraduates (REU) sponsored by the National Center for Supercomputing Applications (NCSA) at the University of Illinois (UIUC). We use the Vanilla Soundserver ( VSS) to sonify real-time interactive CAVE/console animations ( RTICA) and so create auditory CAVE Toys. Under the direction of George Francis and Umesh Thakkar , and the generous assistance of Camille Goudeseune (creator of VSS), and Matt Hall (creator of CAVE Toys), the projects webscaped here have been transformed from purely visual toys to RTICAs with interactive sound.

Sound has been available in the CAVE and used in many applications in the past, but it was never greatly developed because sound effects were not considered as important as visual effects. One reason for this was the lack of octaphonic, or 3-dimensional, sound to accompany the 3-dimensional graphics which make the CAVE unique. Octaphonic sound now has been added to the CAVE so we now can create CAVE applications that can be heard as well as seen in 3-D. An implication of this is the possibility that vision impaired people can experience the CAVE in a way analogous to the way sighted people do.

Here is a shortcut to VSS for Dummies, which is the best place to start for vss.

More recently Ben Shanbaum and Doug Nachand have been experimenting with Csound and OpenAL respectively, seeking to incorporate these packages into our project. Ben has written the beginnings of a Csound tutorial. Come back to see our progress in this direction.


Robert Acar, Matthew Woodruff, Jessica Jackson,

We are reviving the 1989 project "Kairomone" by Daniel Azzi. Ours is a sonified CAVE simulation of Diabrodica (corn root worm) beetles reacting, in a complex manner, to diffusing kairomones (chemical food attractants) wafting across a field of mature corn. This is based on the work of the late Robert L. Metcalf, UIUC professor of entomology and pioneer in integrated pest management.

Audible Cricket

Matthew Woodruff

The CAVE Toy Cricket was created by Matt Hall. Paul Whitaker stepped up the level of interaction, adding the crumbSlider for navigation controlled sound. Matt Woodruff currently is completing a fully sonified version. For discerning position, Charles Huang's tetraphony was mapped to the cricket cage with one instrument dominating a quadrant. Now Matt is adding octaphonic sound, and ball-wall collisions in xyz can be differentiated through directional amplitude.

Audible Billiards

Robert Pinta

Bob created a CAVE game of pool in the Spring'99 Math 198 course. He has now added it to the audible CAVE projects and is creating sound for billiards.

Audible Glom

Robert Pinta, Matthew Woodruff

The CAVE Toy Glom was created by Matt Hall to visualize a familiar dynamical system: Gravity. Our sonification involves a zoo of acoustic geometrical cues such as octaphonic sound, Doppler effects, beats etc.

Audible Lissajou Pad

Jessica Jackson, Matthew Woodruff

We wish to hear an n-dimensional Lissajou Figure, where n=2,3 and 4. For the 2-dimensional version we have adapted Paul Whitaker's VdPol3D project for Math198 which is a sonification of multiple, simultaneous, orbits of the Van der Pol oscillator. Instead of spirals and cyclic attractors, the Lissajou figures provide a richer sample of possible mappings of geometrical position into sound ranges. The 3-dimensional version uses the octaphonic sound system of the CAVE. Once we can train people to hear 2- and 3-dimensional Lissajou figures, which can also be visualized, we can persuade them to hear 4-dimensional Lissajou, which cannot be visualized so easily.


Ande Croll

Previous incarnations of this project were created by Steve Kohen, Dan Cranston, Ann Delano, Sherwin Tam, Our version of Jim Blinn's famous articulated puppet directs a bar of Beethoven's Ninth Symphony. Dan tells you a bit more about this project which has had many contributors over the years.

Van der Pol

Paul Whitaker

Here, sound accompanies the mapping of a phase space. Using an fmslider, frequency, amplitude, mod index, and m/c ratio can be varied with x, y-coordinates, angle, or distance. Matt Woodruff found that by varying freq. with x and MC ratio with y a person can sketch the phase space of the system.

And here we are. (and Ulises too!)