RESEARCH General Arts

MUSIC
Virtual reality environment to give feedback to student conductors

Melissa Mitchell, Arts Editor
(217) 333-5491; melissa@uiuc.edu

4/1/2001

Photo by Bill Wiegand Music professor Guy Garnett and a team of multidisciplinary researchers at the UI are attempting to change the way students practice conducting by creating a virtual conducting environment.

CHAMPAIGN, Ill. -- When observing an orchestral or choral performance, audience members may be tempted to conclude that of all the on-stage performers, the conductor has the easiest job. After all, from the vantage point of the audience, the conductor appears to be doing little more than waving a baton.

In reality, of course, there's much more to the art of conducting.
The movements of the conductor's right hand is often most obvious to the audience. But, according to University of Illinois music professor Guy Garnett, all manner of information is communicated by the conductor to the musicians through a complex range of gestures. Signals are sent not only by the conductor's right hand, but through the movement of the left hand and gaze.

To date, students of conducting can only practice their craft during class time or in front of a mirror. But Garnett and a team of multidisciplinary researchers at the UI are attempting to change that. The researchers are pooling their talents on a project called "The Intelligent Virtual Ensemble (IVE): Creating an Infrastructure for Natural Interactive Skills Acquisition." The overall goal of the work, Garnett said, is "to build a virtual conducting-training environment that will help conducting students learn the craft."

Aided by IVE, conducting students will one day be able to gain valuable practice experience by conducting virtual musical ensembles in 3-D CAVE or CAVE-like environments. In these specially equipped environments, the students' movements and gestures can be tracked and recorded by computers, interpreted through the use of aural and avatar responses. Avatars are visual icons, which, in this case, would represent members of the virtual orchestra. The simulations would provide budding composers with important feedback about their performance.

"With an infinitely patient virtual ensemble, novices could practice an exercise as many times as needed," Garnett said, adding that advanced students would benefit from increased opportunities to practice coordinating multiple gestures for cueing, dynamics and articulation. And IVE would give them access to an ensemble at any time, without the difficulty or expense of assembling live musicians.

"Creating a virtual environment for teaching humans the skills necessary for conducting also will serve as an anchor point for extending current technology and creating new technology critical to the broader development of virtual environments and natural human-computer interaction," Garnett said.

Other members of the IVE research team include Karen Ruhleder, research professor in the Graduate School of Library and Information Science; Klara Nahrstedt, professor of computer science; Rachael Brady, technical program manager, and Hank Kaczmarski, director of the integrated systems laboratory, Beckman Institute of Science and Technology; and Fred Stoltzfus, professor of music.