Application of Wave Field Synthesis
in electronic music and sound art
The limitation of stereo or ambisonic techniques is that it only works perfectly well for one listener, who is positioned on the so-called "sweet spot". Obviously, in common concert environments the intended effect of movement of the sound will in these cases not be heard by a majority of the listeners.
Wave field synthesis is a technique that can overcome the
limitation of only working well for one "sweet spot"
and can provide a good perceptual localisation in a relatively
large listening area. This makes the techique ideal for concert
Wave Field Synthesis
An interesting feature is that it is also possible to synthesize a sound source in front of the speakers (Jansen 1997), which is not possible with other techniques.
Comparisons between measured wave fields and wave fields reconstructed with WFS have shown that the differences between the two are small (Bourdillat 2001); most faults in the WFS reproduction were due to reflections in the reproduction room. Perceptual experiments and practical experience have shown that with WFS one can achieve a large listening area, where the sound source is perceived correctly at the specified location (Vogel 1993, Verheijen 1998). Malham's (2001) comments that WFS cannot achieve a perfect sound image on all locations are true, but perceptually not so relevant that it makes the technique not worth considering for application in spatialisation of electronic music.
Jansen (1997) derived mathematical formulae for synthesising moving sound sources. He took into account the Doppler effect and showed that for its application one would need to have continuously time-varying delays. He also showed that for slowly moving sources the Doppler effect is negligible and one can resort to updating locations and calculating filters for each location and changing those in time.
This approach was chosen in this project. Additionally, in order to avoid clicks in playback, an option was built in to fade between two locations to make the movement sound smoother.
System setup at the TU Berlin
The system consists of a LINUX PC, driving 24 loudspeakers with an RME Hammerfall Soundcard.
The loudspeaker signals are calculated in real time with the program BruteFIR by Torger (2001-3). This program is capable of making convolutions with long filters in realtime. The filter coefficients can be calculated with the interface software described in this paper.
The current system is capable of playing maximal 9 sound sources with different locations in realtime, even when the sources are moving. This is the maximum amount of sources; the exact amount of sources that can be used in a piece depend on the maximum distance range of each source and the amount of reflections added. Both of these aspects influence the total filter length and the filter length determines the amount of calculation power needed.
The main idea is that the composer or sound artist is bothered as little as possible by the actual calculation of filter coefficiënts. The program makes these calcations and lets the user just define the locations he wants his sources to be. This lets the composer think about movements and locations of sounds, instead of having to be a physicist. The advantage is also that the composer is not dependent on what the array looks like exactly. His composition can be played back on another WFS-system with another setup.
The program is roughly divided in three parts: a composition tool, a grid definition tool, and a play-function. Each of these, I will discuss separately.
Experiences with composers
WONDER- Composition tool
For each source, the user can set the type of source (a point source having a specific location or a plane wave having only a direction), whether it is moving or stationary, its location or angle, the sound input channel at which the sound will be supplied and in the case of a point source, whether high frequency damping of the air has to be taken into account and whether reflections have to be calculated or not. If reflections have to be calculated, room characteristics can be defined (these can be different for each source and even vary in time). In the case of a moving source, one can define a path through space and choose to let the movement loop along the path. All input can be either typed in, or drawn with the mouse on a graphical overview. In figure 4 a screenshot of the source definition dialog is given.
After supplying all information and storing it, the user can test his input with the program and after that make the calculations for the filters. He can then store the filters for all positions calculated and a score for playback.
For the movement of the sounds, one can set the number of breakpoints along the path and a fade order. A breakpoint is an intermediary point on a path; movement is created by switching from one breakpoint to another. By using a fade between succesive breakpoints, the movement can become smoother and possible clicks in playback can become softer. The user can choose to let the amount of breakpoints on each segment be calculated automatically. In that case, the program uses a maximum total of 128 breakpoints per source and divides these over the segments of the path, depending on the length of the segment and of the path and on the time interval. The total number of breakpoints that the program will use can be determined per source by the user.
In practice, one needs to experiment with the optimal settings for the amount of breakpoints and the fade order in order to bring clicks to an acceptable level. Whether clicks are audible also depends on the type of sound that is moving. Sounds with a narrow frequency band, tend to create more clicks when moving than broadband signals.
In some instances one cannot get rid of the clicks altogether as BruteFIR has a minimum time after which it can update filter coefficients. The exact time depends on the filter length or block size. In the program the minimum time step was set to 200 ms. between breakpoints and to 50 ms. for a fade step.
WONDER- Grid definition tool
The tool then calculates a grid of points, according to the input of the user, and shows these points on the screen. If the user is content with the points calculated, he can close the window and let the program calculate the filters.
WONDER- Play function
The graphical interface of the program is mainly made to test the grid defined, allowing you to let the sources move from grid point to grid point, and to play back a composition. There is also a recording function included, which allows you to record the movements you create according to your input. A screenshot of the user interface is given in figure 6.
For real time control over the system, the Open Sound Control protocol (Wright et.al. 2003) is being implemented. This will enable the user to use any composition or live performance program or hardware that can send out OSC messages to control the movements of his sounds on the WFS-system. In the next few months, this option will be implemented.
Pingpong Ballet - Marc Lingk
Pollock's Sprechwunsch - Marije Baalman
As the movements were relatively slow and the sound was not very dense, the movements and different positions of the sound could be heard quite clearly.
These two examples show that with WFS it is possible to create more complex paths through space than is possible with most other spatialisation techniques.
Restored to Life - Ilka Theurich
One sound was placed in a rather small room with fully reflecting walls. This resulted in a sound that was virtually at several locations (due to the mirror image source model). As the sound from the actual source location was the first sound to reach the listeners' ear, the sound would however still be located there by the listener.
Other sounds were placed in a larger room, while others were moving without being placed in a room. One of the sources was created as a plane wave, which allowed the listener to get different perspectives on the composition by moving through the listener area. The plane wave sound only had a direction and as such was always in front of one, with a specific angle, whereas the other sounds had clearly defined locations. While the listener moved, the plane wave sound would "walk along", while the point source sounds stay fixed in their position. In this way the listener could determine his own version of the composition by chosing his own location.
The effect of the movement and reflections were the most clear for recorded sounds (having a rich spectrum), as opposed to synthetic sine-based tones.
In order to limit the CPU-load, some compromises had to be made: the total amount of reflections calculated was reduced.
During the work the idea came up to enable the room characteristics to change in time, which possibly can also provide an interesting effect. This was implemented in the current version.
Beurskrach - Marije Baalman, with video by Julius Stahl
Julius Stahl made a video to accompany the piece. The video makes use of pictures of the same object that the sounds were from; as the sounds are alienating themselves from the original sound during the composition, in the images the object becomes clearer. The video is rendered live and has slight variations, each time it is played. In figure 10 three screenshots from the video is given.
The hall in which the concert took place measured about 105 square meters and was relatively reverberant. The array was positioned on the stage a little bit above ear height.
The concert was preceded by a short presentation explaining the wave field synthesis technique and the software that the composers used to create the movements of their sounds.
During the concert, the biggest problem was that the system with its 24 loudspeakers could not create enough loudness for the amount of people who filled the hall (ca. 100 listeners). This had as a disadvantage that the people in the back could not perceive the music very well and were a bit loud as they started to talk. During the sound check (without the sound absorbing people in the hall) the system was loud enough for the whole hall and the effect was even clear in the back of the hall.
For the presentation of a prototype system the concert can be regarded rather as a success. The listeners who were in the front could perceive very well the movements of the sounds in the compositions. Especially when closing the eyes, some people commented that the music created a vivid visual image with its movements through space. Others were quite amazed that they could really move around the source, that is, position themselves on a different relative location to the virtual source. A sound artist, who works a lot with ambisonics, commented that especially the distance of various sources can be much better modelled with WFS than with ambisonics.
The pieces of Lingk, Lippok and Baalman were received best, as the movements of the sounds in these pieces were the clearest. This is probably due to the type of sounds that they used, which all had a broad frequency spectrum, thus enabling listeners to locate the sound more clearly.
Some listeners were disappointed, as the system was not yet a full surround system.
After the concert several other composers showed an interest in applying the system for their own work, varying from electronic music concerts, to sound installations, to a combination of electronic music with dance.
At the Electrofringe festival (at the 4th of October 2003 in Newcastle, Australia) a setup of 16 loudspeakers was used, that had more power than the speakers in Berlin. Here, there were no loudness problems. The room was about 80 square meters large. The concert was preceded by a two hour workshop about the system and the work of the composers, with sound examples. The amount of listeners at the concert was around 100. Here also there were a lot of positive reactions from the public.
After that the main focus will be to let composers and sound artists work with the system, so that we know where the interface needs to be improved and in which direction the technique can be further developed. I plan to make two sound installations with the system, that will show the possibilities of the system.
To Electrofringe for giving the opportunity to present the work on their festival and to the Goethe Institut Sydney for financial support to enable me to do so.
To the Sound Control group of the University of Technology of Delft for using their figures.
Marije A.J. Baalman
Neue Hochstrasse 56