Marije A.J. Baalman
Application of Wave Field Synthesis
in electronic music and sound art

~

Introduction
In this article I want to tell you about my work on Wave Field Synthesis, a project that I have been working on since October 2002 in the Electronic Studio of the University of Technology of Berlin, Germany. I started working on this project after I finished my Master's degree in Applied Physics at the University of Technology of Delft, The Netherlands, and completing the Sonology-Course at the Royal Conservatory of The Hague. For me, the project is a good way to combine both my scientific knowledge and my interest in electronic music. The aim of the project is to make a software interface, to allow composers and sound artists to use the spatialisation technique of Wave Field Synthesis. The studio in Berlin provides for a good environment to work on this project, as it has the technical requirements and also the contact to composers, so that they can be involved in the development process. I think this is a good thing, as in this way their wishes and ideas can be taken along in the process and create new impulses for the research on Wave Field Synthesis itself.

Spatialisation
Spatialisation has been a topic of interest in the development of electronic music since the 1950's; common techniques make use of quadraphonic or octaphonic setups and are based on providing localisation cues based on psycho-acoustics or acoustics (e.g. Chowning 1971). The technique of ambisonics has become popular since the 1990's (e.g. Malham and Matt 1995). There are also various examples where more loudspeakers are used, mostly as setups for one specific piece or location and not as a standardardized setup. A detailed historical overview of spatialisation techniques can be found in Malham & Matt (1995).

The limitation of stereo or ambisonic techniques is that it only works perfectly well for one listener, who is positioned on the so-called "sweet spot". Obviously, in common concert environments the intended effect of movement of the sound will in these cases not be heard by a majority of the listeners.

Wave field synthesis is a technique that can overcome the limitation of only working well for one "sweet spot" and can provide a good perceptual localisation in a relatively large listening area. This makes the techique ideal for concert environments.
Its increasing popularity in audio engineering shows that it is not unlikely that the technique will be available in concert halls and becomes affordable for studios in the near future (see CARROUSO ).

Wave Field Synthesis
The concept of Wave Field Synthesis (WFS) is based on a principle that was thought of in the 17th century by the Dutch physicist Huygens (1690) about the propagation of waves. He stated that when you have a wavefront, you can synthesize the next wavefront by imagining on the wavefront an infinite number of small sources, whose waves will together form the next wavefront (figure 1).
Based on this principle, Berkhout (1988) introduced the wave field synthesis principle in acoustics.
By using a discrete, linear array of loudspeakers (figure 2), one can synthesize correct wavefronts in the horizontal plane (Berkhout, De Vries and Vogel 1993). For a complete mathematical treatment is referred to Berkhout (1988, 1993) and various other papers and theses from the TU Delft .

An interesting feature is that it is also possible to synthesize a sound source in front of the speakers (Jansen 1997), which is not possible with other techniques.

Comparisons between measured wave fields and wave fields reconstructed with WFS have shown that the differences between the two are small (Bourdillat 2001); most faults in the WFS reproduction were due to reflections in the reproduction room. Perceptual experiments and practical experience have shown that with WFS one can achieve a large listening area, where the sound source is perceived correctly at the specified location (Vogel 1993, Verheijen 1998). Malham's (2001) comments that WFS cannot achieve a perfect sound image on all locations are true, but perceptually not so relevant that it makes the technique not worth considering for application in spatialisation of electronic music.

Jansen (1997) derived mathematical formulae for synthesising moving sound sources. He took into account the Doppler effect and showed that for its application one would need to have continuously time-varying delays. He also showed that for slowly moving sources the Doppler effect is negligible and one can resort to updating locations and calculating filters for each location and changing those in time.

This approach was chosen in this project. Additionally, in order to avoid clicks in playback, an option was built in to fade between two locations to make the movement sound smoother.

System setup at the TU Berlin
The prototype system in Berlin was created with the specific aim to make a system for the use in electronic music (Weske 2001).

The system consists of a LINUX PC, driving 24 loudspeakers with an RME Hammerfall Soundcard.

The loudspeaker signals are calculated in real time with the program BruteFIR by Torger (2001-3). This program is capable of making convolutions with long filters in realtime. The filter coefficients can be calculated with the interface software described in this paper.

The current system is capable of playing maximal 9 sound sources with different locations in realtime, even when the sources are moving. This is the maximum amount of sources; the exact amount of sources that can be used in a piece depend on the maximum distance range of each source and the amount of reflections added. Both of these aspects influence the total filter length and the filter length determines the amount of calculation power needed.

Interface software
The interface software has been in constant development since I worked on the project. Starting out as just a composition tool for the movements, it is now capable of both live performance and as a composition tool.

The main idea is that the composer or sound artist is bothered as little as possible by the actual calculation of filter coefficiënts. The program makes these calcations and lets the user just define the locations he wants his sources to be. This lets the composer think about movements and locations of sounds, instead of having to be a physicist. The advantage is also that the composer is not dependent on what the array looks like exactly. His composition can be played back on another WFS-system with another setup.

The program is roughly divided in three parts: a composition tool, a grid definition tool, and a play-function. Each of these, I will discuss separately.

Experiences with composers
For the Club Transmediale festival seven pieces for the system were prepared by seven composers: Frieder Butzmann, Boris Hegenbart, Marc Lingk, Robert Lippok, Markus Schneider, Ilka Theurich and Marije Baalman. All composers had different backgrounds; all had previously composed electronic music. I will elaborate on three works that were created, as well as on one piece which was created for the Electrofringe festival.

WONDER- Composition tool
The user can define various sources, each with their own characteristics. A source in this context is the virtual source from which sound emanates in space, whose spatial parameters can be given by the user.

For each source, the user can set the type of source (a point source having a specific location or a plane wave having only a direction), whether it is moving or stationary, its location or angle, the sound input channel at which the sound will be supplied and in the case of a point source, whether high frequency damping of the air has to be taken into account and whether reflections have to be calculated or not. If reflections have to be calculated, room characteristics can be defined (these can be different for each source and even vary in time). In the case of a moving source, one can define a path through space and choose to let the movement loop along the path. All input can be either typed in, or drawn with the mouse on a graphical overview. In figure 4 a screenshot of the source definition dialog is given.

After supplying all information and storing it, the user can test his input with the program and after that make the calculations for the filters. He can then store the filters for all positions calculated and a score for playback.

For the movement of the sounds, one can set the number of breakpoints along the path and a fade order. A breakpoint is an intermediary point on a path; movement is created by switching from one breakpoint to another. By using a fade between succesive breakpoints, the movement can become smoother and possible clicks in playback can become softer. The user can choose to let the amount of breakpoints on each segment be calculated automatically. In that case, the program uses a maximum total of 128 breakpoints per source and divides these over the segments of the path, depending on the length of the segment and of the path and on the time interval. The total number of breakpoints that the program will use can be determined per source by the user.

In practice, one needs to experiment with the optimal settings for the amount of breakpoints and the fade order in order to bring clicks to an acceptable level. Whether clicks are audible also depends on the type of sound that is moving. Sounds with a narrow frequency band, tend to create more clicks when moving than broadband signals.

In some instances one cannot get rid of the clicks altogether as BruteFIR has a minimum time after which it can update filter coefficients. The exact time depends on the filter length or block size. In the program the minimum time step was set to 200 ms. between breakpoints and to 50 ms. for a fade step.

WONDER- Grid definition tool
The grid definition tool is meant for creating a setup for live performance with the system. It lets the user define a grid of points across the space he wants to use for his sources to move around. The grid can be divided in different segments and the way of spacing the grid points can be done in two ways: either in a cartesian or a polar way. For each segment of the grid, the user can determine to calculate reflections or not and whether or not to take high frequency damping into account. The input can again be given either graphically or typed. It is also possible to choose a background image from a file, to enable the user to define a grid points based on an image. In figure 5 a screenshot of the grid definition dialog is given.

The tool then calculates a grid of points, according to the input of the user, and shows these points on the screen. If the user is content with the points calculated, he can close the window and let the program calculate the filters.

WONDER- Play function
The play function forms the main part of the program. In order to start playing, you first need to initialise the system, so that the program knows which grid it is using (it is even possible to choose more than one grid file) and knows about the hardware setup. You have to define how many sources you are using and tell the program where the convolution engine (BruteFIR) is located on your system and where you want to store the temporary file that the program needs to communicate with the engine. If the initialisation is succesfull, the convolution engine can be started. The dialogs for initialisation are slightly different, depending whether you want to playback a composition, or whether you want to set the system up for live performance.

The graphical interface of the program is mainly made to test the grid defined, allowing you to let the sources move from grid point to grid point, and to play back a composition. There is also a recording function included, which allows you to record the movements you create according to your input. A screenshot of the user interface is given in figure 6.

For real time control over the system, the Open Sound Control protocol (Wright et.al. 2003) is being implemented. This will enable the user to use any composition or live performance program or hardware that can send out OSC messages to control the movements of his sounds on the WFS-system. In the next few months, this option will be implemented.

Pingpong Ballet - Marc Lingk
Marc Lingk, a composer residing in Berlin, wrote a piece called Ping-Pong Ballet. The sounds for this piece were all made from ping-pong ball sounds, which were processed by various algorithms, alienating the sound from its original. Using these sounds as a basis, the inspiration for the movements was relatively easy as the ping-pong ball game provides a good basis for the distribution in space of the sounds. In this way he created various loops of movement for the various sounds as depicted in figure 7. Paths 1 & 2 are the paths of the ball bouncing on the table, 3 & 4 of the ball being hit with the bat, 5 & 6 of multiple balls bouncing on the table, 7 & 8 of balls dropping to the floor. Choosing mostly prime numbers for the loop times, the positions were constantly changing in relative distance to each other. The movement was relatively fast (loop times were between 5 and 19 seconds). In the beginning, the piece gives the impression of a ping-pong ball game, but as it progresses the sounds become more and more dense, creating a clear and vivid spatial sound image.

Pollock's Sprechwunsch - Marije Baalman
I made a piece where the movements were based on a painting created before in a rather improvisational way. The different colours in the painting were mapped to different sounds and they also had different movement characteristics, as seen in figure 8. One source was moving perpendicular to the array, another parallel to the array. Yet another was zigzagging to and from the array, one source was jumping from one location to another. The other two sources had other types of paths that are less easily stereotyped. The exact movement in time was made dependent on the sounds. Silences on the sound input were used to let the virtual source jump to another position for the next sound to start its path.

As the movements were relatively slow and the sound was not very dense, the movements and different positions of the sound could be heard quite clearly.

These two examples show that with WFS it is possible to create more complex paths through space than is possible with most other spatialisation techniques.

Restored to Life - Ilka Theurich
Ilka Theurich, a student of sound sculpture in Hannover, was interested most by the possibilities of including virtual rooms and reflections into the composition.

One sound was placed in a rather small room with fully reflecting walls. This resulted in a sound that was virtually at several locations (due to the mirror image source model). As the sound from the actual source location was the first sound to reach the listeners' ear, the sound would however still be located there by the listener.

Other sounds were placed in a larger room, while others were moving without being placed in a room. One of the sources was created as a plane wave, which allowed the listener to get different perspectives on the composition by moving through the listener area. The plane wave sound only had a direction and as such was always in front of one, with a specific angle, whereas the other sounds had clearly defined locations. While the listener moved, the plane wave sound would "walk along", while the point source sounds stay fixed in their position. In this way the listener could determine his own version of the composition by chosing his own location.

The effect of the movement and reflections were the most clear for recorded sounds (having a rich spectrum), as opposed to synthetic sine-based tones.

In order to limit the CPU-load, some compromises had to be made: the total amount of reflections calculated was reduced.

During the work the idea came up to enable the room characteristics to change in time, which possibly can also provide an interesting effect. This was implemented in the current version.

Beurskrach - Marije Baalman, with video by Julius Stahl
In August and September 2003, I made another composition, to be played on the Electrofringe festival, in Newcastle, Australia. As the program had been developed further, I had new options available to use. In the composition, called "Beurskrach", four sources were defined, that were regarded as being four points on one virtual object, making a movement together. As the sound sources were different variations on the same sound, a real object was simulated, as from a real object also from different parts of it a slightly different variation of the sound it emits, is emitted. Additionally, I played with "perspective", that is, the object starts out being far away, behind the speakers and the width of the object is rather small. During the composition the object comes closer and becomes wider. The object comes even in front of the loudspeakers and then implodes, and quickly explodes again, to make a rotating movement behind the loudspeakers. Eventually, the object falls apart and the four sound sources move away, each in another direction. In figure 11 an overview of the composition is given.

Julius Stahl made a video to accompany the piece. The video makes use of pictures of the same object that the sounds were from; as the sounds are alienating themselves from the original sound during the composition, in the images the object becomes clearer. The video is rendered live and has slight variations, each time it is played. In figure 10 three screenshots from the video is given.

Concert
The concert took place at the Club Transmediale Festival on the 4th of February. This is a festival that includes electronic music both from (underground) club culture and from more academic approaches.

The hall in which the concert took place measured about 105 square meters and was relatively reverberant. The array was positioned on the stage a little bit above ear height.

The concert was preceded by a short presentation explaining the wave field synthesis technique and the software that the composers used to create the movements of their sounds.

During the concert, the biggest problem was that the system with its 24 loudspeakers could not create enough loudness for the amount of people who filled the hall (ca. 100 listeners). This had as a disadvantage that the people in the back could not perceive the music very well and were a bit loud as they started to talk. During the sound check (without the sound absorbing people in the hall) the system was loud enough for the whole hall and the effect was even clear in the back of the hall.

For the presentation of a prototype system the concert can be regarded rather as a success. The listeners who were in the front could perceive very well the movements of the sounds in the compositions. Especially when closing the eyes, some people commented that the music created a vivid visual image with its movements through space. Others were quite amazed that they could really move around the source, that is, position themselves on a different relative location to the virtual source. A sound artist, who works a lot with ambisonics, commented that especially the distance of various sources can be much better modelled with WFS than with ambisonics.

The pieces of Lingk, Lippok and Baalman were received best, as the movements of the sounds in these pieces were the clearest. This is probably due to the type of sounds that they used, which all had a broad frequency spectrum, thus enabling listeners to locate the sound more clearly.

Some listeners were disappointed, as the system was not yet a full surround system.

After the concert several other composers showed an interest in applying the system for their own work, varying from electronic music concerts, to sound installations, to a combination of electronic music with dance.

At the Electrofringe festival (at the 4th of October 2003 in Newcastle, Australia) a setup of 16 loudspeakers was used, that had more power than the speakers in Berlin. Here, there were no loudness problems. The room was about 80 square meters large. The concert was preceded by a two hour workshop about the system and the work of the composers, with sound examples. The amount of listeners at the concert was around 100. Here also there were a lot of positive reactions from the public.

Figure 12. The audience at Electrofringe during the Concert

Figure 13. The workshop at Electrofringe

Future work
Currently, we are working at the TU Berlin on the implementation of the OSC protocol into the program so that the sound movements can be controlled from other software. We hope to have this done by the start of March 2004. The software program should then also become available for composers and other artists.

After that the main focus will be to let composers and sound artists work with the system, so that we know where the interface needs to be improved and in which direction the technique can be further developed. I plan to make two sound installations with the system, that will show the possibilities of the system.

Acknowledgements
To Club Transmediale for giving the opportunity to present the work on their festival and inviting the composers to work with the system.

To Electrofringe for giving the opportunity to present the work on their festival and to the Goethe Institut Sydney for financial support to enable me to do so.

To the Sound Control group of the University of Technology of Delft for using their figures.

References

• Berkhout, A.J. 1988, A Holographic Approach to Acoustic Control, Journal of the Audio Engineering Society, 36(12):977-995
• Berkhout, A.J., Vries, D. de & Vogel, P. 1993, Acoustic Control by Wave Field Synthesis, Journal of the Acoustical Society of America, 93(5):2764-2778
• Bourdillat, E. 2001, Auralization of sound fields in auditoria using Wave Field Synthesis, M.Sc. Thesis, TU Delft, The Netherlands
• Chowning, J.M. 1971, The simulation of moving sound sources, Journal of the Audio Engineering Society, 19(1): 2-6. reprinted in: 1977 Computer Music Journal, June, pp 48-52
• Jansen, G. 1997, Focused wavefields and moving virtual sources by wavefield synthesis, M.Sc. Thesis, TU Delft, The Netherlands
• Huygens, C. 1690, Traite de la lumiere; ou sont expliquees les causes de ce qui luy arrive dans la reflexion et dans la refraction et particulierement dans l'etrange refraction du cristal d'Islande; avec un discours de la cause de la pesanteur, Van der Aa, P., Leiden, The Netherlands
• Malham, D.G. & Matt, A. 1995, 3-D Sound Spatialization using Ambisonic Techniques, Computer Music Journal, 19(4): 58-70.
• Malham, D.G. 2001, Toward reality equivalence in Spatial Sound Diffusion, Computer Music Journal, 25(4): pp. 31-38.
  Torger, A., 2001, BruteFIR, http://www.ludd.luth.se/~torger/brutefir.html
• Verheijen, E.N.G. 1998, Sound Reproduction by Wave Field Synthesis, Ph.D. Thesis, TU Delft, The Netherlands
• Vogel, P. 1993, Application of Wave Field Synthesis in Room Acoustics, Ph.D. Thesis, TU Delft, The Netherlands
• Weske, J. 2001, Aufbau eines 24-Kanal Basissystems zur Wellenfeldsynthese mit der Zielsetzung der Positionierung virtueller Schallquellen im Abhörraum, M.Sc. Thesis, TU Chemnitz/TU Berlin, Germany
• Wright, M., Freed, A. & Momeni, A. 2003, "OpenSoundControl: State of the Art 2003", 2003 International Conference on New Interfaces for Musical Expression, McGill University, Montreal, Canada 22-24 May 2003, Proceedings, pp. 153-160