From: Living Stereo Histories and Cultures of Multichannel Sound

Edited by Paul Thberge, Kyle Devine, Tom Everrett

Bloomsbury Academic. Buy book here: http://www.bloomsbury.com/uk/living-stereo-

9781623565510/

A Symphony of Sound: Surround Sound in Formula One Racing Games

Ruth Dockwray and Karen Collins

Abstract: "Sound has always been an integral part of Formula One racing. The translation

of live and televised experiences to video games has taken this important feature of

racing into account, giving sound a prominent role in games. This prominence is

highlighted by experimentation with surround and spatial sound in racing games, which

alternates between high realism and cinematic exhilaration. This chapter explores the role

of sound in racing games, with a particular focus on the strong emphasis on surround

sound. Comparisons are drawn between the sonic aesthetics of television, live, simulation

and ludic experience with a particular focus on Formula One racing games."

We listen to the sound of your car at a minimum, while accelerating and while
running at full speed. We listen carefully to the sound of the engine. And we
wonder, isnt it turning too fast? Isnt this an engine that is crying out in protest?
No, this is the sound of an engine howling with joy. A sound that no composer
could reproduce, that no orchestra could play. A symphony of sound that brings
joy to the mind and a smile to the face. Letter to Ferrari, cited in Borgomeo 1997:
11718).

1
In 2014, Formula One racing underwent a major change. New regulations dictated the

end of unboosted 2.4-liter V8 engines that peaked at 18,000 rpm, and the introduction of

much quieter 1.6-liter V6 turbocharged engines, limited to 15,000 rpm. The move is

designed to reduce noise disturbance and environmental damage associated with the

motorsport, and to bring the technology closer in line to that of road vehicles (Caswell

2011). The exhaust-driven turbocharger will suppress some of the noise and lower the

overall frequency of the sound of Formula One cars. This change in engines most

notably the difference it will make to the sound of the sport has been much lamented

by racing fans and sports critics alike, as described by Sydney Morning Herald sports

writer Mark Fogarty:

The new configuration will radically alter the most elemental appeal of F1 the

acoustic assault that accentuates the cars speed as part of the sports biggest

technical shake-up in years... It will mean the end of the piercing wail emitted by

the engines since 1989, when the previous generation of turbo motors was

outlawed in favour of a return to normally aspirated powerplants... But with its

impellers spinning at 100,000 rpm, the turbocharger will add a distinctive

whooshing sound to the muffled mechanical snarl as the engine revs rise and fall.

While residents of Albert Park and surrounding areas will rejoice at the prospect

of the sound of the F1s being dialed down from 11 to perhaps eight or nine, racing

enthusiasts are apprehensive. The banshee-like shriek is a big part of the

attraction, emphasising the sheer speed of the cars as they rocket from one corner

to the next and bullet down the straights (Fogarty 2013).

2
The sound is so important to the audience of motorsport many fans obsess over minute

details in engine audio. Various iPad applications like Crankcast Audios Rev and 2XL

Games XLR8 have been released that allow a user to interact with audio-based Formula

One car engines the apps consist of essentially just a few buttons and the sound,

putting all of the focus on the audio and countless video and audio recordings exist

online of the sound of the individual cars and their components.

Formula One engines have always been, and will remain, loud machines.

However, the sound has changed from what had been described as a symphony of

sound, to the new era of turbo-charged engines (or rather power units). Rob White, the

deputy managing director of Renaultsport F1, has explained that engine noise remains a

crucial component to the sound of the new power units (although the new sound is a

turbocharged noise, rather than a normally aspirated noise). White promised fans that

although the overall sound pressure level (so the perceived volume) is lower and the

nature of the sound reflects the new architecture . . . The engines remain high revving,

ultra high output competition engines (Renault 2013).

It is clear that the sound of Formula One is an integral part of the experience. For

racing spectators, there is nothing more thrilling than experiencing the sound of twenty-

two engines revving simultaneously on the start line in anticipation of the impending

race. The multisensory aspects of live racetracks attract many fanatics to fill large-

capacity racing circuits. The growing increase in number of circuits worldwide allows an

increasing number of fans to experience firsthand the live aspect of racing.

But being a live crowd participant is just one of several ways to experience the

thrills of fast cars racing for the coveted top spot on the podium. Many fans also

3
experience Formula One through television and video games, where surround sound is

often used to compensate for some of the loss of the multi-sensory aspects of live circuits.

While there has recently been growing interest in cinema surround sound (e.g. Sergi

2004; Manolas and Pauletto 2009; Kerins 2010; Atkinson 2011; Collins 2013), the

impact of surround sound on media experiences remains surprisingly underexplored, and

the study of surround sound in games in particular has been neglected.1

In many ways, video games have been a driving force in new approaches to

surround sound, as experiments with binaural, ambisonics and real 3D surround have

been fairly common interests of sound designers at game companies (see Tsingos et al.

2010; Heinen 2013). According to a survey of game players by Australian game company

Codemasters, about 90 percent felt that surround sound was important or very

important to game playing (Goodwin 2009). Bearing in mind that this data is now aging,

we can expect these numbers to be even higher today, since prices of surround sound

systems have dropped and quality of sound in games has improved. This does not mean,

however, that all players set up the surround system in the ways recommended by the

manufacturers or surround standard developers. Nevertheless, Formula One game fans, as

Formula One fans more generally, tend to be quite obsessive about the sound, as

described above, and we can expect many of them to have gone to the trouble of

calibrating surround sound setups. It is common for racing game fans to purchase

peripheral devices like steering wheels, pedals or racing seats to improve the playing

1
There are a few exceptions, including Stockburger (2003), Grimshaw (2007) and Kerins (2013).

4
experience. Likewise, we assume that the use of surround-sound loudspeakers is common

among hardcore fans (particularly since many racing seats include built-in speakers).2

In this chapter, we explore the role of sound particularly surround sound on

the Formula One gaming experience. Formula One gaming represents a unique tension

between realism (simulation) and entertainment (games).3 On the one hand, the game

developers go to great lengths to provide fans with a realistic experience, while on the

other hand, this realism is often not a live simulation, but what Collins (2008: 134)

calls a cine-real: The cine-real is a sense of immersion and believability, or

verisimilitude, within a fantasy world. The cine-real space of Formula One game sound

is one that combines an authentic simulation of the cars, with televisual presentation of

the races, and a cinematically designed emotional quality not experienced in either live or

television experiences. Here, we first explore the televisual presentation of Formula One,

followed by an overview of current audio technology in Formula One games. Finally, we

explore the implementation of sound in the games, and the artistic license taken by sound

designers in presenting the auditory experience to listeners. We will show how surround

sound in racing games is a key element in the experience between the realism and fantasy

2
We could find no statistics on the number of players using peripherals, but judging by the
number available on the market and our own experience talking to fans of the games, we can
assume that many players use these devices. For an example of just how far fans will go to
enhance their experience, see the RSeat Gaming Seat (http://www.rseat.net/), which boasts fully
fibreglass and carbon steel adjustable seats with leather upholstery, racing wheel and shifter set,
Clubsport pedals, and speakers, for a price of $1500 US, not including the accessories. The price
and detail in the peripheral suggests a fanaticism not found in most types of games.
3
Formula One simulators are designed with much more strict realism in mind: see, for instance,
the F1 Driving Experience, which has a full-size Formula One replica car made with real parts.
Here, realism is emphasized in marketing material that promises the car is barely
indistinguishable from the real thing and most realistic experience you will ever get, apart from
if you are a Formula One driver (http://www.thef1drivingexperience.com/).

5
of racing games and an important component of the experience that the players have of

these types of games.

Television Presentation of Formula One Racing

One of the most obvious sounds that you experience as a Formula One spectator is the

Doppler effect. The Doppler effect (the change in frequency of a sound as a moving

object, such as a vehicle approaches and passes) emphasizes the extreme speed and

power of the Formula One car. Watching the race from the stands or the trackside offers a

limited view of the road and cars, but an enhanced auditory field and sense of the smell

and sounds emanating from the cars as they approach and pass by. Spectators are treated

not just to the sound of the cars, but also to the pit-lane warning signals, crowd noises,

overhead helicopters and multiple announcers.

In contrast to the live experience, watching Formula One racing on television can

be largely a visual experience, accompanied by the presentation of various camera

viewpoints and race analysis. Throughout the race weekend, the qualifying and race

sessions are transmitted, offering in-car perspectives, long shots of sections of the track

and close-ups of pit stops. This pre-race material, coupled with driver interviews,

commentary, and driver strategies, combines with data analysis (G-force, maximum

speed, tachometers, etc.), delivered as on-screen graphics, constantly informing the

viewer of ongoing changes as the session develops, and is often presented as if the viewer

can see some of the drivers controls. Digital interactive television today offers the

viewer a choice between camera angles, such as on-board feeds for multiple drivers,

leaderboards, rolling highlights, pit channel, and driver trackers. Indeed, the amount of

6
controls on-screen (currently nine in Sky TVs mosaic viewing) has increased in the

past decade with the rise of digital television. Combined with the offering of the choice of

cameras (drivers now have a camera housing on top of an air box behind their heads,

facing them behind the wheel, on the wing mirrors, and nose cameras), the additional

visual material has undoubtedly enhanced the audience experience.

Likewise, television audio now consists of multiple streams of two or more

announcers, pit reports and on-line reports from drivers, along with car noise (both

internal and external; Petkovi et al. 2002). Improvements in noise-cancelling technology

and the incorporation of microphone arrays inside crash helmets have greatly improved

on the performance of the older radio microphones used to transmit sound from inside the

vehicle, where the background noise can reach 140 dB.4 This higher-quality transmission

means that not only can the audience hear more of what is occurring inside the car, but

that the drivers engage in much more conversation with their crew while driving, as

described by Bell Laboratories researcher James West:

In the past, drivers in a race spoke to their pit crews and team managers only

when they slowed down for difficult corners a quieter moment, perhaps, but

given the amount of driving skill needed to handle corners, a less than ideal time

for shop talk. With the [new] microphone array, drivers can now communicate

with members of the team at any point in the race (in Austen 2001).

4
The noise inside the race car is about the same as a 747 jet if youre 300 feet away (Austen
2001).

7
 Cameras on the car, as well as on the track, also tend to have microphones

associated with them: for example, there are currently 160 audio signals that are mixed in

real time in the Italian broadcast of F1 (PSNEurope 2013). Thus, television viewers can

potentially hear inside the car, outside the car alongside the track, in the pit, in the

crowds, from announcers and from helicopters taking overhead shots. All of these

sounds are mixed live, with real-time envelope generators and digital signal processing

(DSP) effects to enhance the experience for the audience (Stevens 2011).

Television clearly offers an audiovisual experience that is distinct from those of

the live spectator and the Formula One driver. The multiple visual and auditory

perspectives available ensure that the audience receives a broad spectrum of the many

experiences associated with Formula One. And it is this broad perspective that is carried

over into racing games indeed, one significant distinction between a racing game and

a racing simulator is the variety of auditory and visual perspectives involved.

An Overview of Sound in Racing Games

Racing games have been common since the earliest days of video games The first racing

game, Gran Track 10, was released by Atari in 1974, along with a space-themed racer,

Space Race. Racing games remain extremely popular, and are often one of the few types

of games still released for arcades.5

5
For a historical overview of racing game sound, and some excellent related resources, see
Kastbauer and Nichols (2012). We are indebted to Kastbauer and Nichols and the related Game
Audio Podcast (Kastbauer and Woldhek 2012) for much of this section.

8
 Before we delve into surround sound in racing games, it is worth exploring in

general terms the use of sound in general in racing games. In the early days of games, the

sound of the car usually consisted of just a few sounds, synthesized with a simple

frequency-to-revs or frequency-to-speed ratio. Today, sound in racing games tends to be

physics-based, in the sense that the real-world laws of gravity, scale, movement and

friction drive the sounds. Creative audio director Nick Wiswell explains the

complications with and level of detail required for implementing sound in racing games:

Most games require lots of one shot sounds for things like character movement,

footsteps, weapons, [voice over], world objects and object interactions etc. with a

small number of constant sounds for moving objects, vehicles or background

ambience. With a racing game most of the objects in the game and making sounds

created from many layers of continuously changing sounds (in Nichols 2012a).

The sound of the car itself is the most important element to a racing game. Drivers listen

to the sound of the engine and tires for key information about their speed, driving tactics

and potential problems with the vehicle:

Obviously, the fundamental frequency is based on the engines rotational speed,

or RPM. A driver will often memorize the pitch of the engine when its time to

shift, and will wait for the sound he or she hears to match with that in his or her

memory, and use that as a shift indicator (since rarely will the driver be looking at

the instrument panel). After a number of hours in the same car, a driver will also

know much of the harmonic content of the engines sound when its running at its

9
 peak. The driver can use this to tell if the sound changes as a possible indicator of

a malfunction. For example, an oil leak will make things like valve tapping much

more clicky. A cylinder that is no longer firing will interrupt the fundamental

frequency. A good driver can use these sonic clues to help diagnose a failure even

before the car returns to the pits (Kastbauer and Nichols 2012).

The sound in a racing game today is thus never a single sound, or even a series of sounds

baked into one (cf. Wright, this volume). They are nearly always constructed of

multiple layers of effects that will alter depending on the players selections, actions and

the state of the car, driven by the physics, meaning at any one time there may be twenty

to forty different layers of audio making up the sound of the car (see Nichols 2012b).

Sound in racing games must also try to compensate for the loss of physical

(haptic) interaction with and feedback from the car in the virtual realm. Tire sound, in

particular, explains Forza sound designer Mike Cavieziel, is a very important part of the

game for us, as its a main source of feedback for the player while racing.

Real-life drivers can feel G forces to know when things are about to get loose, and

we use audio and rumble to help communicate that information in Forza. Weve

done numerous tire recording sessions, on numerous different surface types, to try

and cover the full range of grip and traction sounds that you need to really get

around a circuit quickly (in Nichols 2011).

10
In fact, tire sounds have become a particularly important element of providing the player

with sonic feedback. The most recent development in racing games has been four-wheel

independent sound models, which allow players to hear that the front left tire is squealing

more than the rear right (e.g. Forza 3; GranTurismo 5; Shift 2: Unleashed). When

listened to on a surround playback system, the appropriate wheels sound is played

primarily through the corresponding surround location. Tire sounds can be a useful

substitute, in other words, for G force, with tire audio letting the player know when the

car is about to spin out. As Wiswell explains (in Kastbauer and Nichols 2012), in the

new tyre model, we have hundreds of different sounds and skids for each surface type.

He continues:

In fact, its more complex than the engine model overall because were

tracking lateral and longitudinal forces as well as the amount of load on each tyre

there are different sample sets for loaded/unloaded. As you go into a corner

and turn, one tyres being forced into the ground, while anothers being slightly

lifted off the surface and the sonic characteristics in each case are completely

different. (in Kastbauer and Nichols 2012)

A low-frequency howling also emanates from tires as they reach their grip limit:

When a driver, say, turns too fast, the amount of lateral static friction is exceeded

and the tire begins to slide It is therefore absolutely crucial that the driver can

distinguish exactly how close to that limit he or she is. In the real world, the driver

can feel his or her body (and by extension the car itself) shifting in all directions

11
 and know from that whats going on with the tires. In the absence of physical

communication, sound fills the void in an auditory language the player can learn

through practice. That language mimics sounds that real tires make, but in general

are exaggerated in games. As the tire approaches the grip limit, the sound starts as

a lower-frequency hum or growl. This will get louder and hissier the closer to the

limit the tire is. When the tire exceeds the grip level, you hear the squeal

(Kastbauer and Nichols, 2012).

Despite the attempts to create such highly realistic sounds as independent tire

detail, there is, as mentioned above, also considerable artistic license in racing games.

Racing games are not realistic simulators of the driving experience. Although such

simulators exist and are in use by race teams to help drivers learn tracks, track metrics

and improve race times, games serve a different purpose: They are primarily

entertainment, and despite some of the attempts described above to create an accurate as

possible reproduction of the car, many sonic elements in the game are not realistic.

Physics-based, realistic sound is not always desired when it comes to representing real-

world acoustics: Although a few players may want sound to be as close to the real

experience as possible,6 for the most part sound is adjusted to ensure an entertaining

experience and to compensate for the loss of other sensory modalities. What matters in

the acoustic space, then, is not so much fidelity to a real-world event but, rather, an

appearance of truthfulness and believability the cine-real. Moreover, some of the

6
As mentioned in the Game Audio Podcast special on racing games (Kastbauer and Woldhek
2012), one player wanted the sound to be as if heard by the driver, through earplugs, balaclava
and helmet!

12
sound can be faked, since nobody knows what it sounds like to stand behind a [car]

going 100km/hour (Kastbauer and Woldhek 2012).

One aspect that is drawn from the real world is the, audio recordings are of the

vehicle chosen by the player, and close attention is paid to simulating actual sounds.

Often the sounds of the engine are recorded on a dynamometer, where sound recordists

can get the sound of the car from multiple microphones without any road sounds

interfering. With Formula One cars, though, the cost of the cars is prohibitively

expensive, and thus they are usually recorded while driving on the track, instead of being

put through the stresses required to record cars on a dyno, such as high rpm revving

(Kastbauer and Woldhek 2012). There is, however, some artistic license, particularly in

the use of distortion to give the car a larger, more powerful feel than the real thing: for

example, Forza 5 incorporated non-automotive sounds into the cars, such as human

screams in the tire screeches, and lion roars in the sound of the engine. Wiswell explains,

how can we use non-real-world sounds to deliver that experience at a primal level. Its

all sort of a psychoacoustics where certain sounds will evoke certain emotions in people

and mixing them at a level where its not necessarily audible but it still evokes that

emotion in people because its there subconsciously (Evans-Thirlwell 2013).

There are some other distinct differences between racing games and their

simulator or real-life counterparts. Crowd noise, for instance, can often be heard in

racing games. The roar of the crowd as one passes the stands towards the finish line can

be an invigorating aspect of play. However, race-car drivers (at least, Formula One

drivers) cannot hear the sounds of the crowd over the noise of their own cars (Nichols

13
2012c) and thus strict simulation would make for a quite boring auditory experience for

players.

Most racing games do not have any kind of commentary track, unlike other sports

games. Many players do not want the commentary as they find it too distracting

(Kastbauer and Woldhek 2012). While football or other sports games may be equally

complex for the player in terms of the amount of on-screen content, in racing games, as

described above, it is particularly important to be listening to the sound of the car for key

cues as to what is happening: a simultaneous commentary would disrupt this listening

process, as it is difficult for our brains to tune out verbal cues.

Another significant distinction between driving, television viewing, simulations

and games is the presence of music. Drivers never use music when driving real Formula

One cars (or in simulators) and music is rarely used in television broadcasts during the

race. Studies have shown that driving ability in simulations and games decreases with the

presence of music. In particular, tempo influences driving speed and speed estimates

(Brodsky 2001; cf. North and Hargreaves 1999; Cassidy and Macdonald 2012). Many

games, however, use (or at least allow for) continuous background music. We suspect

that there are several reasons for this use of music. Firstly, the entertainment factor,

whereby playing for long stretches may not be as enjoyable without music. Secondly,

most of our real-life driving experience as players would be in our own cars with music

playing, and therefore music may be expected by the player. Thirdly, music may enhance

the sense of speed of driving.

14
 In Forza 5, released for the Xbox One in 2013, rather than use licensed music (as

is typical for racing games) the music is entirely procedurally generated based on factors

in the game, such as position of the car. Wiswell explains:

If youre towards the back we can dim it out, if youre towards the front we can

push it up. When youre out on your own we can alter the mix compared to when

youre in a dense pack. If you start to mess about and go the wrong way, we can

bring in some negative element. We even have an element where, as you hunt

down the guy in front, there's a sort of tension layer we build up, which releases

when you pass the other guy and starts up again as you get to the next car. (Evans-

Thirlwell, 2013)

This careful balance between matching an expected cine-real experience with that of the

actual experience is carried into the use of surround sound in racing games.

An Overview of Surround Sound in Games

Surround sound offers key insights into how video games are designed, developed,

presented and played; yet it remains an overlooked element of video game studies. While

surround sound was designed originally for film, games today are driving the quest for

greater numbers of speakers and new standards and technologies. Three-dimensional

audio, with speakers above and below the commonly used plane of loudspeakers, has

been a subject of ongoing discussion for many years in the game audio subgroups of the

15
Audio Engineering Society and Interactive Audio Special Interest Group. Game console

maker Microsoft even filed a patent in 2012 designed to detect player position and adjust

that position so that the player is always in the sweet spot, indicating a high degree of

interest in creating an optimal listening environment for players (Flaks et al. 2012).

Binaural recording in games has also been a growing interest, with some game

development companies coming out with specially-designed binaural game engines (such

as the Papa Engine).7 Moreover, regular updates are made to the hardware and software

capabilities of games and their platforms to enhance the surround experience for players.

There are three primary considerations when it comes to surround sound in

games: the technology used, the implementation of the sound, and the impact of sound on

the player. Each of these are dealt with separately below.

Although racing games are certainly available on other devices, such as the iPad,

most racing game players invest considerable time and money into their systems, to play

on the most advanced console systems available (currently, and for the purposes of this

analysis, the Xbox360, XboxOne and Playstation 3 and PlayStation 4). Each of these

consoles fully supports at least 7.1 surround sound.8

Surround sound playback varies greatly, and can depend on the players setup at

home, the environment that setup is in, and the decoding method (see below). Assuming

that the player is using loudspeakers (rather than virtual surround headphones) the

decoding method is perhaps the most important element in playback. The ways in which

7
See the Papa Engine homepage, http://papaengine.com/
8
The 5.1 standard surround sound system places three loudspeakers at the front (left, center and
right channels), and two in the rear (rear right, rear left), as well as one subwoofer (the .1). The
6.1 format adds a rear center loudspeaker, which improves localization and panning from front to
back. The 7.1 format varies widely, with more frontal speakers, two extra channels on the side or
rear, or even two at the front at a height above the other speakers, which commonly exist on a
horizontal plane.

16
the sound is distributed depends on the decoding system used. This means that what the

listener hears when they play a song mixed in stereo over their surround sound setup may

be very different from what the recording engineer heard in testing the stereo mix in the

studio. Game surround is mixed as discrete or matrixed. In discrete mixes, individual

channels are all unique in terms of the information they provide. Since 5.1 has six

channels of sound, there are six different signals being sent to the channels. A matrixed

system (such as Dolby Pro Logic) encodes extra information into a stereo signal and

recovers those channels through a mirror-image decoding. Many video games now use

matrixed surround sound. If the user only has a two-channel stereo system, the

information can still be played back, and amplitude adjustments and phasing is used to

re-create the original mix effect, and if the player has a decoder (Pro Logic or Pro Logic

II) it will play back as a surround mix. Moreover, if the player is playing a game that was

only mixed in stereo, the Pro Logic II system will guess where the sounds should be

routed and transform the sound into a surround mix. The problem with this approach is

that there is crosstalk between channels (meaning it is less discrete, with less accurate

localization). Game consoles therefore typically allow for both matrixed and discrete

mixing.

There are a variety of surround format technologies currently in use: on consoles,

the most common are the Dolby and DTS standard 5.1 and 7.1 systems, but it is worth

also discussing ambisonics, since to date the most common use of ambisonic surround

has been used in racing games (notably those developed by Codemasters, who develop

the majority of major console racing titles such as the DiRT [2007] F1 [2009] and the

GRID [2008] series). Ambisonics consists of four channels of audio, but the outputs are

17
calibrated into a matrix designed to accommodate an unknown speaker setup (potentially

allowing for a nearly infinite number of speakers). In order to create this system,

Codemasters had to develop their own audio engine and use third-party software like

Blue Ripple Audios Rapture3D, also used in F1 2011 (Codemasters 2011).

Sounds in games need to be mixed in terms of priority to the listener, often with

the prioritization of sound tied to the space (i.e. proximity: closer sounds being given

greater priority). The location of sounds, in other words, needs to be constantly updated

as the game is played. Commonly, surround sound is programmed into the game engine

using DirectSound (on Xbox) or OpenAL (cross-platform), or using one of the

middleware engines, such as Fmod or Wwise, which allow for considerable design

control without the need for the implementation specialist to also write much code. In the

middleware software, surround sound is commonly created using a three-dimensional

graphical representation of the audio listener (player position). Both the actual position in

the virtual space and the distance from the listener must be taken into consideration in

order to create the auditory perspective, through attenuation (natural weakening of signal

tied to distance) and spatialization (the position within 3D world). Distance attenuation is

common, where the signal is simply attenuated based on the distance from the emitter.

Most 3D game engines have some form of attenuation cone (an algorithm reducing

attenuation of the audio file based on proximity and location) that can be used to simulate

the propagation of sound in a particular direction. Since many sound sources have a

defined facing angle, the sound will need to be attenuated or filtered at the back, so a

front and back source is defined in the engine. Attenuation cones can be specified,

allowing the designer to determine angles of attenuation (inner and outer angle

18
parameters) and the type of roll-off effect (as the volume attenuates), along with various

filters to alter the sound depending on distance, such as low-pass filters, amount of low

frequency (LFE) and spread amongst the speakers.

Most surround sound layouts have been designed with movies in mind, and for

various reasons these are not always the best setups for games. But since game

developers cannot expect game players to reconfigure their home theatre systems every

time they switch between playing a game and watching television, game sound has been

designed to work with existing systems. The front left and right channels are commonly

used for music and Foley sounds that might extend to the edges or off the screen (such as

footsteps, cars, etc.). The most important channel, however, is the center channel, which

is used for dialog and Foley, menu interface and player-generated sounds. The rear

channels (also known as the surrounds) add a sense of envelopment but may draw the

listeners attention away from the screen if there are discrete sounds placed in those

channels (sometimes called the exit sign effect see below), which in film is to be

avoided (cf. Wright, this volume) but can be a useful technique for games. In games,

discrete sounds can be placed in the surrounds to give the player a full 360-degree sonic

space.

SurroundSoundsImpactonthePlayer

Surround sound is, arguably, more important in video games than in film (depending of

course on the genre of the game). Studies have shown that surround sound is noticed by

19
players, improves their enjoyment of games, and has an impact on their ability to play

(Goodwin 2009; Sklalski and Whitbred 2010; Kerins 2013).

One of the key functions and effects of surround sound in games is to immerse

players in a virtual world through a sense of sonic envelopment. The term envelopment in

relation to spatial sound has been used to describe sometimes overlapping and even

contradictory concepts. Berg (2009) summarizes some of the ways in which it has been

used in the scientific literature to refer to: the sense of spaciousness; the subjective

immersion of the listener; the fullness of sound images around a listener; the sense of

being enveloped by reverberant sound; and the sense of being surrounded by sound. Here

we define envelopment as the sensation of being surrounded by sound and/or the feeling

of being inside a physical space (enveloped by) that sound (Berg suggests the term

surroundedness). Envelopment has been shown to be important to listeners in creating

a sense of presence (Rumsey 2002), which is to say, to reduce the sense of mediation

between audience and virtual space. Sound is thus responsible to a significant extent for

the sensation of being present in a virtual environment through the sensation of

envelopment (Berg 2009).

A second key function of surround sound in games is to improve localization: the

ability of the listener to locate the sound in the three-dimensional space around them.

Localization is of particular importance in games where the player is positioned in first

person visual perspective. In the words of Kerins (2013), In 5.1-equipped FPS titles, this

[first person] visual aesthetic is usually mirrored with an aggressively first-person

approach to the sound design that pans all sounds around the 5.1 soundscape to

correspond with their position around the players avatar; for example, sounds that

20
originate in the diegetic world from somewhere behind and to the left of the avatar are

played through the left surround speaker.

It is this aggressive approach to using the surround sound speakers for discrete

effects (spot sounds) that significantly distinguishes the use of surround in games from

that of film. The Exit Sign Effect refers to a phenomenon whereby with discrete

sounds placed in the rear surrounds, or any pans of fast-moving sounds that exited the left

or right screen boundaries would lead the viewers eye through the space to follow the

sound, away from the screen. For this reason, surround loudspeakers are not usually

given discrete sound information in films, because the surround sounds draw the viewers

attention away from the screen. But with games, the players attention need not

physically draw their vision towards the loudspeaker, because they can move their

character to look in the direction of the sound. In other words, the surround speakers may

in fact alter the ways in which they move the character or camera to attend to that sound

event. Typically the field of view in a first-person game is between 65 and 85 degrees

(Stevens and Raybould 2011, 90), meaning that the majority of the virtual space (that is,

the other 275-295) takes place through the sound, off-screen. Moreover, players are able

to de-acousmatize that space in other words, discover the source of the sounds that

they are hearing off-screen (Stockburger 2003).

ImplementationofSurroundSoundinRacingGames

There are many reasons for the use of surround sound in Formula One games. Perhaps

the most important is the ability of surround sound to employ low frequency effects

(LFE) through the subwoofer channel. LFE is particularly useful in creating a sense of

21
power and an added weight, which is especially important when attempting to recreate

the sound and feel of a powerful engine (cf. Hope 2007). The inability of human beings

to localize low frequency effects means that LFE tends to permeate around us, creating a

feeling of envelopment and immersion in the space.9 Moreover, LFE tends to physically

vibrate the air in a way that we can feel, creating a tactile feedback that, when the speaker

is positioned near the player, can be felt as much as heard.

Surround sound systems are perhaps more realistic than two-channel stereo when

it comes to Formula One in particular, since actual F1 car engines are located behind the

driver, rather in front, and thus much of the noise of the car occurs behind the driver,

rather than the front, making the rear audio channels more important in creating a sense

of believable vehicular space. Surround sound can also be helpful in games where the

viewpoint is through the front windscreen without the benefit of wing mirrors, which

allows players to hear rather than see cars coming up from behind. Again, the rear

channels are particularly helpful in locating where other cars are in relation to the player

character.

Also important in developing a sense of believable physical space is the use of

sound propagation techniques to create the environment in which the car is driving.

Although open-ground circuits such as the Mosport International Raceway in Canada

may not have much in the way of reverberation from walls, on closed-in tracks such as

Monaco, the cars sounds bounce of tunnels, walls and buildings as the car proceeds.

Although it is possible to mimic some three-dimensionality of space using reverberation

in stereo, surround sound is particularly important in creating that sense of three-

9
Sounds with frequencies below about 2kHz have wavelengths longer than the distance between
our ears, and thus our brains cannot determine the location of the source, since our brains respond to
differences in sound between our left and right ears as a means to locate sounds.

22
dimensionality through reverberation off the walls not just beside but also in front,

behind and around the driver. Of course, merely being able to hear the sound outside the

car is another matter of artistic license, as described in a podcast on racing sound audio

by two game sound designers: If you were to strap yourself into a Formula 1 car with a

136dB engine right behind your head, and drive from an echoey stadium and then out to

open air, chances are you would never be able to hear the change in ambiance. However,

on quieter cars or in a 3rd person perspective, it becomes more appropriate that a change

in environment should have a similar change in sound (Kastbauer and Nichols 2012).

In most racing games, camera position can be selected by the player, who is

typically given a choice of several views. For example, the player can select the helmet

cam, a view from the back of the car, as well as a more zoomed-out version of that angle

and a camera angle from the front bumper of the car (on the hood). Often, we hear the

important sounds such as gear shifting as if we are in the car even when we are visually

above the car, behind the car, and so on. Logically speaking, behind the car we should

hear a lot of exhaust sound and none of the internal cues. But we dont. The auditory

position often remains regardless of visual position, or there are only minimal shifts in

the auditory perspective. In the words of Technical Sound Designer Damian Kastbauer,

It sounds to me like for most games, the nose cam and hood cam sound

identical. In games with multiple interiors like NFS Shift 2U, both interiors sound

the same. For games with multiple 3rd person views, both views sound the same.

This basically leaves all the games with 3 mixes to spread across all the views.

The games that transitioned from one view to the next like the F1 games were

23
 much more pleasant than the hard jumps when scrolling through, though this is a

feature I assume is rarely used as most gamers will find the view they like and

stick with it. (Kastbauer and Nichols 2012)

As above, we see here a balance between what is believable (the cine-real) versus what is

actually realistic. Player expectations and experience of Formula One through other

games and television experience clearly influence the design of the sound in the games.

Conclusions

Sound is an integral part of Formula One racing. The current debate about the 2014

change in engines is a clear indicator of how strongly fans of the motorsport feel about

the importance of the sound of Formula One. But beyond the visceral experience of being

a sonic spectator to racing, sound is a critical component of the drivers awareness of her

or his car and performance, indicating speed, faults, tire grip and gear shifting. Although

surround sound is critical to neither the live nor televised experience of Formula One, it is

a critical component when it comes to games.

As we have shown, Formula One racing games balance both the driver and

spectator experiences of racing, the televisual and the live experience, walking a tight

balance between realism and fantasy. Surround sound plays a key role in the feelings of

immersion and envelopment in the game, in the excitement and feeling of power

associated with driving a powerful vehicle, and perhaps most importantly, in providing

critical information about the environment and other drivers in racing games to the

24
player. Without surround sound, the player lacks key information about where other cars

are located, which tires on their car are squealing, and what other changes are occurring

in the driving environment.

The surround mixing of racing games is an indicator of how important surround

sound is to the video game experience. In racing games, the auditory perspective rarely

makes significant changes between viewpoints (whether the point of view is inside, on

top of, or behind the vehicle), meaning that the auditory information given to the player is

deemed critical even if it conflicts with our viewpoint. In other words, we can often hear

the gear shifter outside the vehicle, for example, or hear crowd noises inside our very

loud vehicle. The sound of the exhaust may also not change depending on whether we are

positioning our player-camera directly behind the vehicle or atop of it. This more

uninterrupted auditory vantage point means that the player is usually, in a sense, in a

simultaneous auditory first-person and omniscient perspective, even when the visual is

not. The reason for this auditory vantage point is clear: without this sonic information, the

designers felt that we would be missing key components of the driving experience.

Indeed, surround sound, as we have shown, contributes greatly to the sense of realism

experienced by the player, even though that realism is a fictional cine-real that relies

more on subjective expectations than actual physics. By combining realistic recordings of

the car engines with more subjective sounds, the end result is a sound that feels more real

for the player than if the sound relied on physics or on fantasy alone. This illusory aspect

of sound design is a critical component in understanding how sound functions in all

media.

25
26
References

Atkinson, S. (2011), Surrounded by sound: The aesthetics of multichannel and

hypersonic soundscapes and aural architectures, The Soundtrack 4: 1, 521.

Audio Engineering Society. (2013). AES PNW Section Meeting Report: Vroom: The

Interesting World of Racing Game Sound Design (audio and video).

<http://www.aes.org/sections/pnw/pnwrecaps/2013/jan_vroom/ > [accessed 21 June

2014].

Austen, I. (2001). Whats next; Multiple Microphones Create Order Out of Sonic

Chaos. New York Times, September 13 2001.

<http://www.nytimes.com/2001/09/13/technology/what-s-next-multiple-microphones-

create-order-out-of-sonic-chaos.html> [accessed 21 June 2014].

Berg, J. (2009). The contrasting and conflicting definitions of envelopment. Paper

presented at the 126th AES Convention, May 710, in Mnich, Germany.

Borgomeo, V. (1997). Langelo rosso. Rome: Edizioni Lavoro.

Brodsky, W. (2001). The effects of music tempo on simulated driving performance and

vehicular control, Traffic Psychology and Behaviour, 4,(4): 219-241.

27
Cassidy, G.G. and R.A.R MacDonald (2010). The effects of music on time perception

and performance of a driving game. Scandinavian Journal of Psychology, 51: 455464.

Caswell, B. (2011). The sound of 62 years of F1 cars launching. Jalopnik.

<http://jalopnik.com/5819077/the-sound-of-62-years-worth-of-f1-cars-launching> [accessed 21

June 2014].

Collins, K. (2008). Game Sound: An Introduction to the History, Theory and Practice of

Video Game Music and Sound Design. Cambridge, MA: The MIT Press.

Collins, K. (2013). Sonic Subjectivity and Auditory Perspective in Ratatouille. Animation

Journal. 8(3): 283-299.

Evans-Thirlwell, E. (2013). Forza 5s audio revolution - lion roars, human screaming

and dynamic scores. Xbox Official Magazine. <http://www.totalxbox.com/57555/forza-

5-gameplay-video-and-interview-turn-10s-audio-revolution/ > [accessed 21 June 2014].

Flaks, J., Bar-Zeev, A and J. Tardif (2012). System and Method for High-Precision 3-

Dimensional Audio For Augmented Reality. US Patent Application 20120093320

Fogarty, M. (2013). Formula Ones Quiet Revolution. Sydney Morning Herald, March

16 2013. <http://www.smh.com.au/sport/motorsport/formula-ones-quiet-revolution-

20130315-2g64c.html> [accessed 21 June 2014].

28
Goodwin, S N. (2009). 3-d sound for 3-d games - beyond 5.1, in Audio Engineering

Society Conference: 35th International Conference: Audio for Games, London, February

11-13.

Grimshaw, M. (2007). The Acoustic Ecology of the First-person Shooter. PhD Diss.,

University of Waikato.

Heinen, M. (2013). Audio Special: The 3D future of Game Sound. Develop Magazine.

< http://www.develop-online.net/analysis/audio-special-the-3d-future-of-game-

sound/0117609> [accessed 21 June 2014].

Hope, C. (2009). The Bottom End of Cinema: Low frequency effects in soundtrack

composition. In Sound Scripts Vol 2: Proceedings of the 2007 Totally Huge New Music

Festival Conference. Hope, C and J.W. Marshall (eds.) Western Australian Academy of

Performing Arts Faculty and Education Edith Cowan University Australian Music

Centre, 74-79

Huiberts, S. (2010). Captivating Sound: The Role of Audio for Immersion in Computer

Games. PhD. Diss, Utrecht School of the Arts.

Kastbauer, D. and A. Woldhek (2012). Game Audio Podcast #18.

<http://www.gameaudiopodcast.com/?p=399> [accessed 21 June 2014].

29
Kastbauer, D. and D. Nichols (2012). Racing Game Sound Study

<http://blog.lostchocolatelab.com/2012/05/racing-game-sound-study.html> [accessed 21

June 2014].

Kerins, M. (2010). Beyond Dolby (Stereo): Cinema in the Digital Sound Age.

Bloomington: Indiana University Press.

Kerins, M. (2013). Multichannel gaming and the aesthetics of interactive surround.

Oxford Handbook of Audio-Visual Aesthetics, 585-602. Oxford: Oxford University Press.

Manolas, C. and Pauletto, S. (2009), Enlarging the Diegetic Space: Uses of the Multi-

channel Soundtrack in Cinematic Narrative, The Soundtrack, 2: 1, pp. 3955.

Nichols, David. (2011). Mike Caviezel. THE Mike Caviezel. TrackTime Audio.

<http://www.tracktimeaudio.com/?p=207> [accessed 21 June 2014].

Nichols, D. (2012a). Interview with Nick Wiswell, Audio Director of Turn 10 part 2.

Designing Sound. <http://designingsound.org/2012/04/interview-with-nick-wiswell-

audio-director-of-turn-10-part-2/> [accessed 21 June 2014].

Nichols, D. (2012b). Nick Wiswell Returns for Part 2! TrackTime Audio.

<http://www.tracktimeaudio.com/?p=411 > [accessed 21 June 2014].

30
Nichols, D. A. (2012c). The FM4 Great, Nick Wiswell. TrackTime Audio.

<http://www.tracktimeaudio.com/?p=365> [accessed 21 June 2014].

North, A. C. and D.J. Hargreaves (1999). Music and driving game performance.

Scandinavian Journal of Psychology, 40(4): 285292.

Petkovi, M., Mihajlovi, V. and Jonker, W. (2002). Extending a DBMS to Support

Content-Based Video Retrieval: A Formula 1 Case Study. Lecture Notes in Computer

Science. Berlin: Springer.

PSNEurope (2013). Against The Clock: Formula One Coverage in Italy. 01 July 2013.

<http://www.psneurope.com/newsletteraudio4broadcast-content/full/against-the-clock-

formula-one-coverage-in-italy> [accessed 21 June 2014].

Renault (2013). Come on Feel the Noise RenaultSport.

<http://www.renaultsport.com/Come-on-feel-the-noise,2630.html?lang=en> [accessed 21

June 2014].

Rumsey, F. (2002). Spatial Quality Evaluation for Reproduced Sound: Terminology,

Meaning, and a Scene-Based Paradigm. Journal of the Audio Engineering Society 50(9):

651666.

31
Sergi, G. (2004). The Dolby Era: Film Sound in Contemporary Hollywood. Manchester:

Manchester University Press

Skalski, P. and R.T Whitbred (2010). Image versus Sound: A Comparison of Formal

Feature Effects on Presence and Video Game Enjoyment PsychNology Journal 8(1): 67

84.

Stevens, P. (2011). Using a New Formula: TVBEurope: The Workflow.

<http://www.vialite.com/content/downloads/SIS_Live_Formula_One_Coverage.pdf >

[accessed 21 June 2014].

Stevens, R. and D. Raybould. (2011). The Game Audio Tutorial: A Practical Guide to

Sound and Music for Interactive Games. London: Focal Press

Stockburger, A. (2003). The Game Environment from an Auditive Perspective. In

Proceedings of Level Up, Digital Games Research Conference, November 46, in

Utrecht, NL.<http://www.audiogames.net/pics/upload/gameenvironment.htm> [accessed

21 June 2014] .

Tsingos, N, C. Chabanne, C. Robinson and M. McCallus. (2010). Surround Sound with

Height in Games using Dolby Pro Logic IIz. Audio Engineering Society 129th

Convention, 2010 November 4-7 San Francisco.

32
Audio-Visual References

BioShock (2007) Irrational Games

DiRT (2007) Codemasters.

F1 2011 (2011) Codemasters.

Forza Motorsport 3 (2009) Turn 10.

Forza Motorsport 4 (2011) Turn 10.

Forza Motorsport 5 (2013) Turn 10.

Gran Track 10 (1974) Atari.

GRID (2008) Codemasters.

Space Race (1974) Atari.

Gran Turismo 5 (2010). Polyphony Digital.

Shift 2: Unleashed (2011) Slightly Mad Studios.

Project Gotham Racing 2 (2003) Bizarre Creations.

33