Audio Implementation

Below are a few examples of the adaptive audio systems I have created for my own projects as well as tips on writing and implementing adaptive music.

Dynamic Obstruction and Room Propagation System in Unreal Engine 5

In this video, I go over how my dynamic obstruction and room and portal system works and sounds. This system is built for reel/demo purposes.

Obstruction is based on the number of obstructing objects, the material of the obstructing objects, and how entirely they obstruct the sound path. Line traces are sent towards the listener from multiple positions around the emitter that are relative to the look-at rotation towards the listener. Multi-line traces are used to account for multiple obstructing objects, tags on the obstructing objects are compared against a Data Table to determine how much a certain material should filter the sound, and the multiple line trace positions around the emitter account for gradual obstruction and the acoustic concept of sound diffraction around objects. The diffraction values drive low-pass filters and attenuation in a MetaSound patch.

In the room and portal system (based on the Wwise Spatial Audio system of the same name), every room and doorway/window is assigned a room or portal actor respectively that fill the geometry of the actual room or doorway. If the listener is in a separate room to an emitter, the emitter is spatialized at the position of the connecting portal(s). This repositioning is gradual as the listener enters/exits room, and the emitter-portal-listener angle is used to drive a low-pass filter which simulates the effects of audio diffracting around a doorway. The repositioning is accomplished by sending the audio of emitters to audio Bus objects which are being sent to sources at the position of portals. All of the filtering and sending of audio is accomplished using MetaSounds.

Finally, emitters send to reverbs depending on the room the emitter is in. This is accomplished with SubMixes. Portal emitters send both to the reverb of the room the listener is not in, as well as the room the listener is in. This is to simulate emitters exciting the diffuse field of both their own room and rooms the sound will reach through portals.

In-depth videos explaining how this system is setup in Unreal Engine 5 using Blueprints and MetaSounds can be found below.

Part 1: Dynamic Obstruction

Part 2: Rooms and Reverbs

Vehicle Audio Implementation In Unreal Engine

In this video, I explain how I implemented audio for an off-road vehicle in Unreal Engine. All of the sounds are designed by me, and the implementation was done entirely in Unreal using Blueprints.

The engine and tire rolling sounds are loops of different intensities that crossfade between each other based on the RPM. There is a revving sound if you hit the throttle after not hitting it for a while, and a skid sound if you hit the brakes above a certain speed. Both of these sounds duck the engine and tire loops respectively.

The suspension and rattle noises are created by casting a ray from the body of the vehicle until it collides with the ground. If the ray length exceeds certain thresholds, rattle noises of varying intensities will play. If the ray length hits a significantly high threshold, the vehicle is considered to be jumping, and I fade the tire noise out. Once the ray length goes back below this threshold, the vehicle is considered to have landed, and I play a landing sound that varies in intensity based on how long the vehicle was in the air.

There are also collision sounds that vary in intensity based on the vehicle's velocity at the moment of collision.

Don't Miss Your Entrance!: Natural Layering

In this video, I explain how to add and remove layers in your adaptive music in a way that feels more natural.

This trick is pretty easy, and while it might not always be the best choice, I think it can really help your adaptive music feel convincing.

Normally, when you are adding or removing layers, you will fade them in and out. This technique uses the same idea but only fades the layers during moments of rest. If a layer only fades in or out during silence, you won't hear the fade, and it will sound like an instrumentalist is intentionally starting or stopping.

I did this uses Switch Tracks in Wwise with one switch being an empty track and the other being the layer. I only allow Wwise to transition between the switches during moments of rest. This could be on every bar or every beat if that works for your music, or you could manually add Custom Cues during moments of rest and have Wwise only transition during those Custom Cues.

This technique is not as immediate as fading in or out anytime since the music has to wait for a moment of silence, so this is best used when the music does not have to be quickly responsive. The composer also must write with this technique in mind which could be more difficult, but I always think it's great to compose with the implementation in mind anyway!

Following the Horror

See how I created a dynamic music system that is able to adapt to various situations in the horror game Shadownest

Shadownest has three major gameplay states:

Exploration: when the players are exploring outside of combat
Chase: When the monster is chasing the human players
Hiding: When a human player is currently hidden from the monster after an encounter

I created three music states and silence in Wwise to match these three gameplay states. Each state has beat-locked, musical transitions between them and has two additional sets of intensity layers that can come in and out based on the human players' Fear Level (parameter determined by player health and number of human-monster confrontations).

Hiding successfully might mean you evade the monster, but coming out too soon can lead to your doom! The Hiding state can very quickly and easily transition to the Chase state and visa-versa. The Hiding state also has an additional thumping bass layer that raises and lowers in volume based on how close the monster is to the human player.

Additionally, the Hiding state has Intensity Levels that will go down if the monster is far enough away and the human player is hiding for long enough. This allows hiding to feel intense the moment you stuff yourself in a closet, but to eventually ease off as you feel the coast is clear.

Finally, the Hiding and Chase states both contain random music tracks that contain different musical phrases each time the music loops. The multiple random tracks lead to many different combinations of music playing at once. This is done so the music always feels evolving and eerily inconsistent.

Sounds of the Forest

Polaris Postal Service is a game where you explore a magical forest and deliver letters to its inhabitants. Here's how a creating ambient music that could feel as natural and evolving as that forest

You spend most of your time in Polaris Postal Service exploring the forest and using the stars to guide you while you receive and deliver letters. As well as a dynamic forest ambiance that included owls hooting, crickets, and the wind rustling through the trees, I wanted to create a music system that would support the sound effects to create a continuously evolving forest bed.

The ambient music of the game is composed of many small musical phrases which are grouped into three categories: longer fuller ambient synth phrases, and two types of shorter, quieter phrases made from two different synth patches. I called them Delays and Plucks.

Wwise selects a category of phrase and then selects a music clip from within one of those categories. This way, the player will never hear the same type of phrase twice in a row, and they will never hear the exact same phrase played again shortly after it was played the first time.

The different categories of phrases with different levels of intensity between them create natural dynamics, and the silence between each phrase allows for the music to feel like it is coming in and out of the bed of forest ambient sounds.

In Polaris Postal Service, the player connects stars in the sky to create a constellation. Every time a star in the constellation is clicked on, a musical note plays. When the constellation is fully created, a musical stinger plays in the same key as those musical notes. These stingers toggle between the keys of G and F, so we made sure that the notes that play when you click on a star are always in the same key as the stinger that will play once the player finishes the constellation.

Game Music Design Videos

Check out an ever-growing playlist of videos where I explain the music design of different games!

https://youtube.com/playlist?list=PLNYNTrR8l3xqNtC9paaN7uHI5k4-kKSLt