6 Reasons Why You Should Focus On 360° Audio
As 360° video is being adopted into the mainstream, more focus is now being put on the audio that accompanies it. Consumers expect an immersive experience that includes the sound track. We spoke with two VR industry experts from Littlstar and MANDT VR about what’s happening in 360° audio today. Here are six takeaways from our conversations:
1. 360° audio describes an environment.
360° audio, also known as spatial or binaural audio, has existed for at least 30 years. Traditionally, these audio soundtracks were created by surrounding the listener with a cube of speakers. Today, 360° audio can distill detailed sound information into headphones by head tracking. When you look around in a VR headset, you change your viewing angle, your perspective, and the length of the sound waves bouncing off of the objects around you and back to your ears. Head tracking, or the way the VR headset tracks your movements, takes the information that the virtual reality headsets are sending the player to change the video rotation and perspective.
360° audio takes that same approach. In the real world, when you move your head you perceive a change in the direction that the sound is coming from. To achieve this effect, the audio is anchored to objects in the VR world surrounding the viewer, so if they turn their head they hear the same sound slight differently, just as we perceive a change in the direction of a sound in the real world. So now, you don’t need a bunch of speakers to represent this information — you can just wear headphones.
The difference between typical audio mixes and 360° audio is significant. Andrew Grathwohl, the Director of Media Technology at Littlstar explained the difference:
“With 360° audio, you are designing a sound world. Spatial audio tries to describe an environment with sounds, whereas in most traditional audio setups you can only hear that the sound came from the left or the right. So you get that extra axis — you can get height as well as left-right and then you can go even further and get depth, so that you can actually hear how close sound is to you. It’s pretty remarkable to experience.” — Andrew Grathwohl
To get the 360° audio experience, sound engineers set up multiple microphones to pick up any ambient sound in the shot. Once they collect the sounds that are going to make the spatial audio mix or soundtrack, a post-production team processes multiple sound channels further to get the right dynamics, loudness, and levels of the audio.
2. Advances in audio tech has made 360° more accessible.
Before recent advances in VR camera, microphone, and headphone technology that have made capturing 360° audio more efficient and affordable, most of a producer’s budget was taken up by the visual aspect of VR, and audio was pushed to the backburner. Chadwick Turner, the Chief Strategy Officer of MANDT VR described the budget struggle and the rise of immersive audio:
“Most people I talk to — even some very credible professionals have almost no budget for their projects — try to find out ways to be truly creative and leverage what they want to do without breaking the bank. This has changed recently with new camera and audio technology. You can look at the development of 360° microphones that try to mimic sound hitting your ear or the ossic headphones that make people excited about the ability to consume really high quality audio that feels immersive. Sound is finally getting its proper due.” — Chadwick Turner
3. 360° audio is different for journalistic and cinematic projects.
Spatial audio is now central to VR films, documentaries, and journalistic reporting. It makes the VR experience more immersive as a whole, which engages the audience’s empathy. But there are some audio differences between the genres.
“VR audio for journalism tries to engage the user’s empathy and lets us situate the viewer in something pretty unique that they couldn’t otherwise experience, like The Economist’s Mosul museum tour. But, when you look at the cinematic partners we have, the main difference there is they take really interesting and beautiful artistic liberties with this new format and content is oftentimes music-driven. The difference between the two is the intent of the audio, and for that reason the media often ends up looking very different.” — Andrew Grathwohl
4. 360° audio keeps the storyline moving.
In VR films, audio can be used to prompt the audience or further the storyline. Creators can prompt the viewer to look a certain direction at a certain time, which keeps the story moving. Andrew explained the potential he sees with 360° audio to direct viewers:
“The cool thing about spatial audio is that you can use the spatialization — the very precise direction that sound has in this world — to trigger the viewer to look a certain way. We react very predictably to sounds. For example, when they are behind us or when they are above us we instinctively look that way. When sound is coming from a place you don’t expect it to, spatial audio can be used really cleverly to get the viewer to look where the content creator wants them to look. We’ve seen some beautiful work done cinematically, for example Rapid Fire: a brief history of flight. They used spacial audio to direct the viewer to the different planes that were flying in the air above and without that spacial audio soundtrack it’s very likely that the viewer may miss many of the finer details of the story because they can’t look everywhere.” — Andrew Grathwohl
5. Sound accuracy is important.
Since spatial audio requires a lot of post-production work, it can be easily manipulated. This means that journalists need to be careful not to use audio cues that could mislead viewers or portray the subject inaccurately. While the ability to hone in on the details of a situation can be exploited in VR journalism, it can be also be used to report more accurately by getting the viewer closer to the real experience:
“We had a lot of media coming in from various professional companies that were shooting crowds and rallies and events where there’s a large number of people given the election that just happened. Standing in a crowd doesn’t feel like much with stereo sound, but when you experience it with 360° audio you can close your eyes and can tell where people are speaking from and you can turn and look at them. Audio just needs to be used in a tasteful way, as opposed to gain a cinematic effect. When you’re in a journalistic environment, the accuracy of the sound pickup is important.” — Andrew Grathwohl
6. Sound requires expertise.
The biggest challenge facing 360° audio is how new it is to most video creators since most digital media companies today use stereo audio. Going from one or two channels to as many as 143 channels of sound requires specialized sound technicians with the skills to handle the demands of 360° audio content.
“Production is more expensive, it takes a lot longer, you’ve got to find the right people — not just people who focus on the visual side of things. We need to experiment more as an industry and we’ll definitely pare down all these different options over time once we see what works in an at-scale production environment.” — Andrew Grathwohl
By Jindalae Suh, Writer at Storyhunter
