By Kevin Matthews
An invisible choir hovers in the air behind Brett Leonard ’06, wrapping around from the 3 o’clock position to 9 o’clock and high over his head, because he is a sound engineer and that is where he’s put the singers. Also unseen, the English folk rockers Mumford & Sons and Nashville’s Jerry Douglas, his hands on top of a Dobro resonator guitar, wait up front at stage height.
On this day at McGill University in Montreal, a couple of years ago, Leonard is in possession of original tracks from a recording session by the musicians, who’ve collaborated on their cover of “The Boxer” by Simon & Garfunkel. Lie-la-lie, lie-la-lie-lie-lie-lie-lie.
When the song starts, the small audience is not in Canada anymore. “All the sudden you’re in this close, intimate finger-picking guitar and a single vocalist singing, and then a little lap steel comes in, then bass, and then this choir comes in with the Lie-la-lie part during the chorus, and it blows your mind,” he says. “It’s like you’re in a church.”
Piano, banjo, mandolin and drums arrive from slightly different angles during the song, reverberating off the walls, ceiling and floor. Though not wholly unlike 360-degree surround sound – an audio experience familiar to every moviegoer – this sort of vivid, three-dimensional soundscape is the hallmark of a distinct new phase in audio reproduction.
Given a proper 3-D audio array, sound engineers have so many tools and so much freedom that it’s relatively easy to immerse listeners in a lifelike simulation. The big challenge of re-mixing a song in 3-D is to keep the experience believable and, in various ways, grounded. Amateur work could transport listeners inside of virtual grain silos or cast voices and instruments about like stars at a planetarium. Or it could simply fail, with unstable and unpredictable results that depend too much on where the listeners sit.
A professor at the University of Nebraska at Omaha and, concurrently, a doctoral candidate in sound recording at
McGill, Leonard likes to talk about today’s new audio experiences as hyperreal, in the sense that they offer heightened sensation and emotion. That aim calls for different strategies on different kinds of projects, but it takes for granted that audio playback can be as direct as any listening experience.
Much like reading a book, the act of watching a movie or playing a video game has always required a willing suspension of disbelief, to apply S. T. Coleridge’s words. You accept fantasy as truth for a while. With eyes closed, you may also imagine that you’re listening to a musical performance rather than a recording.
But as far as sound is concerned, that gap between dreaming and waking, between will and belief, is closing fast. We’re reaching a point where, instead of just playing along, we’ll often have to remind ourselves that experiences with recorded and interactive media aren’t in every sense real.
One of the first Cal Lutheran music majors to graduate with an emphasis in the technology of music production – which is now a separate, fast-growing major – Leonard “lived in the studio” at the Spies-Bornemann Center, where he began developing chops as a sound editor and mixer. As the drummer in a band, he also worked on a CD with friends in his dorm.
These days he splits his time among studios, his teaching in Omaha, and live concerts and music festivals, now always as a recording engineer. He is finishing a dissertation and also working with a team at McGill to commercialize a high-end tool for sound professionals under the name of Space Builder. Among other things, the system allows mixers to reliably reproduce sounds that reflect off of walls and ceilings.
Broadly, new 3-D technologies enable two kinds of audio simulation. First, you can now play back a concert with sound that mimics the acoustical features of an original venue. This is one of the applications of Space Builder, a project that spun out of years of work by McGill researcher Wieslaw Woszczyk and others to capture the way that Haydn’s music sounded in the grand salons of Austria and Hungary.
Second, it’s lately become feasible to show your friend Petra what a concert sounded like to Pablo, not that this is necessarily a great idea. As Leonard explains, the human head is an acoustical filter, and since everyone knows the world of sound through his or her own, a simulation passes as “natural” to the degree that it corresponds to an individual’s physiology – “the variation in people’s head size and shape, their earlobe, their pinna’s shape and contour, even the depth of their ear canal.”
These data turn out to matter for sound engineers in some settings, and they may to be crucial for the future development of audio headphones. With just stereo technology and an ordinary pair of earbuds, the means are now available to trick listeners into believing that recorded sounds come from multiple directions. Next up, along with better home theaters, could be headphones sold in hundreds of virtual “sizes” for playing immersive audio on plane trips and hikes.
At 32, Leonard is old enough to remember audiocassette tapes and the “dark days in the Kazaa/Napster era where everything was very low-quality MP3,” a digital compression format that “wrecked audio as it was being coded.”
Given that an entire generation “grew up with bad sound,” he says, the most exciting part of working in the audio field is the return of quality. With the arrival of immersive sound, websites for downloading high-resolution audio and related developments, there’s suddenly a better chance to educate students and consumers.
Today, big-budget motion pictures are routinely recorded in Dolby Atmos, a 3-D surround setup, and hundreds of theaters, including in Nebraska, are equipped to play them back as intended. Now and then, that gives Leonard the chance to ask an unsuspecting viewer about the experience. Did you notice anything different? Anything about the sound?
Although the responses vary, they are usually positive, Leonard says. Often, it’s something indefinable: the movie just felt more real than normal. In a few cases, the person may walk out a little rattled, like someone who just stepped off a roller coaster.
“There have been people who have been kind of creeped out,” Leonard said. “When it’s done well, it’s really hard to separate reality and fantasy.”