This page was first published 2016 on blog.br.de/highend. As BR 2021 decided to terminate the support, I achieved permission to keep this page accessible on my own behalf.
BINAURAL AUDIO: TECHNOLOGIES AND DEMOS
>Listen to all these examples with headphones to experience 3D sound delivered to your ears! In order to get the spatial audio and video effects of the embedded YouTube films, use Firefox or Chrome on desktops or open the films straight in the YouTube app on mobiles on both, Android and iOS.
- STATIC BINAURAL
Dummy Head Recording
Dummy Head Re-Enactment
Binaural `baking’´ in production
`Static´ means: the binaural effect is not responding to the listener moving his head.
Dummy Head Recording
Starting in 1973, ARD Radio (German public broadcasters) have made more than 100 productions of music, radios drama and documentaries using dummy heads.
- The Original Dummy Head Stereo demo disc from Sennheiser (1973)
Although this technology, due to the new developments we discuss here below, seems to be outdated these days – using a real dummy head has still its charms. Quite a few radio dramas and documentaries have been produced in recent years with real dummy heads. Also German public service radio stations regularly replay old productions.
A research project of Stefan Krebs from the C2DH – Luxembourg Centre for Contemporary and Digital History – has provided interesting insights in the technical developments and the application of the dummy head in broadcasting – mainly in Germany during the 1970’s. The sheer numbers of productions Stefan recorded are impressive.
Dummy Head Re-Enactment (November 2016)
Together with Stefan we have organised a dummy head radio drama production re-enactment at BR, involving also the developer of the Neumann KU-100 Stefan Peus and former BR sound engineer Günter Hess, who had recorded many binaural dramas in the 1970s. We’ve produced a little radio play, telling about the history of ‚dummy head stereo‘ and demonstrating the stunning quality of binaural sound from an original KU-81. Here is the slideshow version of the play (switch on the subtitles in English if you don’t speak German) – And of course you have to wear headphones in order to experience the 3D sound yourself!
In the piece, the dummy head himself reflects on being a dummy head, hearing like a human being, but being neglected for a long time. This role was taken over by famous actor, Hans-Peter Hallwachs, who was actually playing the main character in the first dummy head stereo radio play ever on German radio, ‚Demolition‘ in 1973.
Binaural microphones (OKM – Original Head Mics)
Sound obsessed creators of radio documentaries like Ekkehard Kühn and Hansjörg Schmitthenner from BR started early to use portable binaural microphones for field recording. Inspired by them, already in 1993 I bought the excellent Soundman OKM (which are estimated to almost match with the microphones of original Neumann dummy heads), recording onto the portable Sony TCD-D3 digital DAT machine.
Here are just two examples from reports I did for my own radio documentaries:
- At Abbey Road Studios London with Alan Parsons (wb 1997)
- With Henry Steinway in Steinway Hall new York (wb 1996)
In 2017 Sennheiser introduced the AMBEO Smart Headset, a device that comprises of a pair of headphones with an adjustable noise cancellation system plus built-in binaural microphones on their outside. The sound quality of both units is superb too. This even allows for immersive live radio reporting with AOIP being connected to the studio – provided both sides are configured for stereo and not just the default mono connection. The reporter is still able to hear and talk to the host or producer in broadcasting house while delivering an impressive binaural soundscape from the location.
Binaural `baking’´ in production
Due to the growing interest, broadcasters like BBC, Radio France and ARD in Gemany recently started to use binaural for `special mixes´ of productions aired in normal stereo. These versions are designed for dedicated headphone listening on mobile devices.
- 24 hours in Virtual Reality (BR 2016 – excerpt – slideshow by wb)
In January 2019, my own radio documentary for BR-KLASSIK on the history of music on the radio was also produced with ‚baked binaural‘. It is avaible here (German language only):
DYNAMIC BINAURAL RENDERING
`Dynamic’´ means: If you move your head, the audio scene changes accordingly. Therefore this is essential to achieve a realistic ‚immersive effect‘ in any VR and 360º video application. Headtracking or motion sensors are used to control the parameters of dynamic binaural rendering systems.
The 360 video functions of YouTube and Facebook support this feature since spring of 2016, by using `Ambisonics´.
Easy to achieve are recordings with a Zoom H2N in „spatial audio mode“ FOA ambiX (without height z channel). Along with a 360 video or still picture you can shoot it up straight on YouTube.
Here are two examples from MozFest London October 2016:
For the first one the 360° picture was taken with a Ricoh Theta S
This second one is visually imperfect (300° iPhone panorama pic blown up to equirectangular ratio 2:1) – but it’s audiowise better, with the speaker as one distinctive sound source.
This example – a rehearsal of the Bavarian Radio Symphony Orchestra with chief conductor Mariss Jansons during their Asian tour 2016 – was captured by my colleague Felix Hentschel at Tokio’s famous Suntory Hall with a Kodak 360 camara and a Zoom H2N at the same position.
The Sennheiser AMBEO microphone got very popular for capturing First Order Ambisonics properly in all three dimensions X,Y,Z for 360º video productions. Of course, being a tetrahedron sound field microphone – according to the principles of its inventors, Michael Gerzon and Peter Craven – some audio processing has to be made, transforming the A-Format signal into B-Format ambiX before muxing it with a 360º video and uploading to YouTube.
Here, the virtuosic Munich based ensemble Passo Avanti (Eugen Bazijan, Alexander von Hagke, Julia Vom Deich and Vladimir Grizelj) perform the cute Mozart Gigue KV 574, specially arranged in 360° by Alex, which ’spatializes‘ the main melody by dividing it into the different instruments. Move around and try to catch it! (January 2017)
As we’ve captured the Passo Avanti session in various 3DAudio microphone techniques, you may also compare the artificially binaural rendered YouTube version with the ‚human‘ ears of Neumann’s legendary KU-100 dummy head . But of course: this is ’static binaural‘.
Another experiment we made was during the above mentioned „dummy head stereo re-enactment“ (November 2016) in collaboration with Stefan Krebs‘ „Kunstkopf stereophony“ research project of the University of Luxembourg. Here’s an excerpt of that little radio play: A journalist (Stefan Wurfbaum, first located on the right), interviews a media archeologist (Stefan Fickers, left) about the Neumann KU81 dummy head (viewable in the 360º video above you). In the middle of the piece they quote enthusiastic reactions from listeners and the press in the 1970s, thus demonstrating also the capability of the dummy head to capture 3D binaural sound. Actually, the sound for this 360º video was captured through the Sennheiser AMBEO you see in front of you in FOA. Only through the YouTube spatial audio rendering process it is transferred into binaural sound (or multichannel if you hook up your laptopn via HDMI to a surround sound home system).
Facebook also uses Ambisonics and their rendering and reproductionprocess is regarded superior to that of YouTube. When creating a mix with the Facebook 360 Spatial Workstation tools you basically work already in the higher spatial audio resolution of Second or Third Order Ambisonics . In addition, the Facebook tools offer a `headlocked’´ option, for sounds to remain static . Thus some interesting ‚in head‘ VS externalization effects are achievable and may add to the ’story-to-tell‘ (you can create this also with other means, but it’s a little bit more work to do).
Facebook’s interactive spatial audio is also reproduced in both iOS and Android apps out of the box – but for desktop browsers, only Chrome is at the moment officially supported (unless you trick other browsers to be Chrome).
- On the Eve of World War I (2014-2016)
Generated in the browser or in app on the device. FOA (First Order Ambisoncs) with 4 audio channels is used as input by YouTube and rendered into binaural.
Experience audio-only by swiping with your finger or by moving your device or mouse.
This version has an experimental 360 slideshow for illustration
The emerging, future-oriented technology is `object-based audio´. With this approach, an even more versatile adaptive and flexible reproduction of audio is possible: The rendering in the device is optimised not only for binaural or scalable loudspeaker set-ups (mono, stereo, 5.1….) but also in terms of foreground-background audio adjustment, dynamic control, alternative laguages, subitles, or even more interactive options like variable length and even skipping or replacing content parts etc.
The demo piece on WWI above was originally made with an object-based editor and then rendered into FOA to match with YouTube requirements. This screencast shows the objects in the 360 audio sphere. The audio is static binaural (center position facing north).
You can listen here to the binaural rendered mix of this production
Here’s a demo, where the listener as well as the audio objects can be freely moved within a virtual room. You may switch between standard stereo or binaural rendering and all this happens in the browser!
Note: This demo may not run in all browsers and devices.
You can find the ressources here on GitHub
More demos can be found on IRT’s Lab page
The pioneering ORPHEUS project
Between December 2015 and May 2018, under the EU HORIZON 2020 programme BR was a partner in the ORPHEUS-AUDIO consortium. 10 major European players of media technology sucessfully developed and implemented an end-to-end object-based broadcasting solution and demonstrated the benefits of this technology for the audience in interesting pilot pieces for various broadcasting formats and genres.
With the technology and standards developed in the ORPHEUS project, the EBU along with the BBC, Radio France and others are well under way to transform and adapt `object-based audio’´ and Next Generation Audio in recording, production and distribution of broadcast content.
AUDITORY AUGMENTED REALITY
For ICSA 2019 TU Ilmenau’s Media Technology department choose the WWI piece as a demo for their newly developed Auditory Augmented Reality System (http://bit.ly/AARS-TUI). Their team adapted and implemented not only the simulataneous scene seen above, but also all 4 scenes as an interactive 3D Audio soundscape. A tracking system and a low latency wireless headphone system allowed for two users at once to move freely in the room and explore the various sound sources positioned there. Take a glimpse what a realistic impression this technology creates by watching this video. The binaural sound was captured ‚live‘ using the Sennheiser Ambeo Smart Headset in-ear microphones under the Beyer DT 1990 headphones.
And once again: Wear headphones to experience the 3D binaural sound!
Stay tuned for more – link up to: