Why xHE-AAC is staying embraced at Meta

• We’re sharing how Meta provides higher-high-quality audio at scale with the xHE-AAC audio codec.
• xHE-AAC has previously been deployed on Fb and Instagram to deliver increased audio for options like Reels and Stories. 

At Meta, we provide just about every media use scenario conceivable for billions of folks across the globe — from short-type, user-created content material, these types of as Reels, to quality video clip on desire (VOD) and dwell broadcasts. Supplied this, we need to have a next-technology audio codec that supports a variety of running factors with outstanding compression efficiency and modern, technique-level audio functions. 

To address these needs now and into the future, Meta has embraced xHE-AAC as the automobile for providing superior-excellent audio at scale.

The added benefits of xHE-AAC

xHE-AAC is the hottest member of the MPEG AAC audio codec household. The Fraunhofer Institute for Integrated Circuits IIS performed a sizeable position in the growth of xHE-AAC and the MPEG-D DRC standard.

Now, xHE-AAC is presently supplying a exceptional audio encounter on Facebook and Instagram — like on Reels and Stories — and has a number of valuable functions.

Loudness administration

With hundreds of millions of uploads for each day across Fb and Instagram, we get audio tracks with loudness amounts ranging from silence to full scale, and all the things in involving. 

When people play these video clips sequentially, they can understand some audio as remaining too loud or way too tranquil. This produces listener exhaustion from acquiring to consistently regulate the quantity.

xHE-AAC’s built-in loudness administration procedure solves for loudness inconsistency even though meticulously preserving creator intent by bringing the average loudness of all periods to the same focus on level and controlling the dynamic selection of just about every session to match the playback ecosystem.

Instead of burning in a particular focus on degree and dynamic vary compression (DRC) profile during encoding, xHE-AAC permits us to depart the first audio properties untouched and delegate loudness administration processing to the shopper through loudness metadata, for the optimum audio expertise based mostly on context. 

As a result of xHE-AAC’s loudness management, folks can shell out far more time immersed in their beloved information and considerably less time fiddling with the quantity management.

Adaptive bit level audio

Most people who use our apps consume media on mobile units and hope the greatest audio high quality without having interruption. This provides a problem for streaming media for the reason that connection quality varies on cellular and can final result in a incredibly uneven consumer encounter. 

To improve high quality less than dynamic bandwidth constraints, we make various video clip and audio features to match different community situations at playback time. Even though we produce a number of audio lanes, we have traditionally only used adaptive little bit fee (ABR) algorithms to swap video clip features throughout playback since it is difficult to permit adaptive little bit level audio with out compromising high quality through lane transitions.

In get to empower seamless audio ABR, xHE-AAC introduces the concept of rapid playout frames (IPFs) that contain all the knowledge important to start participating in a new audio lane devoid of relying on facts from other frames. By placing an IPF at the starting of just about every Dynamic Adaptive Streaming in excess of HTTP (Sprint) segment and aligning the segment durations of just about every lane, we can seamlessly change in between audio lanes in the course of playback to offer the best-high-quality audio at any out there bandwidth even though averting playback stalls.

Just after launching audio ABR on Fb for Android, we had been ready to make improvements to user knowledge by lessening the selection of classes in which playback stalls. 

How we deployed xHE-AAC

We generate xHE-AAC bitstreams making use of an encoder SDK furnished by the Fraunhofer Institute for Integrated Circuits IIS, and then put together the resulting audio information for Dash streaming with shaka-packager. The xHE-AAC encoder’s two-move encoding method is applied to measure the input loudness envelope and regular program loudness on the 1st go and complete the actual audio info compression on the 2nd go. As an added reward, two-pass encoding makes it possible for us to use loudness vary command (LRAC) DRC, which mitigates pumping artifacts or else released by solitary-pass DRC algorithms.

To put together an xHE-AAC audio adaptation established for ABR shipping and delivery, IPFs are inserted at consistent time intervals, audio configuration parameters these types of as sample price and channel configuration are kept continual, and one of a kind stream identifiers are chosen for each and every lane in the audio adaptation set.

At playback time, we personalized-fit the audio to the listening atmosphere by configuring a target loudness amount and DRC influence sort based on context, and thanks to the embedded loudness metadata, we can adapt a one xHE-AAC bitstream to a wide range of audio use use scenarios, from headphones to system speakers and several amounts of track record sounds. Finally, if the customer is starved for facts or bandwidth is plentiful, audio ABR will instantly change audio features to ensure that the best audio high-quality is performed without interrupting the playback session.

Exactly where can you experience xHE-AAC right now?

You can practical experience xHE-AAC audio on Fb for iOS and Android, as very well as on qualified surfaces on Instagram, these types of as Reels and Tales. We stimulate you to install the most up-to-date variation of Fb and Instagram apps on iOS 13+ and Android 9+ to ensure that you can encounter it.

Acknowledgements

This operate is the collective end result of the full Video Infrastructure and Instagram Media System teams at Meta in collaboration with Fraunhofer Institute for Integrated Circuits IIS. The creator would like to extend distinctive thanks to Abhishek Gera, Tim Harris, Arun Kotidath, Edward Li, Meng Li, Srinivas Lingutla, Denise Noyes, Mohanish Penta, David Ronca, Haixia Shi, Mike Starr, Cosmin Stejerean, Simha Venkataramaiah, Juehui Zhang, Runshen Zhu, and the engineering crew at Fraunhofer Institute for Integrated Circuits IIS.