How Apple developed the world’s first end-to-end hearing health experience

October 28, 2024

UPDATE

Inside the Audio Lab:
How Apple developed the world’s first end‑to‑end hearing health experience

The new hearing health features on AirPods Pro 2 are not available in all regions. For more information, visit apple.com/airpods-pro/feature-availability

Apple’s state-of-the-art Audio Lab in Cupertino, California, supports the innovative work of its acoustic engineers. They use the lab to conduct user studies in various listening rooms and test new features in its anechoic chambers, which completely absorb reflective sounds and isolate external noise.

The Audio Lab is the hub for the design, measurement, tuning, and validation of all of Apple’s products with speakers or microphones. It’s also the center for Apple’s multiyear, cross-team collaboration to build the groundbreaking new hearing health features on AirPods Pro 2. Available today as a free software update,¹ the end-to-end experience helps minimize exposure to loud environmental noise with Hearing Protection, track hearing with an at-home Hearing Test, and receive assistance for perceived mild to moderate hearing loss using AirPods Pro as a clinical-grade Hearing Aid.

According to the World Health Organization, approximately 1.5 billion people around the world are living with hearing loss. “Hearing loss affects individuals in every region and country, yet often goes unrecognized. Hearing is a core component of communication for so many and is an important factor for health and wellbeing,” says Shelly Chadha, M.D., the World Health Organization’s technical lead for hearing. “Technology can play an important role in raising awareness and providing intervention options for those affected by hearing loss.”

“Every person’s hearing is different, so we created an innovative, end-to-end hearing health experience that addresses this variability in a way that’s both simple to use and adaptable to a wide range of needs. That’s especially important because hearing loss affects people of all ages with different levels of tech savviness,” says Sumbul Desai, M.D., Apple’s vice president of Health. “With the Hearing Aid feature, we wanted to build something so intuitive, it felt like an extension of your senses. We knew the results would literally change people’s lives — and democratize access to treatment for a condition that affects more than a billion people.”

Engineers used highly specialized spaces across the Audio Lab to help make these breakthrough features possible.

“From the quietest sounds we can hear for the Hearing Test feature, to speech in noisy restaurants for the Hearing Aid feature, and even concert levels for Hearing Protection, we can bring the real world into our acoustics facilities with playback of calibrated soundscapes from all over the world, or take accurate acoustic measurements at the touch of a button,” says Kuba Mazur, Apple’s hearing health lead engineer within Acoustics Engineering.

The Longwave anechoic chamber was built on a separate foundation that uses springs to isolate it from the rest of the lab, allowing for accurate sound measurements without any noise or vibration disturbances. The chamber includes a custom-built loudspeaker and microphone arc that can measure head-related transfer functions, or in other words, how sound interacts with the human body. Having both the loudspeaker and microphone arrays within this chamber makes it a unique space with many applications, including AirPods, iPhone, and HomePod development.

“Your ears are natural amplifiers, each uniquely shaped and often slightly asymmetric,” Mazur continues. “When sound reaches one ear first before the other, it creates a time difference in how we perceive sound. This is important for us to understand so we can build experiences that accurately represent the sounds in your environment. And we do this in our anechoic chambers by having someone sit in a rotating chair with AirPods Pro to capture the audio.”

On the other side of the Audio Lab, to ensure the highest sound quality in every audio product Apple makes, the Fantasia Lab uses a spherical array of 50 loudspeakers to simulate hundreds of real-world sound scenes — like a shopping mall, busy street, or travel on an airplane — in a tightly controlled, evenly distributed sound field.

To fine-tune and validate the Hearing Aid feature, a broad demographic of study participants with a wide range of hearing levels were put into this controlled environment to complete a speech-in-noise test. The test consisted of a participant sitting in a chair in the middle of the space while a complex sound scene, like a noisy restaurant, played. The participant then had to repeat the words of a single speaker, distinguishing from background conversations.

“This lab is about recreating. Just as our users experience their everyday lives moving through shopping malls or having dinner with loved ones, we had to ensure these features would meet their needs,” says Mazur. “We brought the outside in to tune and validate features that we’re building on AirPods, like the Hearing Aid feature, Conversation Boost, and Transparency mode.”

Additionally, three clinical-grade audiometric booths — the type that patients would typically encounter during hearing tests in a clinician’s office — are permanently installed in the Audio Lab. For internal testing, the engineering team worked with audiologists in the booths to conduct thousands of clinical-grade audiometry tests and software-based hearing tests prior to moving the new Hearing Test feature into clinical validation studies.

Design is also core to the user experience and played an important role in user testing of the new features. One key experience was taking the test itself. The team had to identify design approaches that would simplify the Hearing Test and Hearing Aid setup. It also needed to be easier to understand than the typical series of numbers a person receives during a doctor’s visit.

“Within our health features, we focus on clarity and meeting users where they’re at,” says Heather Daniel, a producer in Apple’s Design Studio who helps manage all of the design work for health features. “Take the Hearing Test feature. We understood that for many people, this might be their very first time taking a hearing test, so we had to make it as seamless as possible.”

Simplifying these experiences required teams across Apple working together every step of the way to build this software to meet the requirements for clinical testing and delivering the best product to customers.

“Just thinking about the innovation that was necessary, the density of the technology that goes into AirPods, and the amount of effort and attention to detail that went into building these complex software features,” Mazur continues, “so many teams came together — including software and hardware engineering, design, health, accessibility, clinical ops, regulatory, and the human factors engineering team — to ensure the best quality and experience.”

The end-to-end hearing health experience on AirPods Pro 2 is just the latest example of Apple’s commitment to helping users on their personal health journeys. For many team members, it’s the epitome of what is possible when innovation meets passion to deliver products and software that are useful and help improve users’ day to day.

“The fact that people can walk around wearing their AirPods, that they can protect their hearing at concerts and get insights on their hearing health using these features over time — AirPods are doing what each person wants or needs them to do,” says Mazur. “They’re truly the interface to the ear.”