Abstract:Commercial automatic speech recognition (ASR) systems underperform for d/Deaf and hard‐of‐hearing (d/Dhh) individuals, especially those with "low" and "medium" speech intelligibility classification, prelingual onset of hearing loss, and sign language as primary communication mode. There is a need for ASR systems ethically trained on heterogeneous d/Dhh speech data. Objective To evaluate the performance of commercial automatic speech recognition (ASR) systems on d/Deaf and hard‐of‐hearing (d/Dhh) speech. Methods A corpus containing 850 audio files of d/Dhh and normal hearing (NH) speech from the University of Memphis Speech Perception Assessment Laboratory was tested on four speech‐to‐text application program interfaces (APIs): Amazon Web Services, Microsoft Azure, Google Chirp, and OpenAI Whisper. We quantified the Word Error Rate (WER) of API transcriptions for 24 d/Dhh and nine NH participants and performed subgroup analysis by speech intelligibility classification (SIC), hearing loss (HL) onset, and primary communication mode. Results Mean WER averaged across APIs was 10 times higher for the d/Dhh group (52.6%) than the NH group (5.0%). APIs performed significantly worse for "low" and "medium" SIC (85.9% and 46.6% WER, respectively) as compared to "high" SIC group (9.5% WER, comparable to NH group). APIs performed significantly worse for speakers with prelingual HL relative to postlingual HL (80.5% and 37.1% WER, respectively). APIs performed significantly worse for speakers primarily communicating with sign language (70.2% WER) relative to speakers with both oral and sign language communication (51.5%) or oral communication only (19.7%). Conclusion Commercial ASR systems underperform for d/Dhh individuals, especially those with "low" and "medium" SIC, prelingual onset of HL, and sign language as primary communication mode. This contrasts with Big Tech companies' promises of accessibility, indicating the need for ASR systems ethically trained on heterogeneous d/Dhh speech data. Level of Evidence 3 Laryngoscope, 2024

Racial disparities in automated speech recognition

Reexamining Racial Disparities in Automatic Speech Recognition Performance: The Role of Confounding by Provenance

Hey ASR System! Why Aren't You More Inclusive? Automatic Speech Recognition Systems' Bias and Proposed Bias Mitigation Techniques. A Literature Review

Improving Speech Recognition for African American English With Audio Classification

Lost in Transcription: Identifying and Quantifying the Accuracy Biases of Automatic Speech Recognition Systems Against Disfluent Speech

Can Voice Assistants Be Microaggressors? Cross-Race Psychological Responses to Failures of Automatic Speech Recognition

Bias in Automated Speaker Recognition

AequeVox: Automated Fairness Testing of Speech Recognition Systems

Global Performance Disparities Between English-Language Accents in Automatic Speech Recognition

FairLENS: Assessing Fairness in Law Enforcement Speech Recognition

Quantification of Automatic Speech Recognition System Performance on d/Deaf and Hard of Hearing Speech

Quantifying Bias in Automatic Speech Recognition

Creating an African American-Sounding TTS: Guidelines, Technical Challenges,and Surprising Evaluations

Automatic Speech Recognition Services: Deaf and Hard-of-Hearing Usability

Self-supervised Speech Representations Still Struggle with African American Vernacular English

To train or not to train adversarially: A study of bias mitigation strategies for speaker recognition

Investigating the Sensitivity of Automatic Speech Recognition Systems to Phonetic Variation in L2 Englishes

Towards Measuring Fairness in Speech Recognition: Casual Conversations Dataset Transcriptions

Quantification of stylistic differences in human- and ASR-produced transcripts of African American English

Toward Automated Detection of Biased Social Signals from the Content of Clinical Conversations

The Balancing Act: Unmasking and Alleviating ASR Biases in Portuguese