Open ASR Leaderboard adds Egyptian and Nigerian accents for better global testing
The Open ASR Leaderboard is significantly expanding its evaluation scope by adding private data splits for Egyptian and Nigerian English accents, marking a major step in global speech recognition testing.
This collaboration with Liva AI is part of a continuous effort to improve the representation of diverse accents and languages, ensuring that ASR (Automatic Speech Recognition) models are trustworthy and reliable worldwide. The use of private, real-world data minimizes 'benchmaxxing' (artificially optimizing for test sets) and provides a more accurate measure of model performance.
By incorporating natural, high-context behavior collected from real consumer products, the leaderboard can now provide a richer breakdown of accents, allowing comparisons across geographical groups like the Inner Circle (US, UK, Australia, Canada) versus the Outer & Expanding Circles (India, Nigeria, Egypt).
Why it matters
- —It increases the global fairness and accuracy of ASR model evaluations by including underrepresented accents.
- —The use of real-world, natural data (from consumer products) makes the benchmarks more trustworthy and less susceptible to gaming.
- —It allows researchers to analyze speech performance based on established linguistic models, such as Kachru’s Three Circles.
Key facts
- New private data splits for Egyptian and Nigerian English accents were added.
- The collaboration is with Liva AI, which collects multimodal data from real consumer products.
- The updates allow for a detailed comparison of accents using linguistic models (Inner vs. Outer/Expanding Circles).
- The goal is to improve overall representation and trust in ASR evaluation metrics.
The full text is in the original source. Here we provide a brief summary and key facts.