🔍 Read the full analysis: What It Means For AI As Egyptian And Nigerian English Join Open ASR on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Hugging Face has added two private Liva AI evaluation splits to its Open ASR Leaderboard: 1.34 hours of Egyptian-accented English and 1.55 hours of Nigerian-accented English. The addition broadens the accents covered, but the recordings total just 2.89 hours and published details about speakers and collection are limited.
Hugging Face has added two private English speech evaluation splits from Liva AI to its Open ASR Leaderboard, covering Egyptian- and Nigerian-accented English, as detailed in the original analysis. The conversational recordings total 2.89 hours, adding two African English varieties to leaderboard comparisons used to assess automatic speech recognition (ASR) systems.
The new splits are named Liva AI Conversational EG and Liva AI Conversational NG. Hugging Face says the Egyptian set contains 1.34 hours of speech and the Nigerian set contains 1.55 hours. Both consist of spontaneous conversation; their transcriptions are punctuated and cased, and retain disfluencies such as hesitations and repetitions.
The recordings are private evaluation data, rather than ordinary public benchmark datasets. They are used to assess systems on the leaderboard but are not available for developers to download and tune against directly. Hugging Face says private splits are intended to make evaluations more trustworthy and limit the opportunity to optimize models against known test examples.
The leaderboard’s Private tab will show two comparisons: US versus non-US English, and Inner Circle versus Outer and Expanding Circles. The non-US group currently includes the UK, Australia, Canada, India, Nigeria and Egypt. Hugging Face says the groupings draw on linguist Braj Kachru’s Three Circles framework and credits Hume AI with the leaderboard idea.
Egyptian and Nigerian English join the Open ASR Leaderboard
Hugging Face added private Liva AI speech splits to broaden accent evaluation. The short recordings add coverage, while leaving questions about how representative the results are.
What was added
Both datasets contain spontaneous conversation with punctuated, cased transcripts that retain disfluencies.
Liva AI Conversational EG
A private evaluation split for Egyptian-accented English.
Liva AI Conversational NG
A private evaluation split for Nigerian-accented English.
New ways to compare accents
The leaderboard’s Private tab will show results across two groupings.
- Non-US includes the UK, Australia, Canada, India, Nigeria and Egypt.
- Inner Circle compared with Outer and Expanding Circles, based on Kachru’s framework.
- Hugging Face plans to separate the latter circles as more accents are added.
Why private evaluation matters
Keeping test examples private may make it harder to tune systems directly to known benchmark material.
Users can inspect recognition results for two additional English varieties.
Scores describe these specific short test sets, not all speakers in either country.
More Accents in ASR Scores
Speech recognition can perform differently across accents and speech varieties. A single overall score may conceal those differences, making accent-specific results useful to developers and people deciding whether a system is suitable for their needs. Adding Egyptian and Nigerian English gives leaderboard users a way to inspect performance on test material from two varieties that were absent from its previously listed accent groupings.
The change is a limited expansion, not a broad measure of recognition quality across either country. The two splits contain less than three hours of speech in total, and the announcement does not establish how representative the recordings are. Scores can show how systems perform on these particular test sets; they cannot, by themselves, establish how a system works for Egyptian or Nigerian English speakers generally.
Keeping evaluation data private can make it harder to tailor a model to exact test recordings, which may help preserve the value of a benchmark. But privacy alone does not show that a test set captures varied speakers, settings or conversational styles. The usefulness of the new scores will depend partly on information about how the data was assembled and how results are reported.
Popular right now
Bose QuietComfort Headphones (2nd Gen), Wireless Over-Ear Noise-Cancelling Headphones, Black
Bose QuietComfort True Wireless Earbuds with Active Noise Cancellation, Black
Shark Navigator Lift-Away Upright Vacuum Cleaner, White (NV356E)
As an affiliate, we earn on qualifying purchases.
Earlier Private Benchmark Additions
Hugging Face says the Liva AI splits follow earlier additions of 11 private splits from Appen and DataOceanAI, covering scripted and conversational speech in five accents. The leaderboard later added a private split for Indian-accented English in collaboration with Voice Arena. The Liva AI collaboration extends that work to Egyptian and Nigerian English.
The new comparison labels are based on Kachru’s Three Circles framework, which groups varieties of English as Inner Circle, Outer Circle or Expanding Circle. In the current leaderboard presentation, the Inner Circle comparison includes the US, UK, Australia and Canada; India, Nigeria and Egypt appear in the Outer and Expanding grouping. Hugging Face says it plans to separate the latter two circles as more accents are added.
“We’re excited to announce a new collaboration with Liva AI to add even more private data with two new accents: Egyptian and Nigerian.”
— Hugging Face
What the Small Samples Show
The announcement does not say when the recordings were collected, how many speakers contributed, or how participants were selected. It also provides no breakdown by region, age or other speaker characteristics. Without those details, readers cannot judge how well these short samples reflect the range of Egyptian and Nigerian English speech.
Hugging Face has not provided baseline scores or said how the new splits affect existing leaderboard rankings. The announcement also does not report an independent review of the datasets’ representativeness, or say whether the recordings or fuller documentation will later be made public. The scores should be understood as results on specific private test sets, not as a complete assessment of either national variety.
More Accents, Separate Circles
Hugging Face says it plans to keep adding accents and languages and to split the Outer and Expanding Circle comparisons when more accents are available. The announcement gives no date or threshold for that change. For now, users can find the newly described comparisons in the leaderboard’s Private tab.
Further detail on speaker composition, recording methods and per-accent results would help users interpret the new scores. The next additions and any published evaluation results will show how the leaderboard’s coverage develops, but no schedule or additional dataset release was specified.
Key Questions
Which English accents do the new splits cover?
They cover Egyptian-accented English and Nigerian-accented English, in two Liva AI conversational splits.
How much speech is in the new evaluation data?
The Egyptian split contains 1.34 hours and the Nigerian split 1.55 hours, for a combined 2.89 hours.
Can developers download the recordings?
Hugging Face describes them as private evaluation data. They are used for leaderboard assessment but are not available as ordinary public benchmark datasets.
Do the splits show how well ASR works for everyone in Egypt or Nigeria?
No. The announcement gives limited information about speakers and collection, and the recordings total less than three hours. Results describe performance on these specific test sets, not all speakers in either country.
What will happen to the accent comparisons?
Hugging Face says it plans to separate the Outer and Expanding Circles as more accents are added, but it has not announced a date or a threshold for doing so.
Primary source: Hugging Face · via ThorstenMeyerAI.com
