130 hours of Mapuzugun bought 0.82 BLEU. 30 hours of Central Kurdish bought 21.09
Across the five low-resource pairs in the IWSLT 2026 speech translation task that have published results, the Spearman rank correlation between hours of provided speech and best BLEU is exactly 0.00. Mapuzugun-Spanish came with more than 130 hours of transcribed and translated speech and produced 0.82 BLEU. Central Kurdish-English came with 30 hours and produced 21.09. Irish-English came with 13 real hours plus 196 synthetic ones and produced 2.4. If you are budgeting a data collection for a new locale, hours is the wrong line item to argue about.
Ten language pairs, one campaign, every number published
July 2026, San Diego. The IWSLT low-resource speech translation track ran 10 typologically diverse pairs plus a data track inviting new open-sourced corpora. This year's stated focus was explicitly multilingual systems handling as many languages as possible. Results are in Speech Translation and Metrics in 2026, Track II, Tables 2 to 7, alongside the African and Celtic speech-to-speech track that shares its Hausa, Igbo and Yoruba data.
What makes the track worth reading is that the organisers publish the corpus behind each pair, in hours, with its provenance, next to the score every team got on it. That pairing is rare. Most dataset announcements give you hours and no downstream number; most benchmark papers give you a number and no corpus description.
The corpus, the speakers and the score, in one table
| Pair | Speakers | Speech provided | Domain | Best BLEU | chrF++ | Best system | Source |
|---|---|---|---|---|---|---|---|
| Quechua to Spanish | >8 million | ~50 h transcribed + 8 h synthetic post-edited + 15 h Quechua Collao | mixed, multi-variant | 27.2 | 51.4 | QUESPA contrastive 2 (SpeechT5 end-to-end) | Table 4 |
| Central Kurdish to English | ~8 million | 30 h COMMUTE-Kurdish | spontaneous Kurdish media | 21.09 | 49.48 | LIUM (pseudo-labelling) | Table 7 |
| Bhojpuri to Hindi | 50.58 million | ~24 h | news (News On Air) | 14.7 | 43.0 | ADAPT-MTU primary (Whisper large-v3 + NLLB-200) | Table 3 |
| Irish to English | ~170,000 L1 | ~13 h real + 196 h synthetic | news, Common Voice, Living-Audio-Dataset | 2.4 | 16.0 | MTU primary | Table 5 |
| Mapuzugun to Spanish | 100,000 to 200,000 | >130 h | language isolate corpus | 0.82 | 14.31 | KK contrastive 2 | Table 6 |
| Bemba to English | >10 million | 180 h + 28 h transcribed mono + 60 h untranscribed mono | image-grounded dialogues | not reported | not reported | no results table published | Track II section 2 |
| Catalan to English | 4.1 million L1 | not stated | not stated | not reported | not reported | CATENG submitted, no table | Track II sections 2, 3 |
| Hausa to English | >100 million | shared with African/Celtic track | newly collected | 18.6 spBLEU | 41.9 | SeamlessM4T mono fine-tuned (organiser baseline) | Table 2 |
| Igbo to English | 30 to 45 million | shared with African/Celtic track | newly collected | 17.6 spBLEU | 39.2 | SeamlessM4T mono fine-tuned (organiser baseline) | Table 2 |
| Yoruba to English | ~50 million | shared with African/Celtic track | newly collected | 21.1 spBLEU | 43.5 | SeamlessM4T mono fine-tuned (organiser baseline) | Table 2 |
Note what does not predict the score. Speaker population does not: Bhojpuri has 50.58 million speakers and scores 14.7, Mapuzugun has at most 200,000 and scores 0.82, but Irish has fewer speakers still and also fails, while Central Kurdish has a comparable population to Quechua and both do well. Hours do not either, and that one is measurable.
Four things that did predict the score
The Central Kurdish column is the sharpest illustration. Two teams, one 30-hour corpus. LIUM scored 21.09 BLEU and 49.48 chrF++ with a pseudo-labelling pipeline that produced silver translations for untranscribed audio through an automated ASR and MT chain, and reported 6.98 CER and 19.76 WER on the recognition side. SLC scored 0.16 BLEU on the same data. A factor of 131 between two systems on the same corpus says the corpus was not the limiting factor for either of them.
No frontier model beat a fine-tuned NLLB-200 on Quechua
The QUESPA team, in its fourth consecutive year on this pair, ran a separate text machine translation case study: GPT-5, Gemini 3, Claude, DeepSeek-V3 and Qwen, prompted in Spanish with guided prompts. The best prompt-based result was 10.8 BLEU, from Gemini 3 Flash. The fine-tuned NLLB-200 baseline from the previous year sits at 19.5 BLEU and 23.5 chrF. None of the frontier models passed it.
The team's stated causes: hallucinations, dialectal confusion between Quechua variants, and a tendency of models to prioritise high-resource language signals over low-resource Quechua input. That third one is the structural problem. A model trained overwhelmingly on Spanish will read Quechua input through Spanish priors, and prompt engineering in Spanish reinforces exactly that.
The same pattern shows on the speech side of the African track, where the organisers ran three baselines rather than accepting submissions as the reference. A monolingually fine-tuned SeamlessM4T beat a cascade of Omnilingual ASR (OmniASR LLM 1B) and NLLB-200 on all three languages, and also beat an end-to-end system built on a frozen Gemma-4-E2B language model on all three, though that system took second place on Yoruba ahead of the cascade.
The data track wrote down what a licensable speech corpus has to carry
Buried in Track II is the clearest public statement of dataset hygiene any shared task has published, and it reads like a procurement checklist. The requirements for a contributed corpus:
- Human verification is mandatory. "Raw, unverified machine translated outputs are not allowed." Post-editing of automatic output is allowed; the submitted data must be 100% verified by humans if not created by them.
- The MT you used must permit the downstream use. The organisers name DeepL, Google Translate and ChatGPT as examples whose terms of service disallow reusing outputs to train other translation models. That is a licence problem, not a quality problem, and it disqualifies the fastest path to volume.
- Translation by qualified native speakers, verified by at least one more.
- Identifiers, not names. An ISO 639-3 individual language tag, a Glottocode, and an ISO 15924 script code on the dataset card.
- CC BY-SA 4.0 or similarly permissive, research use at minimum.
One corpus met all of it and published its own baseline. FLEURS-Badini extends FLEURS to the Badini variant of Northern Kurdish: 2,000 English FLORES sentences translated by English and Translation students at the University of Duhok, recorded by native speakers through an online platform, with both translations and recordings manually reviewed by faculty. The result is 5,224 utterances, 15 hours 40 minutes, 45 speakers, split 2,022 utterances (5h47m) train, 1,165 (3h36m) dev, 2,037 (6h17m) test. A fine-tuned Whisper scores 5.24 BLEU and 29.57 chrF++ on it.
That 5.24 is the honest number for a brand-new variant with 15 hours behind it, and it is worth holding next to the 21.09 that 30 curated hours of Central Kurdish produced. Same language family, roughly double the data, four times the score.
How to spend a low-resource data budget in 2027
Buy annotation depth before duration. The corpus that produced 21.09 BLEU was 30 hours, manually segmented, transcribed and translated, across politics, culture, economy, sports, art and science. The corpus that produced 0.82 was over 130 hours. If your vendor quotes per recorded hour with transcription and translation as options, the options are the product.
Do not buy synthetic speech as a substitute for recorded speech. 196 synthesised hours moved Irish to 2.4 BLEU. LIUM tested the same idea deliberately and reported it not fruitful for spontaneous speech, while pseudo-labelling real untranscribed audio worked. If you have untranscribed audio in the language, that is worth more than an equivalent budget of text-to-speech.
Check whether your target has a high-resource neighbour before you forecast quality. Bhojpuri gets Hindi. Quechua and Mapuzugun get Spanish on the target side but nothing on the source side, and Mapuzugun, as an isolate, gets nothing at all. Judgement, not measurement: an isolate or a lone branch should be budgeted at two to three times the data of a language with a well-covered sibling, for the same expected quality.
Split your dialects at collection time. The Quechua data is labelled with the que macro-code covering both Chanka (quy) and Collao (quz), and dialectal confusion is named as a failure cause. Recording metadata that distinguishes variants costs nothing at capture and cannot be recovered afterwards.
Write the data track's checklist into your supplier contract. ISO 639-3 plus Glottocode plus ISO 15924, named-speaker consent, a second native verifier, an explicit licence, and a warranty that no output of a model whose terms forbid training reuse is present in the delivery. Every one of those is cheap to require at the start and impossible to retrofit.
Do not assume a frontier model closes the gap. Five of them, prompted properly, lost to a fine-tuned NLLB-200 on Quechua text by 8.7 BLEU. For a genuinely low-resource locale, fine-tuning a specialised translation model on data you commissioned is still the stronger play in 2026.
Check it yourself
# 1. the track (corpora, hours, provenance) and the results tables
curl -sL -o iwslt2026.pdf https://aclanthology.org/2026.iwslt-1.39.pdf
python3 -c "
import pypdf
t='\n'.join(p.extract_text() or '' for p in pypdf.PdfReader('iwslt2026.pdf').pages)
i=t.find('Track II Low-resource'); print(t[i:i+22000])"
# 2. the correlation quoted in the first line
python3 - <<'PY'
pairs = [("Irish",13,2.4), ("Bhojpuri",24,14.7), ("C. Kurdish",30,21.09),
("Quechua",50,27.2), ("Mapuzugun",130,0.82)]
def rank(xs):
order = sorted(range(len(xs)), key=lambda i: xs[i]); r=[0]*len(xs)
for pos,i in enumerate(order): r[i]=pos+1
return r
h = [p[1] for p in pairs]; b = [p[2] for p in pairs]
rh, rb = rank(h), rank(b); n = len(pairs)
d2 = sum((x-y)**2 for x,y in zip(rh, rb))
print("hour ranks", rh, " BLEU ranks", rb, " sum d^2", d2)
print("Spearman rho =", 1 - 6*d2/(n*(n*n-1)))
PY
# hour ranks [1, 2, 3, 4, 5] BLEU ranks [2, 3, 4, 5, 1] sum d^2 20
# Spearman rho = 0.0
# 3. the corpora themselves
# COMMUTE-Kurdish https://lium.univ-lemans.fr/en/corpus-commute-kurdish/
# Bhojpuri-Hindi https://github.com/shashwatup9k/iwslt2026_bho-hi
# Irish-English https://github.com/shashwatup9k/iwslt2026_ga-eng
# https://hf.co/collections/ymoslem/irish-english-speech-translation-datasets-665dd9e8fbaa279db3474ca0
# Bemba https://github.com/csikasote (Zambezi Voice, BembaSpeech)
# YODAS https://hf.co/datasets/espnet/yodas
# 4. the data-track requirements, verbatim
python3 -c "
import pypdf
t='\n'.join(p.extract_text() or '' for p in pypdf.PdfReader('iwslt2026.pdf').pages)
i=t.find('Data Submission Requirements'); print(t[i:i+1800])"To price a collection against this evidence, count hours that are segmented, transcribed and translated separately from hours that are only recorded, and separately again from hours that are synthetic. The 2026 results do not let you add them together.
What would prove this wrong
The claim under test is that raw recorded duration is not what limits low-resource speech translation quality at these scales, and that annotation depth and transfer from a related language are. It is wrong if, by 31 December 2027, the IWSLT low-resource track publishes results across at least five pairs with a Spearman rank correlation of 0.6 or higher between provided speech hours and best BLEU, without a change in the mix of corpora. The 2026 figure across five pairs is 0.00. One more edition on the same pairs will settle it, because the corpora mostly carry over.
It is also wrong on the Mapuzugun reading specifically if a 2027 submission reaches 10 BLEU or more on Mapuzugun-Spanish using only the existing 130-hour corpus and no newly collected Mapuzugun data. That would show the 0.82 was a recipe failure rather than a transfer failure. The current best is 0.82 from three submissions, all of which the organisers describe as struggling to produce meaningful outputs.
A third prediction, marked as judgement: a frontier general-purpose model will still trail a fine-tuned specialised translation model on Quechua-Spanish text at IWSLT 2027. If a prompted or few-shot frontier model clears 19.5 BLEU there without fine-tuning on commissioned Quechua data, this reading was wrong.
FAQ
How many hours of speech do you need for a low-resource language?
Hours are not the binding constraint at this scale. Across five IWSLT 2026 pairs the rank correlation between provided hours and best BLEU is 0.00. Thirty curated hours of Central Kurdish produced 21.09 BLEU; over 130 hours of Mapuzugun produced 0.82.
Can a frontier LLM beat a specialised model on a low-resource language?
Not for Quechua in 2026. GPT-5, Gemini 3, Claude, DeepSeek-V3 and Qwen were benchmarked with guided Spanish prompts; the best was 10.8 BLEU from Gemini 3 Flash, against 19.5 BLEU and 23.5 chrF for a fine-tuned NLLB-200 baseline.
Does synthetic speech work as training data?
The 2026 evidence says no. Irish got 196 synthetic hours on top of 13 real ones and reached 2.4 BLEU. LIUM compared synthesis with pseudo-labelling on Kurdish and found synthesis not fruitful for spontaneous speech, while pseudo-labelling matched cascades at lower latency.
What does a contributed dataset have to carry?
100% human verification with no raw MT output, an MT licence that permits training reuse (which the organisers note DeepL, Google Translate and ChatGPT do not grant), translation by qualified native speakers with a second verifier, ISO 639-3 plus Glottocode plus ISO 15924 identifiers on the dataset card, and CC BY-SA 4.0 or a similarly permissive licence.
Sources
- IWSLT 2026 organisers (60 authors), Speech Translation and Metrics in 2026: Findings of the IWSLT Campaign, Proceedings of the 23rd International Conference on Spoken Language Translation, San Diego, July 2026. Track II (Low-resource SLT): per-pair corpus descriptions and speaker counts, submissions, Tables 2 to 7, data track requirements and FLEURS-Badini. PDF.
- QUESPA (Ortega et al., 2026), Quechua-Spanish submission described in Track II: the frontier-LLM case study (GPT-5, Gemini 3, Claude, DeepSeek-V3, Qwen), SIDON audio enhancement, and the 27.2 BLEU SpeechT5 system.
- LIUM (Mohammadamini and Tahon, 2026), Central Kurdish-English submission: pseudo-labelling against speech synthesis, 21.09 BLEU, 6.98 CER and 19.76 WER.
- Mohammadamini et al., 2026, FLEURS-Badini: 5,224 utterances, 15h40m, 45 speakers, University of Duhok, fine-tuned Whisper at 5.24 BLEU and 29.57 chrF++. Described in Track II, Data Track Results.
- Corpora referenced by the organisers: COMMUTE-Kurdish, Bhojpuri-Hindi, Irish-English, YODAS, Zambezi Voice and BembaSpeech (Sikasote et al.), and Common Voice.
- Rank correlation in this note computed by BLOMEGA on 16 September 2026 from the Track II tables. Code in "Check it yourself".
Related BLOMEGA guides: Meta's 8B Omnilingual MT by resource tier · A 30-trillion-token corpus buys Basque a 160-million-parameter model · Consented AI training data providers.