AraYoungVoices: 151.72 hours of Arabic child speech stay closed, and 46% of its fine-tuning gain needs prompts seen in training

Nearly half of AraYoungVoices' fine-tuning gain appears only on prompts the model has already seen

The paper tests every model on two sets recorded by the same 82 held-out speakers: USUP (unseen speaker, unseen prompt, 2,500 utterances, 4.78 h) and USSP (unseen speaker, seen prompt, 20,686 utterances, 41.25 h). Both report WER against the reading prompt (WER_P). Zero-shot, the two sets are about equally hard: averaged over the four models, prompt WER is 7.45 vs 7.31 for L1 kids, 36.77 vs 37.09 for L2 kids, 9.72 vs 11.96 for L1 teens and 29.31 vs 31.03 for L2 teens (USUP vs USSP, our averages of Table 6). After joint fine-tuning (JT) they split apart. Summed over the four age-nativity groups, the mean WER reduction is 64.82 points on USSP and 34.82 on USUP. The 30.0-point difference, 46% of the USSP gain, needs the prompt text to have been in training.

The split is uneven across groups, which is the part a data team should read twice. For L1 children and L1 teens, the native speakers reading grade-level text, fine-tuning on new prompts does almost nothing: +0.72 and -0.19 points of mean WER_P reduction on USUP, against +4.45 and +6.92 on USSP. For L2 children, the group with the highest error, most of the gain transfers: 19.28 points on USUP against 25.03 on USSP. Taking the best of the paper's three fine-tuning recipes per cell instead of JT changes the pooled share only from 46% to 43%.

Our reading, marked as a judgement: USSP measures how well a model recognises a new child reading a text it was trained on. That is a real use case (a fixed reading curriculum in a literacy app), and it is a different quantity from recognising a new child reading anything. The paper's abstract and conclusion do not distinguish the two when they say fine-tuning "significantly" improves recognition.

What QCRI published on 4 October 2026

arXiv:2610.05044v1, AraYoungVoices: A Diverse L1/L2 Corpus of Arabic Child and Adolescent Speech, by Shammur Absar Chowdhury, Zien Sheikh Ali, Houssam Eddine-Othman Lachemat and Hamdy Mubarak of the Qatar Computing Research Institute, submitted 4 October 2026 (cs.CL), funded by HBKU Flagship Research Grant HBKU-INT-VPR-FRG-03-09. It is a six-page paper with an anonymised release footnote, so it reads as a conference submission. No v2 existed on 6 October 2026.

The corpus has two halves: AraKids (ages 7 to 12) and AraTeens (13 to 18). L1 speakers come from Egyptian, Gulf, Levantine and North African backgrounds; L2 speakers from Asia, Europe, the Americas and Africa. Children read age-matched prompts on mobile devices through a collection platform: fully diacritised Grade 1 to 4 material for L1 kids, undiacritised Grade 9 to 12 material for L1 teens, separately chosen material for L2 readers. The paper states guardian consent was obtained, that metadata covers age, gender, nationality, mother tongue, years of learning Arabic and education level, and that "no personally identifiable information was retained".

Four models are benchmarked: Aura-STT (via Fanar), Cohere Transcribe Arabic (CohereLabs/cohere-transcribe-03-2026, 2B), Qwen2.5-Omni-Arabic (7B) and Whisper large-v3 (1.55B), each zero-shot and fine-tuned on AraKids only (AK), AraTeens only (AT) or both (JT). USUP has a manual verbatim transcript as well as the prompt, so it reports WER_V and WER_P; USSP reports WER_P only.

The paper's tables, our recomputation, and the release check

All paper values from arXiv:2610.05044v1. Sums, divisions, counts and release checks are BLOMEGA computations, 6 October 2026.

The inventory is clean. Every utterance and speaker count in Table 5 reconciles, the hour sums differ from the printed totals by 0.01 at most (rounding), and dividing Table 5 hours by utterances reproduces Table 2's average durations to two decimals. For a corpus paper, that is the check most often failed, and this one passes.

The prose about the test set does not match the table. Section 2 describes USUP as 2,000 AraKids utterances "using 5-7 L1 speakers per region" plus 500 AraTeens utterances "using two speakers per region". Table 5 puts 1,000 L2 kid utterances from 40 speakers inside USUP, and 8 L2 teen speakers alongside 8 L1 teens. The L1 part fits the prose (26 kid speakers over four regions is 6.5 per region; 8 teens is two per region), so the sentence most likely omits the L2 half. It matters because the L2 groups carry the largest WER and the largest fine-tuning gains.

Two small data-entry signs are worth flagging without over-reading. Qwen's AK score for L1 teens is 23.28 in both the USUP WER_V column and the USSP column, and Qwen's L2-kids USSP score is 34.21 under both zero-shot and AT. Either is possible by coincidence; both are the kind of copy error a v2 should confirm or fix.

The abstract says age-specific fine-tuning "improves the matched age group". On the harder USUP set, measured against the verbatim transcript, AK and AT beat zero-shot in 11 of 16 matched cells. Every miss is an L1 group: Qwen's L1 teens go from 14.44 to 18.10 WER_V after AT, Aura's from 11.64 to 12.75, Cohere's L1 kids from 6.60 to 6.65. Only Whisper improves on all four matched groups. The paper's Section 4 does say the matched-group gain is "on USSP"; the abstract drops that qualifier.

How the USSP split lets the model meet the test text in training

Read-speech corpora reuse prompts: many children read the same story. AraYoungVoices keeps speakers disjoint between train and test but lets prompts cross the boundary in USSP. A fine-tuned decoder then learns the prompt word sequences as a language-model prior, and on a seen prompt it can output the text even where the acoustics are ambiguous. Because USSP is scored only against the prompt, that prior is rewarded twice: once for recognising the words and once for "correcting" the child's misreadings back to the page.

The effect is starkest for L2 teens, where USSP prompt WER after joint fine-tuning falls to 1.40 to 3.25 across the four models, while the same models on unseen prompts stay at 11.43 to 17.83. Qwen2.5-Omni-Arabic shows a 12.7x ratio (17.83 vs 1.40). These are the same 8 speakers reading different texts.

Two caveats keep this from being a clean measurement. USUP is small: 250 utterances per teen group from 8 speakers, about 0.4 h each, so a single fluent or struggling teenager moves the cell. And the USUP prompts were drawn separately from the USSP ones, so they need not be the same difficulty. The zero-shot columns are the control for that: across all four groups, mean zero-shot WER_P differs by at most 2.24 points between the two sets (L1 teens), and USSP is the slightly harder set in three of four. Prompt difficulty therefore does not explain mean post-fine-tuning gaps of 3.86 to 11.69 points (JT, USUP minus USSP, our averages).

The prompt-versus-verbatim result points the same way from a different angle. On every one of the 64 USUP cells, WER against the prompt is lower than WER against what the child actually said, by 2.74 points on average for L1 groups and 7.30 for L2 (our means over Table 6). Models are rewriting misreadings toward the page, and a USSP-only score cannot see that, because USSP has no verbatim transcript.

What this means for teams collecting or buying child and L2 speech

• Split on prompts as well as speakers. If your collection reuses reading scripts, hold out whole prompts for the test set and report that number first. AraYoungVoices shows the size of the error you avoid: about 46% of the headline gain, and nearly all of it for native readers.
• Pay for verbatim transcripts on at least the test set. Prompt-scored WER hides the model's tendency to normalise. The paper's own USUP verbatim layer (substitutions, insertions, deletions, repetitions, fillers, dialectal letters) is what made that visible; the 20,686-utterance USSP set has none.
• Hold out L2 speakers in development too. The AraYoungVoices dev set is 5 L1 speakers. Our judgement: checkpoints selected on L1 data are tuned for the group fine-tuning helps least on new text.
• Write the release scope into the guardian consent form before recording. The authors chose not to release audio and to release models "subject to the applicable consent and institutional requirements". That conditional is the gap: a model fine-tuned on a minor's voice is a derivative of that voice, and the paper does not state whether guardians agreed to model release, whether children gave their own assent, which ethics board approved it, or what licence will apply. None of those four is reported.
• "No PII retained" does not make child audio shareable. Voice is biometric, and the retained metadata (age, nationality, mother tongue, years learning Arabic) can narrow a small cohort. The QCRI team's decision to keep the audio closed is consistent with that; buyers of comparable data should expect the same and price on-premise evaluation access instead of download.

For comparison on Arabic ASR outside the classroom, see our note on the best recogniser getting 51% of words wrong on Moroccan YouTube speech; on sourcing speech with consent that survives redistribution, our list of consented, licence-clear training data providers.

Check it yourself

The release state (run 6 October 2026, all returned empty lists or zero hits):

for q in AraYoungVoices arayoung AraKids AraTeens; do
  curl -s "https://huggingface.co/api/models?search=$q" ; echo
  curl -s "https://huggingface.co/api/datasets?search=$q" ; echo
done
curl -s "https://api.github.com/search/repositories?q=AraYoungVoices" | grep total_count
curl -s "https://huggingface.co/api/models?author=QCRI&sort=lastModified&direction=-1&limit=15" \
  | python3 -c "import json,sys;[print(m['id'],m['lastModified']) for m in json.load(sys.stdin)]"
curl -s -o /dev/null -w '%{http_code}\n' https://arxiv.org/abs/2610.05044v2   # 404 on 6 Oct 2026

The paper text (no poppler needed):

curl -sL https://arxiv.org/pdf/2610.05044 -o 2610.05044.pdf
python3 -c "import pypdf;r=pypdf.PdfReader('2610.05044.pdf');print(r.pages[2].extract_text(extraction_mode='layout'))"
# page 3: Table 5 (splits), page 2: Data Splits prose, page 4: Table 6, page 5: Limitation and Distribution

The leakage decomposition, with Table 6 WER_P typed in (USUP second value of each pair, USSP column):

python3 - <<'PY'
from statistics import mean
# per model: (ZS USUP_P, JT USUP_P, ZS USSP_P, JT USSP_P) for Kids-L1, Kids-L2, Teens-L1, Teens-L2
T={'Aura':   [(8.25,6.32,7.24,2.36),(33.90,17.53,34.03,12.16),(8.38,10.19,9.89,3.55),(28.92,11.55,27.71,3.25)],
   'Cohere': [(4.60,3.98,4.37,2.44),(37.70,12.92,39.97,9.87),(6.69,6.69,7.65,4.64),(32.62,11.43,28.89,2.90)],
   'Qwen':   [(7.17,8.83,7.61,2.38),(32.49,18.99,34.21,11.63),(11.58,12.81,15.07,5.22),(23.43,17.83,26.80,1.40)],
   'Whisper':[(9.76,7.78,10.02,4.28),(43.00,20.55,40.16,14.58),(12.22,9.95,15.21,6.73),(32.29,16.37,40.71,2.86)]}
gu=gs=0
for i,g in enumerate(['Kids-L1','Kids-L2','Teens-L1','Teens-L2']):
    u=mean(T[m][i][0]-T[m][i][1] for m in T); s=mean(T[m][i][2]-T[m][i][3] for m in T)
    gu+=u; gs+=s; print(g,'USUP gain',round(u,2),'USSP gain',round(s,2))
print('pooled',round(gu,2),round(gs,2),'seen-only share',round(100*(gs-gu)/gs),'%')
PY

Expected: 0.72 / 4.45, 19.28 / 25.03, -0.19 / 6.92, 15.02 / 28.43; pooled 34.82 / 64.82; 46%. Table 5 sums: add each column over Train, Dev, USUP and USSP for utterances and hours, and over Train, Dev and USUP only for speakers (the paper says USSP "uses the same speaker pool as USUP").

What would prove this wrong

Our claim is that the USSP gains in AraYoungVoices overstate what fine-tuning buys on new reading material, by roughly 30 WER points summed over the four groups, and that for L1 readers the unseen-prompt gain is near zero.

It would be wrong if the authors' released models, or a v2 of the paper with a larger USUP set (say 1,000 or more utterances per group), show joint fine-tuning reducing L1 kids' and L1 teens' unseen-prompt WER_P by 3 points or more on average across models. We predict that by 30 June 2027 no such result is published, that the AraYoungVoices raw audio remains unavailable for download, and that if the fine-tuned models are released, an independent test on unseen-prompt Arabic child read speech will put their L2-teen WER_P at three times or more their USSP figures (above about 4 to 10%, against 1.40 to 3.25 in the paper).

Sources

1. Shammur Absar Chowdhury, Zien Sheikh Ali, Houssam Eddine-Othman Lachemat, Hamdy Mubarak (Qatar Computing Research Institute). AraYoungVoices: A Diverse L1/L2 Corpus of Arabic Child and Adolescent Speech. arXiv:2610.05044v1, submitted 4 October 2026. Sections 1 to 5, Tables 1 to 7, Limitation and Distribution.
2. QCRI organisation on Hugging Face, model and dataset lists checked via the Hub API on 6 October 2026: no AraYoungVoices, AraKids or AraTeens artefact.
3. CohereLabs/cohere-transcribe-03-2026, one of the four benchmarked models; Apache-2.0, created 24 March 2026 (Hub API, checked 6 October 2026).
4. Mouhand Alkadri, Dania Desouki, Khloud Al Jallad. Arabic Little STT: Arabic children speech recognition dataset. arXiv:2510.23319, 2025. The prior Arabic child resource the paper compares against (355 utterances).
5. BLOMEGA computations, 6 October 2026: Table 5 reconciliation, Table 2 and Table 3 cross-checks, Table 6 counts and the seen-prompt decomposition, GitHub and Hugging Face release searches.

FAQ

Can I download the AraYoungVoices audio or fine-tuned models?

Not as of 6 October 2026. The paper (arXiv:2610.05044v1, Limitation and Distribution) says the raw audio and full speaker-level metadata will not be publicly released because the speakers are minors; it promises de-identified aggregate statistics, evaluation protocols and the fine-tuned ASR models, subject to consent and institutional requirements. The model link is 'removed for anonymity', and Hugging Face and GitHub searches for AraYoungVoices, AraKids and AraTeens returned nothing on 6 October 2026. No licence is stated.

How much of the AraYoungVoices fine-tuning gain is prompt memorisation?

By our decomposition of Table 6, about 46%. Averaged over the four models, joint fine-tuning cuts prompt WER by 64.82 points summed over the four age-nativity groups when test speakers read prompts seen in training (USSP), and by 34.82 points when the prompts are new (USUP). Zero-shot WER is nearly the same on both test sets, so the extra 30.0 points appear only for seen prompts. Taking the best of the three fine-tuning recipes per cell instead gives 43%.

Do the AraYoungVoices corpus totals add up?

Yes for counts: utterances, speakers and hours in Table 5 reconcile across Train, Dev, USUP and USSP to 86,471 utterances, 286 speakers (146 L1, 140 L2) and 151.72 hours, with hour sums off by at most 0.01 from rounding, and the Table 2 average durations follow exactly. The prose description of the USUP test set does not: it says 2,000 AraKids utterances from L1 speakers, while Table 5 has 1,000 L1 and 1,000 L2 kid utterances from 26 and 40 speakers. The Aura-STT size is 347M in the text and 342M in Table 6.