The "32% to 79%" code-switching phoneme loss is one recognizer on two datasets, and the code is not out

Code-switched input costs every tested model 32% to 79% of its unshared-phoneme recall, and the range is set by the dataset more than the model

The paper's metric is unshared phoneme recall: of the phonemes that occur in one language of a pair and never in the other (English /ɹ/ next to French, Spanish /ɾ/ next to English), how many does the model keep? It scores each language span alone (SEP) and the whole utterance (TOG), takes the worse of the two spans, and reports the drop Δ. Table 5 gives Δ and Δ as a fraction of the SEP level for 4 model heads on 2 datasets. We back-solved the SEP and TOG recall from those two columns and checked them against Table 1 (SwitchLingua TOG recall) and Table 6 (spliced FLEURS SEP and TOG recall): all 11 overlapping values match within 0.001, and only two differ at all (PhoneticXeus 0.581 vs 0.582 and 0.395 vs 0.396, rounding). The tables are internally consistent.

Sorted, the eight relative losses are 32.0%, 42.4%, 44.3%, 71.8%, 73.0%, 75.4%, 76.9% and 78.6%. The minimum and maximum are both PhoneticXeus. On SwitchLingua, 288 recordings of bilingual speakers reading sentences with one switch, the four heads sit at 42% to 79%. On spliced FLEURS, two monolingual clips joined end to end, they sit at 32% to 73%. POWSM's two heads barely move between datasets (75% and 77% against 72% and 73%); PhoneticXeus drops from 79% to 32%. Our reading, marked as judgement: whatever the spliced condition lacks (natural coarticulation, a single speaker across the switch, a usually short English second span), it matters to PhoneticXeus far more than to POWSM.

The absolute levels are worth reading too. Even alone, the worse span keeps only 54% to 71% of its unshared phonemes (r_SEP 0.538 to 0.712). Together, PhoneticXeus on SwitchLingua keeps 13.8% of them, POWSM's attention decoder 12.4%. For a pronunciation-sensitive pipeline, that is the number that matters: roughly 1 in 8 English-only phonemes survive in the worse span.

What Yanuka, Giryes and Alper published on 8 October 2026

arXiv:2610.11275v1, Phonological Interference in Multilingual Speech Models, by Moran Yanuka and Raja Giryes (Tel Aviv University) and Morris Alper (University of Miami), was submitted on 8 October 2026 (arXiv API published 2026-10-08T05:38:01Z) and is marked "under review". It tests POWSM (E-Branchformer encoder, attention decoder plus a CTC head), PhoneticXeus (XEUS encoder, CTC) and CosyVoice3 fine-tuned with LoRA to take IPA input across 16 languages, with no language tags.

Three claims sit on top of the measurement. First, unseen languages: across 43 DoReCo languages, recordings the probe assigns more confidently to a training language lose 0.053 (POWSM) and 0.060 (PhoneticXeus) more unshared recall, while shared recall rises 0.015 and 0.029. Second, cause: a 15-dimensional "language subspace", under 2% of hidden dimensions, built from 16 language centroids; overwriting it toward another language on monolingual speech removes 0.777 (POWSM), 0.728 (PhoneticXeus) and 0.346 (TTS) of unshared recall, while a random subspace of equal rank removes 0.025 to 0.062 (Table 9). Third, a fix: windowed language estimation (WLE) replaces only those 15 coordinates with values computed from a 2 s window, and removes "34% to 69%" of interference. The same subspace edit moves Whisper large-v3's language token by +0.201 toward the target (Table 11).

Every headline figure in the abstract recomputes from the tables, with two that need a footnote

Paper values from arXiv:2610.11275v1. Back-solves, ratio interval, WLE shares, Table 4 counts and release checks are BLOMEGA computations, 9 October 2026.

The first footnote is the 8.2× ratio. It divides PhoneticXeus's unshared exact-match drop on spliced FLEURS (0.181) by a shared drop of 0.022 whose 95% interval is 0.009 to 0.036. Holding the numerator fixed, the ratio could be anywhere from 5.0× to 20.1×. The direction is solid; the precise multiple at the top of the range is not. It is also the same cell that gives the 32% minimum, so the cell with the smallest relative loss carries the largest selectivity ratio, because its shared phonemes barely move.

The second is the WLE range. "34% to 69%" reads as a spread across models, but both ends are the TTS model under two window variants. On SwitchLingua the recognizers sit inside it (PhoneticXeus 57.8%, POWSM 35.7%). For POWSM a plain 2 s sliding window removes 65.8% against WLE's 35.7%; the paper's case for WLE there is the monolingual cost, where the window adds 0.96 to 2.14 pp PER on seen benchmarks and WLE adds at most 0.10 pp (Table 2). On spliced FLEURS, which the abstract does not quote, WLE removes 74.7% (PhoneticXeus) and 49.0% (POWSM).

How a whole-utterance language guess overwrites local phonemes

The paper's account is that each model forms one language estimate from the whole input and imposes that language's inventory. Its probe shows the estimate shifting: on SwitchLingua the score for a span's own language drops by 0.31 (POWSM), 0.29 (PhoneticXeus) and 0.53 (TTS) when the other language is present (Figure 5a, Appendix F). The substitutions are not near misses. In PanPhon feature distance the added replacements are 3.0 to 4.3 times farther than the other language's closest phoneme; English /ɹ/ next to French goes to /ʁ/ (distance 6.9) rather than /j/ (1.6) (Table 8).

We asked whether the size of the inventory gap could explain the per-pair picture. The paper does not report interference per language pair, so that test cannot be run from its tables. What we could compute is the gap itself, from Table 4, which lists every unshared phoneme with its count in the reference spans. In 8 of 11 English pairs, English has more unshared phoneme types than its partner. English-Spanish on spliced FLEURS has 16 English-only types (797 tokens: /ɪ ə ɹ z æ ð ʌ ʊ/ and more) against 6 Spanish-only (176 tokens: /ɾ r x ʝ ɲ ʎ/). English-Italian is 10 against 1, Italian keeping only trilled /r/. The exceptions are English-German on SwitchLingua (7 vs 8) and English-Hindi (9 vs 10, 11 vs 11), where Hindi aspirates and retroflexes balance English's vowels.

These sets are small compared with reference inventories, and that is a choice worth knowing before you copy the method. Against PHOIBLE, which holds 3 to 9 inventories per language for these 7 languages, English-Spanish has 25 to 37 English-only and 11 to 35 Spanish-only segments depending on which pair of inventories you take. Stripping diacritics (dental /t̪/ to /t/, aspiration, length) brings that to 16 to 29 and 4 to 30. The paper's eSpeak-based, frequency-thresholded lists (16 and 6) sit at or near the bottom of both ranges. Hindi shows the widest spread: PHOIBLE inventories run from 55 to 94 segments. Our judgement: a team that builds its own unshared lists from a different G2P or inventory will get a different denominator, and the numbers will not be comparable to this paper's.

What this means for teams dubbing or transcribing Hinglish, Taglish and Spanglish

• The worst damage lands on the short English span. In SwitchLingua "the second span is usually shorter and in English" (Section 2.2), which is the brand-name and tech-term pattern of Hinglish and Spanglish scripts. English also holds most of the unshared phonemes in 8 of 11 pairs. Expect /ɹ/, /ð/, /ɪ/ and /æ/ in English inserts to drift toward the matrix language first (Table 3 lists ɹ→ʁ, ð→d, ɪ→i, z→s, æ→a).
• Separate spans beat clever chunking. Processing each span alone is the paper's zero-interference baseline. Chunking a TTS sentence span by span while keeping prior context, without the edit, gave Δ 0.329 against 0.302 unchunked (Table 12). If your engine can synthesise each language segment independently and you can stitch prosody, that is the safe default today.
• Pass the language per segment wherever a model accepts it. POWSM's attention decoder with each span's own tag cut Δ by 0.312 on SwitchLingua and 0.197 on spliced FLEURS (Table 18). Whisper large-v3 picks one language per 30 s window; on spliced FLEURS it wrote nothing for the other-language span in 218 of 286 clips, and its WER went from 0.152 to 0.540 (Table 10). Segment-level language ID in front of Whisper is cheap insurance; see our note on routing Whisper by dialect.
• Tagalog is not in the test. The 16 probe and TTS languages exclude Tagalog, and the code-switched pairs cover English with German, Spanish, French, Italian, Hindi and Korean only. Taglish and other pairs are extrapolation until someone measures them.
• Add unshared-phoneme recall to pronunciation QA. A word-level metric hides this: Whisper's span in its picked language changes WER by −0.010 on SwitchLingua while the other span gains +0.223. A per-pair list of unshared phonemes and a recall count on the inserted span catches the drift before a native reviewer does.
• Check licences before building on the stack. PhoneticXeus, which is both a tested recognizer and the scorer for every TTS number, is CC BY-NC-SA 4.0 on Hugging Face. SwitchLingua audio is CC BY-NC 4.0 with an access gate. POWSM (CC BY 4.0) and Fun-CosyVoice3-0.5B-2512 (Apache 2.0) are commercially usable. The same non-commercial pattern showed up in KyaraBench's dubbing voice benchmark.

We also ran a small probe to see whether an older recognizer outside the paper behaves the same way. Allosaurus 1.0.2 (a 2020 universal phone recognizer, BiLSTM, CPU) transcribed macOS synthetic voices: 2 English sentences paired with 2 sentences each in Spanish, French, German and Italian, both orders, plus same-language splices as a control. Counting each span's language-only phones, English spans kept 168 alone and 137 next to another language (−18.5%); same-language splices went from 37 to 38 (+2.7%). The phone string of the same audio span changed by a mean normalised edit distance of 0.581 across languages against 0.380 within one language. That is 32 cross-language spans of synthetic speech: direction consistent with the paper, magnitude not to be quoted. German spans went the other way (6 to 11) and English next to German barely moved (42 to 43), which fits English /ɪ/ and /ʊ/ being shared with German.

How to rerun the recount, the inventory count and the release check

The headline recount uses only values typed from Tables 1, 5, 6 and 17. The script and its output are in our work directory as headline.py and headline.out:

curl -sL https://arxiv.org/pdf/2610.11275 -o p.pdf
python3 -c "import pypdf;print('\n'.join(p.extract_text() for p in pypdf.PdfReader('p.pdf').pages))" > p.txt
python3 -I headline.py      # prints 32.0% (Spliced FLEURS PhoneticXeus) and 78.6% (SwitchLingua PhoneticXeus)
# by hand: 0.186 / 0.320 = 0.581 rSEP, 0.581 - 0.186 = 0.395 rTOG (Table 6: 0.582, 0.396)
#          0.507 / 0.786 = 0.645 rSEP, 0.645 - 0.507 = 0.138 rTOG (Table 1: 0.138)

The inventory asymmetry counts the subscripts in Table 4 and compares them with PHOIBLE:

curl -sL -o phoible.csv https://raw.githubusercontent.com/phoible/dev/master/data/phoible.csv
python3 -I inventory.py phoible.csv   # Table 4 types/tokens per side, PHOIBLE min-max per pair, diacritic-stripped

Release state, checked 9 October 2026:

curl -s https://api.github.com/repos/moranyanuka/phonological-interference | grep -E '"(created_at|pushed_at|license)"'
# created 2026-10-06T18:06:19Z, pushed 2026-10-09T07:13:58Z, license null
curl -s "https://api.github.com/repos/moranyanuka/phonological-interference/git/trees/HEAD?recursive=1" | grep -c '"blob"'
# 67 (63 .mp3, 2 .woff, index.html, .nojekyll); project page says "Code (coming soon)"
for r in models/espnet/powsm models/changelinglab/PhoneticXeus models/FunAudioLLM/Fun-CosyVoice3-0.5B-2512 datasets/Shelton1013/SwitchLingua_audio; do
  curl -s https://huggingface.co/api/$r | grep -o '"license:[^"]*"\|"gated":[^,]*'; done
# cc-by-4.0 | cc-by-nc-sa-4.0 | apache-2.0 | cc-by-nc-4.0, gated "auto"

The probe needs about 1.1 GB for torch: python3 -m venv .venv && .venv/bin/pip install allosaurus && .venv/bin/python -I probe.py. It writes the clips to probe_audio/ and every SEP and TOG transcript to probe.jsonl.

What would prove this reading wrong

Our claims are that the 32% to 79% range is driven by PhoneticXeus's sensitivity to the dataset, and that interference concentrates on the language with more unshared phonemes, usually English.

We would be wrong if, once the code is released, a per-pair breakdown on SwitchLingua shows the partner-language span (Spanish, Italian, Korean) losing a larger share of its unshared phonemes than the English span in most pairs, or if PhoneticXeus on a second natural code-switched corpus (CS-FLEURS read speech, for example) lands near its 32% spliced figure rather than near 79%. We predict that by 30 April 2027 an independent replication on natural code-switched speech will report at least 40% relative loss of unshared-phoneme recall for at least one of POWSM or PhoneticXeus, and that the authors' repository will by then contain runnable code; if it still holds only the demo page on that date, readers should treat the WLE numbers as unreproduced.

Sources

1. Moran Yanuka, Raja Giryes, Morris Alper. Phonological Interference in Multilingual Speech Models. arXiv:2610.11275v1, submitted 8 October 2026 (Tel Aviv University; University of Miami). Abstract, Sections 2 to 4, Tables 1 to 18, Appendices A to K.
2. github.com/moranyanuka/phonological-interference and its project page, checked via api.github.com on 9 October 2026: created 6 October 2026, 14 commits, 67 files, no licence, code "coming soon".
3. changelinglab/PhoneticXeus (CC BY-NC-SA 4.0), espnet/powsm (CC BY 4.0), FunAudioLLM/Fun-CosyVoice3-0.5B-2512 (Apache 2.0), Hugging Face API, 9 October 2026.
4. Shelton1013/SwitchLingua_audio, CC BY-NC 4.0, gated, 16,925 files, Hugging Face API, 9 October 2026. Xie et al., SwitchLingua, NeurIPS 38.
5. Moran, S. and McCloy, D. (eds.). PHOIBLE, data/phoible.csv from the dev branch, fetched 9 October 2026.
6. Xinjian Li et al. Allosaurus universal phone recognizer, version 1.0.2 via pip, model latest; ICASSP 2020.
7. BLOMEGA computations, 9 October 2026: SEP/TOG back-solve and cross-checks (headline.py), Table 4 type and token counts and PHOIBLE comparison (inventory.py), allosaurus SEP/TOG probe (probe.py).

FAQ

Do multilingual phone recognizers and TTS really lose up to 79% of language-specific phonemes on code-switched speech?

The 79% is a relative drop in recall of the worse span's unshared phonemes, for one model on one dataset: PhoneticXeus on SwitchLingua falls from 0.645 recall with each span alone to 0.138 with both languages together. The 32% low end is the same model on spliced FLEURS (0.581 to 0.395). Across all eight model and dataset cells the relative loss is 32% to 79% for unshared phonemes and 3% to 24% for shared ones (Table 5, our recomputation).

Can I reproduce the paper or use its models commercially?

Not fully. On 9 October 2026 the GitHub repository held 67 files, 63 of them demo MP3s, and the project page marks code as coming soon; the IPA fine-tuned CosyVoice3 LoRA is not published. POWSM is CC BY 4.0 and CosyVoice3 is Apache 2.0, but PhoneticXeus, which is also the scorer for the TTS results, is CC BY-NC-SA 4.0, and the SwitchLingua audio is CC BY-NC 4.0 behind a gate.

What can a dubbing or ASR team do about code-switching interference today?

Process each language span on its own where the pipeline allows, because that is the paper's zero-interference baseline. Chunking alone is not enough: TTS generated span by span with shared context and no edit showed interference of 0.329 against 0.302 unchunked. Giving POWSM each span's language tag cut interference by 0.312 on SwitchLingua, and Whisper given the right language token transcribes nearly correctly, while with its own single pick it wrote nothing for the other span in 218 of 286 spliced clips.