LLMs default to MSA when writing Arabic dialects, but a layer-by-layer probe finds no internal MSA hub to blame
On 8 October 2026 researchers at TU Dresden and MBZUAI adapted a language-dominance probing framework to 26 Arabic varieties (25 dialects plus Modern Standard Arabic) across five model families. If LLMs internally routed dialectal Arabic through MSA the way they default to generating MSA output, a dominance score should rise well above the uniform no-hub baseline of 1/26 ≈ 0.0385. Across layers and architectures, it does not: dominance scores stay close to that baseline, and the share of tokens aligning more strongly with a competing dialect than their own sits flat at around 30% across depth, not concentrated on MSA.
Arabic dialects overlap heavily in hidden representations, but no single variety, MSA or otherwise, acts as a hub
The paper separates two questions prior work had conflated: whether Arabic varieties are separable in an LLM's hidden states, and whether one variety is dominant, meaning other varieties' representations systematically align with it. Using the parallel MADAR corpus (25 city-level Arabic dialects plus MSA, travel-domain sentences), the authors measure separability with normalized mutual information (NMI) between dialect labels and unsupervised cluster assignments at each transformer layer, and dominance with a token-level likelihood ratio adapted from Shani and Basirat (2025). For mGPT and BLOOM, average NMI across the middle transformer layers is around 0.1 and 0.2 respectively, which the paper describes as "substantially lower and smoother" than the curves it reproduces for clearly distinct languages using the same probing method on Parallel Universal Dependencies treebanks. Despite that low separability, confusion matrices still show clear diagonals (dialect identity is not erased), and a supervised linear probe confirms dialect labels remain predictable above chance.
What Sallouh, Popovič and Färber tested, and why Arabic is a hard case for "language dominance"
arXiv:2610.11510v1, Does Modern Standard Arabic (MSA) Dominate Arabic Dialects in LLMs? A Representation-Level Analysis, by Abdu Sallouh (TU Dresden), Nicholas Popovič (MBZUAI, work done at TU Dresden/ScaDS.AI) and Michael Färber (TU Dresden, ScaDS.AI), was submitted 8 October 2026. Prior work (Robinson et al., 2025) documented that LLMs often default to MSA when generating Arabic even when prompted in a specific dialect. This paper asks whether that output-level bias has a mechanistic explanation: does MSA act as an internal "dominant language," the way some multilingual models have been shown to route cross-lingual processing through a single pivot language more frequent in pretraining (Shani and Basirat, 2025)?
Arabic is a deliberately hard test case: its varieties form a dialect continuum sharing script, vocabulary and morphology with MSA, unlike the clearly bounded languages the original dominance framework was built on. The authors sample 400 parallel sentences per variety for mGPT and BLOOM (four times the per-language sample size of the original framework) and 100 per variety for three additional models (Meta-Llama-3.1-8B-Instruct, Qwen2.5-7B, and ArabianGPT-08B-V2, an Arabic-native pretrained model), extracting hidden states across all transformer layers for each.
What the dominance statistics actually show, against what "MSA dominance" would require
| Measurement | What dominance would predict | What was found | Source |
|---|---|---|---|
| Layer-wise dominance score (26 varieties) | MSA (or another variety) scores well above uniform across most layers | stays close to the uniform no-hub baseline, 1/26 ≈ 0.0385, across layers | §4, Appendix E Fig. 9 |
| Share of tokens with Λ<1 (aligns more with a competitor than own dialect) | a middle-layer spike concentrated on alignment toward MSA specifically | nearly flat around 30% across layers, no MSA-specific concentration | §4, Figure 3 |
| Same Λ probe on clearly distinct languages (replication check) | (reference case) | produces a clear middle-layer dominance spike, confirming the probe detects real dominance when present | Appendix E, Figure 14 |
| Dialect separability (NMI), mGPT / BLOOM | not directly a dominance measure | ~0.10 / ~0.20, "substantially lower" than the distinct-language replication | §4, Figure 1 |
| Architecture dependence of separability location | not predicted either way | Llama-3.1 stable; Qwen2.5 early drop then partial recovery; ArabianGPT high until late layers then sharp collapse | Appendix B, Figures 4-6 |
| Stability check: function-word / sentence-initial artefacts removed | effect should vanish if driven by superficial cues | main trends preserved under POS filtering and prefix removal | Appendix C |
| Exact cross-lingual (distinct-language) NMI baseline value | – | not reported as a single number in the main text, only as "substantially lower" with a figure reference | §4 |
All rows reflect arXiv:2610.11510v1's own reported statistics; the final row is noted as an explicit gap in what the paper states numerically, not a value we computed independently.
Why "MSA-biased output" and "MSA-dominant representation" are different claims
The logical structure of the paper's central result is a contrast between two possible internal pictures that would both be consistent with the known, separately-established fact that LLMs tend to generate MSA output for dialectal prompts.
The same Λ probe, applied to the authors' own replication on clearly distinct languages, produces a visible middle-layer dominance spike (Appendix E, Figure 14), which is the control that makes the Arabic null result informative rather than a sign the method simply cannot detect dominance. If the probe were insensitive to real dominance, finding none for Arabic would mean nothing; because it detects dominance when it exists elsewhere, failing to find it here is evidence against the hypothesis rather than an absence of evidence.
What this means for teams building or evaluating dialectal Arabic systems
- Fixing MSA-biased output is unlikely to be a representation-surgery problem. If the bias were a routing artefact (dialectal input gets internally rewritten toward MSA), a targeted intervention on a small hidden-state subspace might fix it, the way the phonological-interference literature's windowed language estimation targets a 15-dimension subspace for code-switching. This paper's result suggests that approach would not apply here: there is no equivalent subspace to intervene on, because no dominance was found to intervene against.
- Training-data share and orthography remain the more actionable levers. The authors name pretraining/instruction-tuning data composition and dialectal spelling standardization as the plausible remaining explanations, both addressable through data collection and normalization choices rather than model-internals work.
- Arabic-native pretraining does not uniformly fix dialect separability. ArabianGPT-08B-V2, trained predominantly on Arabic data, does not show better separability than the general multilingual models across most layers; it shows a qualitatively different pattern (late, sharp collapse), which the authors read as a shift in where dialectal compression happens rather than an improvement.
- Behavioral benchmarks and representation probes answer different questions. A dialect-generation benchmark (the kind BLOMEGA has covered for Arabic error spans and sentiment-copy baselines) measures what a model outputs; this probe measures how it organizes information internally. A system can fail the first while this analysis finds nothing unusual in the second, and a team should not expect a representation-level fix for a behavior-level problem.
How to rerun the probe yourself
# The paper's own code and data, confirmed live
git clone https://github.com/abduABS/MSA_Dominance_LLM
# MIT licence, pushed 25 May 2026 (predates the 8 October 2026 arXiv submission)
curl -s https://api.github.com/repos/abduABS/MSA_Dominance_LLM \
| python3 -c "import json,sys;d=json.load(sys.stdin);print(d['license']['spdx_id'], d['pushed_at'])"
# Re-fetch the paper and confirm the dominance-baseline and Lambda<1 figures
curl -sL https://arxiv.org/pdf/2610.11510 -o msa.pdf
python3 -c "
import pypdf
t = '\n'.join(p.extract_text() for p in pypdf.PdfReader('msa.pdf').pages)
i = t.find('uniform baseline')
print(t[i-200:i+300])
"The MADAR corpus itself (Bouamor et al., 2018) requires a data-sharing agreement from its maintainers (CAMeL Lab, NYU Abu Dhabi) and is not redistributed in the paper's own repository; reproducing the exact NMI and dominance figures requires obtaining it separately.
What would prove this reading wrong
Our claim is that this paper's evidence (dominance scores near the uniform baseline, flat ~30% Λ<1 rates not concentrated on MSA, confirmed against a working control on distinct languages) genuinely rules out simple internal-routing-through-MSA as the mechanism behind MSA-biased generation, for the five architectures tested. We would be wrong if a larger-sample replication (the additional models here used only 100 sentences per variety, a quarter of the mGPT/BLOOM sample) found a dominance signal that this smaller sample was underpowered to detect, particularly for ArabianGPT, whose late-layer collapse pattern is the least explored of the five. We predict that by 31 December 2027, if someone runs the identical Λ probe on a newer Arabic-native model with at least 400 sentences per variety (matching the mGPT/BLOOM sample size) and finds a dominance score exceeding twice the uniform baseline (>0.077) concentrated specifically at MSA in more than half of tested layers, that would overturn this paper's central negative result for at least that model family.
Sources
- Abdu Sallouh, Nicholas Popovič, Michael Färber. Does Modern Standard Arabic (MSA) Dominate Arabic Dialects in LLMs? A Representation-Level Analysis. arXiv:2610.11510v1, submitted 8 October 2026 (TU Dresden; MBZUAI; ScaDS.AI). Abstract, §1-6, Figures 1-6, Appendices A-E.
- github.com/abduABS/MSA_Dominance_LLM, MIT licence, checked via api.github.com on 13 October 2026.
- Houda Bouamor et al. MADAR Arabic dialect corpus and lexicon, cited in arXiv:2610.11510v1 as the parallel 25-dialect-plus-MSA dataset used throughout.
- Nadav Shani, Ali Basirat. Language Dominance in Multilingual Large Language Models, 8th BlackboxNLP Workshop, 2025. Source of the dominance-probing framework adapted in arXiv:2610.11510v1 from cross-lingual to cross-dialectal analysis.
- Nathaniel R. Robinson, Shahd Abdelmoneim, Kelly Marchisio, Sebastian Ruder. AL-QASIDA: Analyzing LLM Quality and Accuracy Systematically in Dialectal Arabic, Findings of ACL 2025, cited in arXiv:2610.11510v1 as the prior work establishing MSA-biased generation for dialectal Arabic prompts.
- BLOMEGA computations, 13 October 2026: chart rendering of §4's reported NMI and dominance statistics, GitHub release-state check.
FAQ
Does Modern Standard Arabic act as an internal hub when LLMs process Arabic dialects?
No evidence of it, per arXiv:2610.11510. Across mGPT and BLOOM, layer-wise dominance scores stay close to the uniform no-dominance baseline of 1/26 (about 0.0385) across 26 Arabic varieties, and the share of tokens whose representation aligns more with a competing dialect than their own stays flat at around 30% across layers, rather than spiking toward MSA specifically.
Why do LLMs generate MSA when prompted with Arabic dialects, if not internal dominance?
The paper does not identify the cause, but rules out one candidate explanation (internal representational routing through MSA) and lists plausible remaining contributors: a disproportionate share of MSA in pretraining and instruction-tuning data, MSA's more standardized orthography compared to unstandardized dialectal spelling, and decoding-time preferences shaped by RLHF and higher-frequency token paths.
Do all Arabic language models show the same representational pattern?
No. Of five tested models, ArabianGPT-08B-V2, trained predominantly on Arabic data, shows the most distinctive pattern: dialect separability (NMI) remains high until later transformer layers and then collapses sharply, unlike mGPT, BLOOM, Llama-3.1 and Qwen2.5, which separate dialects less sharply but earlier or more gradually.