The English refusal direction transfers to zero of nine languages, and layer choice explains why
A pre-registered study published 28 October 2026 in the Proceedings of the 6th Workshop on Multilingual Representation Learning (MRL 2026) tested whether the single internal direction that makes an LLM refuse harmful prompts in English is the same mechanism in nine other languages. Under its own pass or fail rule, transfer held in zero of nine languages, for both Qwen2.5-7B-Instruct and Llama-3.2-3B-Instruct, at every layer tested. The same paper shows that number is partly a trap: in Qwen, the cosine similarity between the English direction and each language's own version swings from 0.93 at layer 19 to 0.38 at layer 28, the layer the study's own pre-registered rule selected, while the model's confidence in separating refusal from compliance barely moves, 0.9904 to 0.9942, across that entire span.
What changed, and when
Arditi et al. showed at NeurIPS 2024 that refusal in English chat models is mediated to a striking degree by one linear direction in the residual stream: remove it and a model stops refusing; add it and a model refuses harmless requests. A May 2026 arXiv paper, Refusal Direction is Universal Across Safety-Aligned Languages (2505.17306), extended the claim across languages, reporting high geometric alignment and arguing an English-derived vector can be used to bypass refusals in other languages.
The MRL 2026 paper, Does Safety Live in a Shared Multilingual Subspace? Cross-Lingual (Non-)Transfer of the Refusal Direction, by Ruiyang Zhang of FLock.io Ryonix Labs Inc., pages 245 to 256 of the workshop proceedings, dated 28 October 2026 and co-located with EMNLP 2026 in Budapest, re-runs that question with a pre-registration filed before any activation was computed. It uses MultiJail (Deng et al., 2024: 315 human-translated unsafe prompts in ten languages spanning high, medium and low resource tiers) and 100 benign prompts per language from the Aya evaluation suite's Dolly split. Two open, instruction-tuned models receive the full analysis: Qwen2.5-7B-Instruct in 4-bit NF4 quantization, and Llama-3.2-3B-Instruct in bf16 as an unquantized check. A third model, Aya-Expanse-8B, was pre-registered but did not finish on the available hardware and is left for future work.
The method: estimate a per-language, per-layer refusal direction as a difference in means between unsafe and benign prompt activations, exactly as Arditi et al. did for English. Compare each language's direction to English's with cosine similarity, and compare that to a matched non-safety control direction built from the same corpus. Then test causally: does ablating the English direction reduce refusal, and does adding it induce refusal, in each of the other nine languages, by at least 20 percentage points. The pre-registered bar for "the mechanism is shared" requires both halves to clear 20 points in at least 7 of the 9 non-English languages.
The per-language numbers behind the headline
At the layer each model's own pre-registered rule selected (layer 28 of Qwen's 28 transformer layers, layer 13 of Llama's), the study reports a cosine similarity between each language's refusal direction and English's, alongside each language's baseline refusal rate. A matched non-safety control direction, built from the same corpus with the same estimator, shows whether a high cosine means anything specific to safety or is just how the model's multilingual representations sit in general.
| Language | Resource tier | Qwen2.5-7B cosine vs. English | Qwen2.5-7B refusal rate | Llama-3.2-3B cosine vs. English | Llama-3.2-3B refusal rate | Source |
|---|---|---|---|---|---|---|
| English | pivot | 1.00 (self) | 79.7% | 1.00 (self) | 72.1% | Zhang, MRL 2026, Table 1 |
| Chinese | high | 0.56 | 76.5% | 0.80 | 71.1% | Zhang, MRL 2026, Table 1 |
| Italian | high | 0.34 | 71.7% | 0.79 | 73.7% | Zhang, MRL 2026, Table 1 |
| Vietnamese | high | 0.40 | 67.9% | 0.83 | 69.8% | Zhang, MRL 2026, Table 1 |
| Arabic | medium | 0.43 | 71.4% | 0.74 | 67.6% | Zhang, MRL 2026, Table 1 |
| Korean | medium | 0.52 | 70.2% | 0.78 | 42.5% | Zhang, MRL 2026, Table 1 |
| Thai | medium | 0.49 | 66.0% | 0.80 | 63.5% | Zhang, MRL 2026, Table 1 |
| Bengali | low | 0.38 | 29.5% | 0.76 | 31.4% | Zhang, MRL 2026, Table 1 |
| Swahili | low | 0.09 | 5.7% | 0.64 | 25.7% | Zhang, MRL 2026, Table 1 |
| Javanese | low | 0.23 | 37.8% | 0.66 | 64.4% | Zhang, MRL 2026, Table 1 |
| Median across 9 languages | n/a | 0.38 | not reported (single figure) | 0.84 | not reported (single figure) | Zhang, MRL 2026, abstract and §4 |
| Matched non-safety control | n/a | 0.33 | not applicable | 0.86 | not applicable | Zhang, MRL 2026, §4 (H1c) |
Qwen's refusal directions are only weakly aligned across languages, and barely more so than a direction with nothing to do with safety. The 0.38 median cosine sits just 0.05 above the 0.33 control (confidence interval 0.01 to 0.09, excluding zero), so the alignment is real but small. Llama's refusal directions look nearly parallel across languages, 0.84, but its control direction is just as aligned, 0.86 (difference -0.02, interval -0.06 to 0.02, including zero). The paper's reading: a high cross-lingual cosine in Llama says nothing about safety specifically, it is a generic property of how that model arranges any concept across languages.
Why the causal test returns zero of nine, and why that number is not the full story
The pre-registered transfer criterion is conjunctive: for a language to count as "transferred," ablating the English direction has to cut that language's refusal rate by at least 20 percentage points, and adding the English direction has to induce at least 20 points of refusal on benign prompts in that language. Both halves, same language, same layer. Under that rule, the result is 0 of 9 non-English languages, in both models, at every layer the paper tested, including each model's pre-registered layer of record.
Reading the two halves separately changes what that zero means. In Qwen, at the mid-depth layer 16 rather than the selected layer 28, ablating the English direction removes 61.9 percentage points of refusal in English itself, and 47.9 to 59.7 points in six of the nine other languages, close to or exceeding what each language's own direction removes. But at that same layer, adding the English direction barely moves anything, +7.0 points in English, 2.0 to 17.0 points elsewhere, so the addition half never clears the bar and the conjunction fails. At layer 28, the layer the pre-registered rule actually selected, neither half works even in English: ablation is -1.3 points, addition is +5.0 points. A zero recorded at that layer is not a test the mechanism failed; it is a test that was never run on a live signal.
Llama dissociates the opposite way. At its layer of record, 13, adding the English direction induces 32 to 92 points of refusal on benign prompts in eight of the nine other languages, 1.5 to 5.3 times what each language's own direction induces, while moving English itself by only 15.0 points, below the 20-point bar. The ablation half is inert everywhere, including English (-0.6 points). Across both models, the paper reports no layer at which the English direction moves refusal in English and fails to move it elsewhere; the two halves just never happen to both fire at once.
The paper is explicit that this cuts both ways: a low cross-lingual cosine, as in Qwen, does not predict causal fragmentation either, since Qwen is also the model where ablation crosses to six languages. Geometry at one layer tells you less than it looks like it tells you, in either direction.
What this means for training or buying a multilingual model
An English-only red-team result does not certify a multilingual model, and it does not have to, to still be dangerous. The paper's own defensive reading: where the English refusal direction is an effective lever at all, it tends to be effective across languages too. An alignment edit, or an attack, estimated in English should be assumed to reach a model's other languages until shown otherwise. Per-language safety evaluation stays necessary because the behavioral gap in Table 1 is large and real, not because the representational question is settled.
Do not cite a single cosine number as proof either way. A vendor or paper reporting "the refusal direction transfers across languages, cosine 0.84" has told you the number at one layer in one model. The same quantity in the same model ranges from 0.38 to 0.93 depending only on depth, while the model's own task confidence (AUROC) stays essentially flat, 0.9904 to 0.9942, across that range. Ask for the full per-layer curve, or the layer-selection rule and whether it was fixed before results were seen, not a single headline cosine.
Null causal results need a check that the test was live. A "transfer failed" finding is only informative if the intervention worked in the source language at that same layer. This study's own 0/9 headline includes layers where neither ablation nor addition moved English refusal at all, which makes those particular zeros uninformative rather than disconfirming.
Our judgment: for anyone purchasing or red-teaming a multilingual model for deployment in low-resource languages, the operational takeaway is narrower than either "safety is shared" or "safety is fragmented." It is that English-language safety testing cannot stand in for per-language testing, and that native-speaker-labeled harmful and benign prompt sets, built and consented to per language rather than machine-translated from an English benchmark, remain the only direct way to measure what a model actually does in Swahili or Javanese, independent of what its internal geometry suggests.
Check it yourself
The paper, its pre-registration, deviations log, and a verifier script that re-derives every reported number from persisted activations and labels are released publicly.
# 1. Read the paper directly from the ACL Anthology
curl -sL -A "Mozilla/5.0" -o mrl2026-refusal.pdf \
"https://aclanthology.org/2026.mrl-1.19.pdf"
pdftotext mrl2026-refusal.pdf - | less
# Table 1 (per-language cosines and refusal rates) and Table 2
# (the layer-16 vs layer-28 dissociation) are both in the plain text.
# 2. The proceedings front matter confirms venue and date
curl -sL -A "Mozilla/5.0" "https://aclanthology.org/2026.mrl-1/" \
| grep -io "october 2026"
# 3. Code, pre-registration, deviations log and the verifier (verify.py),
# tagged v1.0-mrl2026:
# https://github.com/ryonzhang/refusal-direction-multilingual
# verify.py re-derives directions, cosines, held-out AUROCs, the layer
# of record, refusal rates and steering deltas from persisted artefacts;
# raw activations (~26 GB) regenerate deterministically via run_all.sh.
# 4. Read the prior claim this complicates
curl -sL "https://arxiv.org/abs/2505.17306" | grep -io ".* "
The MultiJail benchmark the study uses (Deng et al., 2024) and the Aya evaluation suite's Dolly split (Singh et al., 2024) are both publicly released, so the prompt sets behind every number in Table 1 are independently inspectable, not something taken on the author's word.
What would prove this wrong
The claim under test is that the paper's 0-of-9 cross-lingual transfer result is partly a layer-selection artifact rather than clean evidence that safety mechanisms fragment by language. This is wrong if a future pre-registered study, using a layer-selection rule fixed before any cross-lingual or steering result is seen (not post-hoc, as this paper's own ℓ* criterion was, chosen only from held-out English AUROC), finds the conjunctive 20-point ablate-and-add criterion still fails in at least 7 of 9 languages at the layer where ablation is demonstrably live in English. That would show the dissociation reported here, between a live layer and a null layer, does not generalize beyond this one pre-registered run. A reasonable date to check: by 1 October 2027, the next MRL workshop cycle, given the original study flags Aya-Expanse-8B and the behavior-conditioned direction control as its own stated next steps.
A second, narrower prediction, marked as judgment: once Aya-Expanse-8B or a comparable model completes the same protocol, we expect the layer-16-style dissociation (ablation live, addition inert, or vice versa) to recur rather than resolve into a clean 7-of-9 pass, because the paper's own evidence is that whichever intervention works in English also tends to cross languages, just not always at the layer a fixed selection rule happens to land on. If a completed third-model run instead shows both halves firing together at the pre-registered layer, that reading was wrong.
FAQ
Does an LLM's safety training in English transfer to other languages?
Behaviorally, only partly. Qwen2.5-7B-Instruct refuses 79.7 percent of harmful prompts in English but only 5.7 percent in Swahili and 29.5 percent in Bengali, out of 315 prompts per language from MultiJail. Llama-3.2-3B-Instruct shows the same pattern, 72.1 percent in English against 25.7 percent in Swahili. Whether that gap traces to one shared internal mechanism that weakens outside English, or to separate per-language mechanisms, is what the MRL 2026 study tested, and its answer was not a clean yes or no.
What is a refusal direction in a language model?
A single linear direction in a model's internal activations that, when subtracted, stops the model from refusing harmful requests, and when added, makes it refuse harmless ones. Arditi et al. established this for English at NeurIPS 2024. Ruiyang Zhang's MRL 2026 paper estimates the same kind of direction independently in ten languages for two open models and tests whether the English-estimated direction also controls refusal when applied to the other nine.
Did this study prove multilingual AI safety is shared or fragmented across languages?
Neither, by the paper's own framing. Its pre-registered conjunctive test (ablate and add the English direction, each must move refusal 20 points) was met in 0 of 9 non-English languages in both models at every layer. But at Qwen's layer 16, ablation alone cut refusal by 61.9 points in English and 47.9 to 59.7 points in six other languages; at the layer the study's rule actually selected, layer 28, neither intervention moved English refusal at all. A null result at an uninformative layer is not evidence that safety fragments by language.
Why does this paper disagree with the earlier claim that the refusal direction is universal across languages?
The earlier paper, Refusal Direction is Universal Across Safety-Aligned Languages (arXiv 2505.17306), reported high cross-lingual alignment. The MRL 2026 study measures that same alignment across every layer of Qwen2.5-7B and finds it swings from a median cosine of 0.93 at layer 19 to 0.38 at layer 28, while the model's own confidence in separating refusal from compliance stays flat, 0.9904 to 0.9942 AUROC, across that range. Which number gets reported, and which conclusion follows, depends heavily on which layer a study happens to measure.
Sources
- Ruiyang Zhang, Does Safety Live in a Shared Multilingual Subspace? Cross-Lingual (Non-)Transfer of the Refusal Direction, Proceedings of the 6th Workshop on Multilingual Representation Learning (MRL 2026), pages 245 to 256, 28 October 2026, Association for Computational Linguistics. PDF: aclanthology.org/2026.mrl-1.19.pdf. Code and verifier: github.com/ryonzhang/refusal-direction-multilingual (tag v1.0-mrl2026).
- ACL Anthology, Proceedings of the 6th Workshop on Multilingual Representation Learning (MRL 2026), front matter confirming venue (Budapest, co-located with EMNLP 2026) and date (28 October 2026).
- Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee and Neel Nanda, Refusal in Language Models Is Mediated by a Single Direction, Advances in Neural Information Processing Systems (NeurIPS), 2024. Establishes the English-only refusal-direction mechanism this study extends.
- Wang et al., Refusal Direction is Universal Across Safety-Aligned Languages, arXiv 2505.17306, May 2026. The prior claim the MRL 2026 paper's layer-dependence finding complicates.
- Deng et al., MultiJail, 2024. 315 human-translated unsafe prompts in ten languages across high, medium and low resource tiers; the benchmark underlying every refusal-rate number in Table 1 above.
- Singh et al., Aya evaluation suite (machine-translated Dolly split), 2024; Conover et al., Dolly, 2023. Source of the 100 benign prompts per language used for the control direction and the addition intervention.
Related BLOMEGA guides: Multilingual RAG reranker metric gap · The tokenization premium in Bengali and Yoruba.