DuraS2ST says its chain of thought plans dub length. In the 557,358 released rows, length was fixed before the plan was written

In DuraSet, the audio always matches the source length and the phoneme plan usually does not

Every DuraSet CoT ends with the same two sentences: "The source phoneme sequence is: [...] with a length of N" and "The target phoneme sequence is: [...] with a length of M". We parsed N and M from 534,555 of the 557,358 rows (the rest use another wording) and compared M/N with the ratio of the two *_duration fields in the same row.

The duration ratio barely moves. Its 5th to 95th percentile is 0.992 to 1.011 for English to Chinese and 0.987 to 1.008 for Chinese to English, standard deviation 0.009, which reproduces the paper's Figure 6 (mean 0.995, sd 0.008). The phoneme ratio spans 0.53 to 1.13 and 0.89 to 1.94 over the same percentiles. Only 50.6% of English to Chinese rows and 41.1% of Chinese to English rows have a phoneme ratio inside the plus or minus 20% window that SLC-0.2 uses for audio. Across all rows the two ratios correlate at -0.26: where a plan calls for relatively more target phonemes, the audio is, if anything, marginally shorter. All of this is our count; the paper reports no phoneme-ratio statistic.

Phoneme counts are not a common unit across English and Mandarin (the lexicon writes tone-marked finals), so a median of 0.81 in one direction and 1.25 in the other is expected. The spread is the point. The same model is trained to emit plans that range over a factor of two and audio that ranges over 2%.

The demo page shows the same pattern on model outputs. Its ten sentence clips have CoT phoneme ratios from 0.50 ("computing in parallel", 18 source phonemes planned as 9) to 1.47 (a 34-phoneme Chinese sentence planned as 50 English phonemes). We measured the audio: every output is between 0.96 and 1.075 of its source length. A plan that halves the phoneme count and a plan that adds half again both produce a source-length clip.

What CUHK and Microsoft released between 25 and 29 September 2026

arXiv:2609.33742, by Yayue Deng, Dingdong Wang, Yuxuan Hu, Jinyu Li, Yanqing Liu, Yuanyuan Wang, Weidong Chen, Helen M. Meng, Shujie Liu and Xixin Wu (The Chinese University of Hong Kong and Microsoft), was submitted on 27 September 2026. The repository README says it is accepted to the EMNLP 2026 Main Conference. The model is Step-Audio-2-mini-Think fine-tuned for 4,000 steps on DuraSet-440K with a <think> rationale before interleaved text and speech tokens, then 1,000 steps of GRPO with LoRA (rank 16) using a Duration Margin Reward and a span-level credit rule the authors call MARA (Sections 2 and 4.1, Appendix C.1).

The release, as we found it on 30 September 2026:

• github.com/Mia11939/DuraS2ST: one commit, authored 25 September and committed 29 September 2026, Apache 2.0. It holds inference.py, duras2st.py, token2wav.py and the CosyVoice2 decoder modules. There is no training, reward or evaluation code.
• Mia11939/DuraS2ST-Think-RL, created 26 September: four safetensors shards (16.6 GB) plus the token2wav decoder, Apache 2.0. The Baseline, Instruct and Think ablation checkpoints are not released.
• Mia11939/DuraSet-440K, created and last modified 27 September (revision 42b5782): 29 metadata parquets and 88 audio ZIP shards, 94.0 GB in total, CC BY-NC 4.0.
• A demo page with ten sentence clips, four video dubs and two dialect clips from the Wikitongues archive.

Every DuraSet target is synthetic twice over. GPT-4o wrote five candidate translations per source (Appendix B.1), an in-house zero-shot TTS voiced them with the source as timbre prompt (B.2), and GPT-4o at temperature 0 wrote the 150 to 300 word rationale for the winning pair (B.4). For a commercial dubbing team the licence matters first: the corpus is non-commercial even though the model trained on it is Apache 2.0.

The release reproduces the paper's duration statistics and not its row count

Dataset revision 42b5782 (last modified 27 September 2026), read in full over HfFileSystem on 30 September 2026. Row counts also confirmed by datasets-server.huggingface.co.

The duration statistics reproduce to the third decimal, so the release is the corpus the paper describes, in a larger form. We cannot tell from the files whether the extra 116,653 rows are the swap augmentation counted twice, a later build, or rows the paper filtered out; the card does not say. Our judgement is that a buyer or reproducer should treat 557,358 as the number and ask the authors which subset the 4,000 SFT steps saw.

On English to Chinese, UniSS beats DuraS2ST on Speech-BLEU while missing the 20% window on 1.3% more clips

The abstract says DuraS2ST outperforms "competitive open-source and commercial baselines". The paper's tables say that holds for Chinese to English and not for English to Chinese, where UniSS (Cheng et al., arXiv:2509.21144), the one baseline whose own abstract claims "duration consistency", keeps the translation lead.

Numbers copied from arXiv:2609.33742 Tables 1 and 2 (CVSS-T test, 4,897 utterance pairs). Differences below are our arithmetic.

On English to Chinese, DuraS2ST gives up 1.73 Speech-BLEU and 1.65 AVG BLEU to UniSS (Q) in exchange for 0.013 more SLC-0.2, about 64 of 4,897 clips, and 0.11 s less mean absolute error. On Chinese to English it wins both ways: +1.34 Speech-BLEU, +0.021 SLC-0.2, MADE 0.42 s against 0.74 s. GPT-4o also beats DuraS2ST on English to Chinese Speech-BLEU (31.46) and COMET (0.845).

No baseline was told how long to be. Appendix C.3 gives each one a plain translation prompt and says "we do not apply additional input preprocessing or output post-processing". That leaves out the baseline a dubbing engineer would try first: take the best translation and time-stretch it to the source length. By construction it scores SLC-0.2 of 1.0 and MADE near zero. The stretch it needs is roughly the MRDE column (our approximation): 6.7% on average for UniSS (Q) on English to Chinese, against 25% for GPT-4o and 42% for Step-Audio-2-mini. The paper dismisses time-stretching because it "may distort prosody and speaker timbre" (Section 1) but never measures that on any system.

Four smaller inconsistencies, all from the paper's own numbers:

• Section 4.3 says GRPO leaves duration "largely preserved or further improved" and cites MRDE (0.039 to 0.038). On English to Chinese, MADE rises from 0.18 s to 0.23 s, 28% worse.
• Table 4 is offered as proof that the duration reward "is the main driver". On Chinese to English, removing it gives better SLC-0.2 (0.319 against 0.302) and better MADE (1.93 s against 1.97 s) than keeping it. On English to Chinese, Zero-GRPO lowers SLC-0.2 below the untrained backbone (0.455 against 0.487).
• The "stepwise improvements from Baseline to Instruct, Think, and Think-RL" break on Chinese to English, where adding the CoT drops AVG BLEU from 26.12 to 24.88.
• The tables and Section 4.2.3 name GPT-4o; Appendix C.3 says the queried system was GPT-Audio.

The human evaluation does not settle the timing question either way. Its three scales are adequacy, naturalness and speaker similarity (Appendix E), with no video and no synchrony rating. The untrained backbone, whose SLC-0.2 is 0.487 on English to Chinese, scores 4.01 MOS there against UniSS's 3.85.

How a 2x frame-scaling step does the work the chain of thought is credited with

The corpus pipeline explains the flat duration line. In Appendix B.2 each candidate translation is synthesised, its natural duration estimated, and the predicted acoustic frames multiplied by s = clip(budget / natural, 0.5, 2.0) before the waveform is made. That is a time-scale change of up to 2x in either direction, done inside the TTS rather than on the waveform. The filter then keeps pairs within 20% and 5 s of the source and picks the one with the smallest duration error (B.3). Only then does GPT-4o write the rationale, given the phoneme counts of the pair already chosen (B.4).

Our reading, marked as a judgement: DuraS2ST learned to produce source-length speech because every one of its targets was source-length speech, stretched or squeezed by the synthesiser to get there. The learned speaking rate does the fitting. Some of the data shows it: synthesised English targets run at 11.4 phonemes per second at the median and 17.9 at the 95th percentile, against 10.7 and 14.6 for natural English sources in the same corpus (our count, using the CoT phoneme counts). The CoT still helps on the paper's numbers, cutting MADE from 0.63 s to 0.18 s on English to Chinese against the no-CoT Instruct variant, so the reasoning may act as a useful scratchpad. What the corpus does not contain is evidence that the phoneme arithmetic in it drives the length.

The RL reward and the headline metric part ways above a 15% gap

The Duration Margin Reward uses tolerance 5%, smoothing 0.02, sharpness 0.5 and clip bound 5 (Appendix C.2). We evaluated Equation 4 with those constants. A perfect match earns 0.93, not 1.0, because the upper clip is only reachable at a negative gap. A 5% gap earns 0.75, a 10% gap 0.43, and from 15% on the reward is exactly 0. SLC-0.2 still counts a 19% miss as a full success, and SLC-0.4 a 39% miss.

The reward is far stricter than the metric, which is a reasonable design choice, and it means SLC-0.2 near 1.0 says little about how tight the timing is. MADE and MRDE carry that information, and the paper does report them. For lip-sync work the useful figure is Think-RL's 0.23 s mean absolute error on English to Chinese, not the 0.997.

What this means for teams building dubbing pipelines

• Run the stretch baseline before adopting a duration-trained model. Take your current best translation, time-stretch it to the source span, and score Speech-BLEU, MADE and a naturalness MOS. On the paper's numbers, UniSS (Q) would need about 6.7% average stretch on English to Chinese and already has the higher Speech-BLEU. A duration-trained model earns its place only if it beats that.
• Read MADE, not SLC-0.2. SLC-0.2 saturates: 0.984 and 0.997 are 64 clips apart out of 4,897. MADE separates the same systems by 0.11 s, which is the scale at which a lip flap is visible. Our view of how a sync metric can reward the wrong behaviour is in the never-silent lip-sync baseline.
• Expect source-length output whatever the text. The demo clips stay within 0.96 to 1.075 of the source even when the plan halves the phoneme count. For content with fixed pauses (a speaker who stops mid-shot) that is a feature; for a translation that is naturally 25% longer it means faster speech, the same trade a time-stretch makes.
• Plan for offline only. The Limitations section says the full CoT decodes before any audio token, so this is not a live dubbing design. Latency budgets for that case are in our IWSLT 2026 latency note.
• Check licences before training on DuraSet. CC BY-NC 4.0, GPT-4o translations and in-house TTS voices cloned from LEMAS speakers. It is research data, not a commercial dubbing corpus.

Check it yourself

The row count takes one request:

curl -s "https://datasets-server.huggingface.co/size?dataset=Mia11939/DuraSet-440K" \
  | python3 -c "import json,sys; d=json.load(sys.stdin)['size']; print(d['dataset']['num_rows'], [(c['config'], c['num_rows']) for c in d['configs']])"
# expect 557358 [('en-zh', 314229), ('zh-en', 243129)]; the paper says 440,705

The plan against length check on the smallest shard (3,129 rows, about 2 MB) runs in seconds:

pip install huggingface_hub pyarrow
python3 - <<'PY'
import re, statistics as st
from huggingface_hub import HfFileSystem; import pyarrow.parquet as pq
fs = HfFileSystem()
f = 'datasets/Mia11939/DuraSet-440K/data/zh-en/train-00012.parquet'
pat = re.compile(r'The source phoneme sequence is:.*?with a length of (\d+)\..*?'
                 r'The target phoneme sequence is:.*?with a length of (\d+)', re.S)
with fs.open(f) as h:
    t = pq.read_table(h, columns=['source_duration', 'target_duration', 'cot']).to_pylist()
dur = [r['target_duration'] / r['source_duration'] for r in t]
ph = [int(m[2]) / int(m[1]) for r in t if (m := pat.search(r['cot']))]
print('rows', len(t), 'duration sd', round(st.pstdev(dur), 4),
      'within 5%', round(sum(abs(x - 1) <= .05 for x in dur) / len(dur), 4))
print('CoT phoneme ratio within 20%', round(sum(abs(x - 1) <= .2 for x in ph) / len(ph), 3))
PY
# expect: rows 3129, duration sd 0.0091, within 5% 0.9965; CoT within 20% 0.223

For the full numbers, glob datasets/Mia11939/DuraSet-440K/data/*/*.parquet (29 files, 476 MB of parquet, about 8 minutes on a home line) and run the same loop. Expected: duration ratio p5 to p95 0.992 to 1.011 (EN→ZH) and 0.987 to 1.008 (ZH→EN); phoneme ratio inside 20% on 50.6% and 41.1%; Pearson between the two ratios -0.26. The reward curve is three lines:

python3 -c "
import math; ls = lambda x: -math.log1p(math.exp(-x))
R = lambda d: (ls(0.5*max(-5, min(5, (0.05-d)/0.02))) - ls(-2.5)) / (ls(2.5) - ls(-2.5))
print([(d, round(R(d), 3)) for d in (0, .05, .10, .15, .20)])"
# expect [(0, 0.931), (0.05, 0.754), (0.1, 0.431), (0.15, 0.0), (0.2, 0.0)]

Demo durations: download assets/audio/s1_en2zh_source.wav and s1_en2zh_duras2st.wav (and s2 to s5, both directions) from https://mia11939.github.io/DuraS2ST-Demo/ and run ffprobe -v error -show_entries format=duration -of csv=p=0 <file>. The repository contents are listed by curl -s "https://api.github.com/repos/Mia11939/DuraS2ST/git/trees/main?recursive=1".

What would prove this wrong

Our claim is that DuraS2ST's duration comes from being trained on speech that a synthesiser had already fitted to the source length, and that the phoneme plan in its CoT does not set the output length.

It would be wrong if the released Think-RL model, run with --show-reasoning on 200 or more CVSS-T utterances, produced output duration ratios that track the CoT's stated target/source phoneme ratio with a Pearson correlation above 0.5. We predict below 0.3. It would also be wrong if a time-stretch baseline (UniSS Q output stretched to source length) scored lower Speech-BLEU than DuraS2ST's 30.43 on CVSS-T English to Chinese. We predict that by 31 March 2027 any published comparison that includes a stretched UniSS or GPT-4o baseline on this test set will show it within 1 Speech-BLEU of its unstretched score and ahead of DuraS2ST on English to Chinese, with the open question being naturalness MOS. A revised DuraSet card that reconciles 557,358 with 440,705 would settle the count, not the mechanism.

Sources

1. Yayue Deng, Dingdong Wang, Yuxuan Hu, Jinyu Li, Yanqing Liu, Yuanyuan Wang, Weidong Chen, Helen M. Meng, Shujie Liu, Xixin Wu. DuraS2ST: Chain-of-Thought and Reinforcement Learning for Duration-Aligned Speech-to-Speech Translation. arXiv:2609.33742v1, 27 September 2026. Sections 1 to 4, Tables 1 to 4, Figure 6, Appendices B, C and E.
2. Mia11939/DuraSet-440K on Hugging Face: card and all 29 data/*/*.parquet files, revision 42b5782, last modified 27 September 2026, read 30 September 2026.
3. Mia11939/DuraS2ST-Think-RL model repository, created 26 September 2026, file list checked 30 September 2026.
4. github.com/Mia11939/DuraS2ST, commit f9a5b1c (committed 29 September 2026), README and tree checked 30 September 2026.
5. DuraS2ST demo page, ten sentence clips downloaded and measured 30 September 2026.
6. Sitong Cheng et al. UniSS: Unified Expressive Speech-to-Speech Translation with Your Voice. arXiv:2509.21144, 2025. Abstract, for its duration-consistency claim.
7. BLOMEGA analysis, 30 September 2026: full read of DuraSet-440K (row counts, hours, duration and phoneme ratios, swap and script checks, speaking rates), Equation 4 evaluated at the Appendix C.2 constants, and arithmetic on Tables 1, 2 and 4.

FAQ

How big is the released DuraSet-440K?

557,358 rows (314,229 English to Chinese, 243,129 Chinese to English) and 1,278.7 hours of source audio, by the Hugging Face card, the datasets-server row count and our own read of all 29 parquet files on 30 September 2026. The paper reports 440,705 utterances (229,410 and 211,295) and roughly 1,000 hours. 134,795 rows are marked as source-target swaps. Licence is CC BY-NC 4.0.

Does DuraS2ST's chain of thought control how long the dubbed speech is?

Not in its training data. In the released corpus the target audio is within 5% of the source length on 99.75% of rows, because the TTS that made the targets scaled its frames by up to 2x to hit the source duration. The phoneme counts the CoT cites for source and target are within 20% of each other on only 50.6% of English to Chinese rows and 41.1% of Chinese to English rows, and what little duration variation remains does not rise with the planned phoneme ratio (Pearson -0.26). The paper does show the CoT variant has lower duration error than the no-CoT variant.

Is DuraS2ST better than UniSS for English to Chinese dubbing?

On the paper's own Table 2, UniSS (Quality mode) has higher Speech-BLEU (32.16 against 30.43) and AVG BLEU (32.60 against 30.95), while DuraS2ST lands 1.3 points more clips inside the plus or minus 20% duration window (0.997 against 0.984) and has lower MADE (0.23 s against 0.34 s). On Chinese to English DuraS2ST leads on both. No baseline was given the source duration or a time-stretch step.