"Rather than" is 8.0 times more frequent in cs.CL abstracts than in 2019, and two readers agree on which uses annoy at kappa 0.18

What the Coruña group posted on 7 October 2026

Olga Zamaraeva, Adrián Gude, Roi Santos-Ríos and Carlos Gómez-Rodríguez (Universidade da Coruña, CITIC) posted "I would rather quit NLP than read another paper like this: The rise of antithesis in NLP papers", arXiv:2610.10092v1, cs.CL, 34 PDF pages (the arXiv comment says 30 pages, 2 figures, 48 tables). It is the only version as of 8 October 2026. Code and data are at olzama/rather-than-public, MIT licence, created 6 October 2026, two commits.

The study compares 4,744 ACL Anthology papers from 2019 with 1,821 ACL-style cs.CL arXiv papers from January to September 2026, plus papers that GPT-4o, GPT-5.6-sol, GPT-6-sol, three Claude models and three Tulu 3 8B checkpoints wrote from the same titles and abstracts. Its four questions: did the construction get more frequent (yes, 0.134 to 0.993 per 1,000 words of body text, Table 5); do readers find some uses annoying (two author-annotators labeled 1,000 sampled uses blind to source, Table 8 and 9); what separates annoying uses (the rejected alternative is presented less favorably, Tables 13 and 14); and does post-training reward it (odds ratios in four preference datasets, Table 23, and Skywork-Reward-V2 minimal pairs, Table 24).

For anyone producing preference data, the claim that matters is the last: raters of preference pairs favor responses with "rather than", people more consistently than LLM judges (HelpSteer2 odds ratio 1.37, HelpSteer3 1.24, UltraFeedback 0.99, Tulu 3 1.09). The authors conjecture that a pairwise judgment credits a disavowal in one response and cannot see its cost across a whole text.

Do the paper's numbers hold when recounted from public data?

Mostly yes, with one qualification on the preference claim. We ran three checks: an independent count in arXiv abstracts, a recomputation of the agreement table from the released label file, and the authors' own preference-pair method applied to UltraFeedback (to replicate) and to Anthropic HH-RLHF (new data).

Paper numbers from arXiv:2610.10092v1. BLOMEGA rows computed on 8 October 2026: abstracts from export.arxiv.org/api/query; labels from data/annotation/human_labels.jsonl and data/annotation_b3/human_labels.jsonl at olzama/rather-than-public main; preference pairs from the Hugging Face files named in the check section. Regexes are the authors' antithesis_patterns.py.

The rise replicates in abstracts, family member by family member

Our sample is the last 250 cs.CL submissions (including cross-lists) of each month of 2019 and the last 280 of each month from January to September 2026, by submittedDate; the API returned 203 and 237 for January and February 2019, so the totals are 2,940 and 2,520 abstracts. "Rather than" occurs 51 times in 457,211 words in 2019 and 446 times in 496,558 words in 2026: 0.112 per 1,000 words (bootstrap 95% interval 0.081 to 0.143) against 0.898 (0.816 to 0.986). The ratio is 8.0, close to the paper's 7.4 in body text, from a different text type and a different paper sample.

The rest of the family moves the way Table 5 says it does in bodies. "X, not Y" rises from 0.026 to 0.173 (6.7x) and "not X but Y" from 0.035 to 0.133 (3.8x). "Instead of" falls from 0.153 to 0.103 and "as opposed to" from 7 uses to none. The emphatic "not only/just/merely X but Y" is flat at 0.206 and 0.189, where the paper reports a modest body-text rise from 0.079 to 0.119. Abstracts get longer too, 155.5 to 197.0 words on average, which does not affect a per-word rate.

Month by month, the 2026 rate runs 0.59 (January), 0.50, 0.77, 0.91, 1.21, 1.24 (June), 1.11, 1.10 and 0.60 (September), on 26 to 70 uses per month. No 2026 month falls below four times the pooled 2019 figure; the 2019 months range from 0 to 0.20. We read the September dip as noise until more months exist: 36 uses in 280 abstracts, and our sample takes the last submissions of each month, which may differ from the month as a whole.

How can a 1.08 odds ratio per pair turn into a 7x habit?

The argument is a compounding one. A rater comparing two single responses sees one "rather than" in one of them and gives it a small, local credit: the paper's human-rated HelpSteer2 and HelpSteer3 pairs favor it at odds ratios of 1.37 and 1.24 with log length held fixed. Four Skywork-Reward-V2 models then score a sentence higher with its "rather than Y" clause than without it, by 0.83 to 2.46 reward units, and still by 0.26 to 1.13 against a same-length non-contrastive clause (Table 24, 424 items). Optimized against such a signal, a model writes papers where the construction recurs: arXiv 2026 papers average 8.0 uses (median 6), against 0.65 in ACL 2019.

The cost appears only at that last step. Neither annotator finds more than about one 2026 use in nine annoying, but with eight uses per paper the chance of meeting at least one is 51.1% for A1 and 42.1% for A2 (Table 10). No preference pair ever contains eight uses side by side, so no rater can price that.

Two pieces of the paper's own data push against a simple version of this story, and the authors report both. The three Tulu 3 8B checkpoints, which differ only by post-training stage, write the construction at 0.44 per 1,000 words or less, and the rate is no higher after DPO than after SFT (Table 7). And UltraFeedback's GPT-4 judge is indifferent to it (0.99). Our HH-RLHF result adds a third: in Anthropic's 2022 human preference data, the effect is small and appears to ride on length.

Does Anthropic's HH-RLHF show the same rater preference? Barely, and not at equal length

HH-RLHF is the human-labeled preference set behind Anthropic's 2022 assistant work, and the paper did not test it. We ran the authors' method on all four training subsets: for each pair, whether each pattern occurs in the final assistant turn of the chosen and the rejected response, then a conditional logistic model of which response was chosen on the difference in pattern presence and in log word count, fit on both orderings without an intercept (the code in counting/preference_pairs.py, reimplemented). We first ran it on UltraFeedback binarized and got the paper's numbers to two decimals for all five patterns: 0.99, 0.92, 0.94, 1.24 and 1.39.

Pairs with an empty or identical final turn dropped (924 of 160,800 HH-RLHF rows). Length-matched: word counts within 10% of each other (16,643 HH-RLHF pairs, 7,752 UltraFeedback pairs). Sign-test p for HH-RLHF all, unmatched: 0.0006; matched: 1.0.

Pooled, HH-RLHF raters lean toward the response with "rather than": 1,668 pairs where only the chosen response has it against 1,474 where only the rejected one does (sign test p = 0.0006), and an odds ratio of 1.078 with log length held fixed, with a lower bound of 1.005. That points the same way as the paper's HelpSteer results at a quarter to a third of their effect size on the log-odds scale (0.075 against 0.315 and 0.215). Two subsets carry it: harmless-base at 1.27, where a chosen reply often redirects the user ("Rather than fighting people directly with your fists, I think your aim should be to avoid a fight", from our sample of chosen texts), and helpful-base at 1.16. Helpful-online, the subset collected with Anthropic's iterated online models, sits at 0.999.

Matching on length removes it. In the 16,643 pairs whose two responses are within 10% of each other in word count, the construction appears only in the chosen response 203 times and only in the rejected 202 times. A log-length covariate assumes length acts on a straight line; the matched comparison assumes nothing, and it shows nothing. UltraFeedback's matched pairs even lean the other way, 130 against 160 (p = 0.09). Our reading, a judgement: in human pairwise data from 2022, "rather than" is mostly a passenger on longer, more elaborated answers, which raters are known to prefer. The paper's HelpSteer effects may survive the same test; its released code does not run one, and we did not download HelpSteer for this note.

Why does the "half of papers annoy a reader" figure drop to one in seven with two readers?

The released human_labels.jsonl reproduces Table 9 exactly: on 989 pool items garbled by neither annotator, A1 labels 91 annoying, A2 66, both 19, giving kappa 0.178, Gwet's AC1 0.859 and positive agreement 0.242. Only 29% of A2's annoying items are also A1's. The AC1 is high because 241 ACL 2019 items, a quarter of the pool, are rated not annoying by both readers; restricted to the 386 random arXiv 2026 items, kappa is 0.147 and AC1 0.778. On the items drawn from first and latest arXiv versions of revised papers, kappa is 0.012 and 0.066, which is chance. The one stratum with real agreement is the 100 items from papers with 35 or more uses, at kappa 0.432 and 6 shared annoying items.

The paper's encounter estimate averages 1 - (1 - p)^k over the per-paper counts k of all 1,821 papers. Those counts are not in the release (the repo ships instance files for ACL 2019 and the GPT papers, not for arXiv 2026), so we fit a negative binomial to the two moments Table 10 does report, mean 8.0 uses and 94% of papers with at least one. The fit has median 6, as the paper reports, and it reproduces Table 10 within 0.3 points: 51.4% for A1, 42.4% for A2 and 63.7% for either reader.

The paper uses "either" as its union label. Swap in the intersection, a use both readers flag (13 of 648 random arXiv 2026 items, p = 0.020), and the same model gives 14.2%, about one 2026 paper in seven. Both numbers are correct; they answer different questions. "Half of papers annoy a given reader" is a statement about each reader's taste. Whether there is a shared, labelable property called annoying antithesis is a different question, and at kappa 0.15 to 0.18 the released data says it is mostly not shared. The paper's own straw-man and stance analyses, which find a common thread in how the rejected alternative is treated, are the better candidates for a label someone else could reproduce.

What should a preference-data or annotation team change?

Audit chosen versus rejected for stylistic markers, matched on length. The paper's odds ratios and our HH-RLHF pooled number both use a log-length covariate. The matched comparison moved HH-RLHF from 1.08 to an exact tie. Before treating any phrase as a rater bias, run both views; it costs one extra line, and we do the same on per-annotator length bias.

Give raters a document-level view for writing tasks. The conjectured failure is that pairwise judgment of single responses cannot see repetition. For long-form writing preferences, a pass where a rater sees the whole generated document and counts the tics is a cheap structural fix. In our judgement it addresses the mechanism better than a banned-phrase list, which the paper predicts would push the same move into "X, not Y" (Claude and GPT-6-sol revisions already raise that form, Tables 6 and 7).

Do not train on "annoying" as a single gold label at kappa 0.18. If a team wants a style reward from human annoyance labels, treat each rater's labels as a separate signal or model rater identity; a majority or union label at this agreement level encodes whose taste won. The high-count stratum (kappa 0.43) suggests agreement rises when the tic is dense, so sampling items from saturated documents gives a cleaner training signal than random sentences.

Report the aggregation rule next to any prevalence headline. 63.7% under "either" and 14.2% under "both" come from the same labels. A data card that states only one invites a reader to assume the other. The same issue appears in HH-RLHF's own label noise.

Check it yourself

Everything runs on a laptop in under ten minutes: 5,460 abstracts (21 API calls), two label files of 216 KB and 69 KB, 75 MB of HH-RLHF and 226 MB of UltraFeedback. Python 3 with numpy, pandas, pyarrow and scipy.

# 1. arXiv abstracts: one call per month, last N submissions of the month (https only)
curl -s "https://export.arxiv.org/api/query?search_query=cat:cs.CL+AND+submittedDate:[202605010000+TO+202605312359]&sortBy=submittedDate&sortOrder=descending&max_results=280"
#    2019: N=250 for each month 201901..201912; 2026: N=280 for 202601..202609; 4 s pause between calls
#    count re.findall(r"\brather than\b", summary, re.I); words = len(re.findall(r"\w+", summary))
# -> 2019: 2,940 abstracts, 51 / 457,211 words = 0.112 per 1,000
# -> 2026: 2,520 abstracts, 446 / 496,558 words = 0.898 per 1,000

# 2. agreement, from the authors' labels (fields: item_id, annotator, label)
B=https://raw.githubusercontent.com/olzama/rather-than-public/main
curl -sO $B/data/annotation/human_labels.jsonl
curl -sO $B/data/annotation/items_provenance.jsonl     # fields: item_id, corpus, doc_id, pattern_id
#    keep A1 and A2, drop item_id starting "rep__" (hidden repeats) and any item either marked "garbled"
#    binary: annoying vs other -> n 989, A1 91, A2 66, both 19, kappa 0.178, AC1 0.859, PA 0.242
#    corpus == "arxiv2026" only -> n 386, kappa 0.147

# 3. preference pairs (patterns from data_collection/antithesis_patterns.py)
curl -sLO https://huggingface.co/datasets/Anthropic/hh-rlhf/resolve/main/harmless-base/train.jsonl.gz
#    likewise helpful-base, helpful-online, helpful-rejection-sampled
#    final assistant turn = text after the last "\n\nAssistant:" in "chosen" and "rejected"
curl -sLO https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized/resolve/main/data/train_prefs-00000-of-00001.parquet
#    final turn = last message with role "assistant" in the chosen / rejected lists
#    model: logit P(first is chosen) ~ d_presence + d_log1p(words), both orderings, no intercept
# -> UltraFeedback OR 0.994; HH-RLHF all OR 1.078 (1.005-1.156)
# -> length-matched (|w_c - w_r| <= 0.1 * max): HH-RLHF 203 only-chosen vs 202 only-rejected

# 4. encounter model: k ~ NegBin(mean 8.0, P(k=0)=0.06) gives r = 1.545, median 6
#    mean over k of 1-(1-p)^k: p=0.115 -> 51.4%, 0.083 -> 42.4%, 0.179 -> 63.7%, 13/648 -> 14.2%

Repo state checked on 8 October 2026: github.com/olzama/rather-than-public returns HTTP 200, 308 tree entries, MIT licence text (the GitHub API labels it "Other"), last push 6 October 2026 20:42 UTC. The arXiv 2026 paper texts and their per-paper instance counts are not in it, so the 0.993 body-text rate and Table 10 cannot be recomputed from the release; the ACL 2019 sections file (39.6 MB) and the GPT-paper instance files are.

What would prove this reading wrong?

Our reading has two parts: the frequency rise is real and large, and the per-pair rater preference behind it is small enough that length explains much of it in at least one major human dataset.

First, if the HelpSteer2 and HelpSteer3 pairs, restricted to responses within 10% of each other in word count, still show the construction only in the chosen response at least 1.3 times as often as only in the rejected one, then human raters do reward it beyond length and our HH-RLHF null is specific to 2022-era data. If they tie, as HH-RLHF does, the length account covers the human-rated evidence too.

Second, a dated prediction: in the same arXiv sampling (last 280 cs.CL submissions per month), the "rather than" rate in abstracts for January to March 2027 will stay above 0.5 per 1,000 words, at least four times the 2019 rate, even as style guides and reviewer complaints spread. If it falls below 0.3 by 31 March 2027, authors are editing the habit out faster than models put it in, and the paper's revision analysis (no drop between arXiv versions, Appendix I) no longer describes current practice.

Sources

1. Zamaraeva, O., Gude, A., Santos-Ríos, R., Gómez-Rodríguez, C. (Universidade da Coruña, CITIC). I would rather quit NLP than read another paper like this: The rise of antithesis in NLP papers. arXiv:2610.10092v1, 7 October 2026. Tables 1, 5, 7, 8, 9, 10, 23, 24; Sections 4.5, 5.2, 5.4.
2. Zamaraeva, O. et al. olzama/rather-than-public, code and data (MIT), created 6 October 2026. Accessed 8 October 2026.
3. Anthropic. Anthropic/hh-rlhf, Hugging Face, train splits of harmless-base, helpful-base, helpful-online, helpful-rejection-sampled. Accessed 8 October 2026.
4. Hugging Face H4. HuggingFaceH4/ultrafeedback_binarized, train_prefs split, derived from UltraFeedback (Cui et al., 2024). Accessed 8 October 2026.
5. arXiv. arXiv export API, queried 8 October 2026 for cs.CL, 2019 and January to September 2026.
6. Wang, Z. et al. HelpSteer2 and HelpSteer3-Preference (NVIDIA), 2025, as cited and analysed in arXiv:2610.10092.
7. Liu, C. et al. Skywork-Reward-V2, 2025, as scored in arXiv:2610.10092 Table 24.
8. BLOMEGA. HH-RLHF preference label noise.
9. BLOMEGA. Per-annotator length bias in MultiPref.

FAQ

Has "rather than" really become more common in NLP writing since LLMs?

Yes. arXiv:2610.10092 reports 0.134 per 1,000 words in ACL 2019 paper bodies against 0.993 in 1,821 ACL-style arXiv papers from 2026. In an independent sample of 5,460 arXiv cs.CL abstracts pulled by BLOMEGA on 8 October 2026, the rate rises from 0.112 per 1,000 words in 2019 to 0.898 in January to September 2026, a factor of 8.0, and the share of abstracts using it from 1.7% to 15.75%. "Instead of" and "as opposed to" fall over the same period.

How well do annotators agree on which "rather than" uses are annoying?

Poorly. Recomputed by BLOMEGA from the authors' released human_labels.jsonl, on 989 pool items Cohen's kappa for annoying versus other is 0.178 (bootstrap 95% interval 0.09 to 0.275), positive agreement 0.24, and only 19 items are annoying to both readers out of 91 and 66. Gwet's AC1 of 0.86 looks high because 241 ACL 2019 items are rated not annoying by both. On the random arXiv 2026 items alone kappa is 0.147.

Do preference datasets reward responses that use "rather than"?

Weakly, and not everywhere. The paper reports length-controlled odds ratios of 1.37 and 1.24 in the human-rated HelpSteer2 and HelpSteer3, and 0.99 in UltraFeedback, which BLOMEGA reproduces exactly (0.994). In Anthropic HH-RLHF, which the paper did not test, the pooled odds ratio over 159,876 pairs is 1.08 (95% interval 1.005 to 1.156), and in the 16,643 pairs whose lengths differ by at most 10% the construction appears only in the chosen response 203 times and only in the rejected 202 times.