A reversed CLIP prompt that flags its own wrong guesses beats five active-learning baselines

What changed, and when

On 4 February 2026, Qian-Wei Wang, Yaguang Song and Shu-Tao Xia, of Tsinghua Shenzhen International Graduate School and the Institute of Perceptual Intelligence at Peng Cheng Laboratory, posted Explicit Uncertainty Modeling for Active CLIP Adaptation with Dual Prompt Tuning (arXiv:2602.04340). The setting is active learning for adapting a frozen CLIP model to a new classification task with prompt tuning, where the annotation budget is small on purpose, 1%, 2% or 5% of an unlabeled pool, and the question is which images in that pool are worth a human label.

Most acquisition functions for this problem estimate uncertainty indirectly, from prediction entropy or from where a sample sits in embedding space. This paper's acquisition signal is more direct: a second prompt, trained with a loss built to invert on purpose, learns to output the probability that the first prompt's prediction is correct. The paper calls it a clean probability, and the lowest-scoring samples per class are the ones queried for annotation.

The evidence table

All results below use a CLIP ViT-B/16 backbone, a 1% annotation budget, and average top-1 accuracy over six standard fine-grained classification datasets: Caltech101, DTD, EuroSAT, FGVC-Aircraft, Flowers102 and UCF101. CEC is the strongest prior active-learning baseline the paper reports at this budget; BADGE (gradient-embedding active learning) and Entropy (uncertainty from prediction entropy) are older, widely used baselines. The paper's own method is listed per fine-tuning paradigm.

Source: arXiv:2602.04340, Tables 1 to 3, average top-1 accuracy at a 1% annotation budget, CLIP ViT-B/16.

The gain over CEC is consistent across paradigms (2.4, 1.8 and 2.0 points) and holds on a larger backbone: swapping ViT-B/16 for ViT-L/14 with CoOp keeps CEC at 70.0 and lifts this paper's method to 71.4, a 1.4-point gain (arXiv:2602.04340, Table 4). It does not hold evenly across datasets. On EuroSAT, the gap over CEC is 1.7 points; on DTD, a harder and more visually ambiguous texture dataset, it is 0.2 points.

Source: arXiv:2602.04340, Table 1, per-dataset breakdown.

How a prompt trained to be wrong becomes an uncertainty score

CLIP's text branch normally gets one learnable prompt per class. This method gives each class two: a positive prompt, trained the ordinary way with cross-entropy over the predicted class (loss L1, with 16 learnable context tokens), and a negative prompt, trained on a loss built to invert on purpose. For a sample with pseudo-label ŷ, that second loss is L2 = −log p_clean(ŷ) + log(1 − p_clean(ỹ)), where ỹ is a randomly sampled wrong label standing in for label noise. The combined objective is L1 plus a weighted L2.

p_clean itself is a two-way softmax over the cosine similarity between a sample's image embedding and its positive versus negative text embeddings, scaled by a temperature. In plain terms, it is CLIP's own estimate of how likely its predicted label is correct, produced by a prompt trained specifically to notice when the model is probably wrong rather than trained to be confident.

Selection then runs per round: within a budget of B samples, the lowest-p_clean samples are queried for human annotation, with a per-class quota of ⌊B/C⌋ so no single class dominates the batch, and any remainder going to the globally most uncertain samples. At the other end, the highest-p_clean samples per class are kept as confident pseudo-labels rather than queried, feeding the next round's training without spending budget on them. The paper runs this for up to six rounds, each adding 1% of the pool, retraining the model from scratch each round.

What it means for anyone budgeting annotation on a CLIP-style model

An uncertainty score does not have to come from entropy or embeddings. Entropy and BADGE, the two older baselines in the table, both score worse than CEC and worse again than this paper's method at every budget reported. The common thread in what beats them, CEC and this paper, is that both learn a dedicated signal for "is this prediction trustworthy" rather than reading uncertainty off the main classifier's own confidence. Training a second prompt specifically to be wrong when the first one is wrong is one way to build that signal; it is not obviously the only way, and the paper does not test whether a simpler calibration method would close most of the gap.

The gain is real but not large, and not even. A 1.8 to 2.4 point average accuracy gain over the strongest baseline, at a fixed 1% budget, is a genuine improvement, not a rounding difference, and it survives moving to a bigger backbone (1.4 points on ViT-L/14). But it is not a uniform multiplier: on EuroSAT the gain over CEC is 1.7 points, on DTD it is 0.2 points. Anyone deciding whether the extra complexity of a second prompt is worth it should expect a result closer to DTD than to EuroSAT on a harder, more visually ambiguous task.

This is an academic result, not a vendor benchmark, and the two are not interchangeable. Lightly, Voxel51's FiftyOne and Encord sit in the same broad space of label-efficiency tooling but occupy different slots: an Ultralytics comparison post dated 21 July 2026 describes Lightly's role as "embedding-based selection of which unlabeled data to label," FiftyOne as a "curation layer, not a labeling workforce or UI," and Encord as providing "automated annotation across image, video and medical formats." None of those vendor pages publish an accuracy-versus-budget table like this paper's; a search for an independent 2026 benchmark comparing active learning or data selection across the three turned up only vendor-authored comparison blogs, which is a different kind of evidence than a reported, numbered ablation against named baselines.

Our judgement: the benchmark is six moderate-size, mostly fine-grained classification datasets, not a large-scale benchmark like full ImageNet, and the paper reports results from a single NVIDIA RTX 3090 GPU with no stated error bars or seed count in the tables we could extract, and no reported training-time or compute overhead for running two prompts instead of one. A reversed training signal recovering a model's own blind spots is a real idea worth tracking, but a 1.8 to 2.4 point gain on six small datasets is not yet evidence that it changes labeling economics at production scale.

Check it yourself

The paper and its tables are public. The comparison that matters is CEC against this paper's method at the same budget and backbone, not either number against random sampling, which both already beat by a wide margin.

open https://arxiv.org/abs/2602.04340          # abstract, authors, submission date
open https://arxiv.org/html/2602.04340         # full text: Tables 1 to 4, Section 3 (dual prompt method)

python3 - <<'PY'
# recompute the headline gaps directly from the paper's reported numbers
settings = {
    "CoOp, ViT-B/16":  (60.2, 62.6),
    "VPT, ViT-B/16":   (58.1, 59.9),
    "MaPLe, ViT-B/16": (59.2, 61.2),
    "CoOp, ViT-L/14":  (70.0, 71.4),
}
for name, (cec, ours) in settings.items():
    print("%-16s CEC %.1f  ours %.1f  gap %+.1f" % (name, cec, ours, ours - cec))

euro_cec, euro_ours = 82.8, 84.5
dtd_cec, dtd_ours = 47.9, 48.1
print("EuroSAT gap: %+.1f, DTD gap: %+.1f" % (euro_ours - euro_cec, dtd_ours - dtd_cec))
PY
# CoOp, ViT-B/16   CEC 60.2  ours 62.6  gap +2.4
# VPT, ViT-B/16    CEC 58.1  ours 59.9  gap +1.8
# MaPLe, ViT-B/16  CEC 59.2  ours 61.2  gap +2.0
# CoOp, ViT-L/14   CEC 70.0  ours 71.4  gap +1.4
# EuroSAT gap: +1.7, DTD gap: +0.2

Every gap above is a plain subtraction of the paper's own reported means; none of them carries a significance test in the tables we extracted, so treat the ranking as the paper's finding, not as a statistically tested claim.

What would prove this wrong

The six datasets, the budgets (1%, 2%, 5%), and the choice of CEC as the strongest comparison baseline are all the paper's own. A method tuned and evaluated by its own authors on datasets they selected can look stronger than it would on a benchmark chosen by someone else, without anyone having manipulated anything.

A dated prediction: by 31 December 2027, if an independent group runs this dual-prompt acquisition method, or a close reimplementation, on a large-scale benchmark outside these six datasets (full ImageNet-1K classification, for instance, or a long-tailed or medical-imaging set) and the gap over the strongest active-learning baseline available at that budget falls below 1 point or reverses, the generalizable version of this claim is wrong, whatever these six datasets show. If instead an independent replication holds a comparable 1 to 2 point gap at low budgets on a dataset the original authors did not choose, that is the evidence this result currently lacks.

Sources

1. Wang, Q-W., Song, Y., Xia, S-T. Explicit Uncertainty Modeling for Active CLIP Adaptation with Dual Prompt Tuning. arXiv:2602.04340, 4 February 2026. Tables 1 to 4, Section 3; HTML version.
2. Deans, M. Auto-labeling and active-learning tools compared. Ultralytics, 21 July 2026. Role descriptions for Lightly, Voxel51 FiftyOne and Encord.
3. Lightly. Looking for a Voxel51 alternative?. Lightly.ai, 2026.
4. BLOMEGA. Zero-label coreset selection edges out a labeled baseline on ImageNet at 10%.
5. BLOMEGA. Active learning beats full-dataset training on the hardest anatomical structures.