Nine residents, eight volume datasets: the model-in-the-loop annotation speedup is 10.4x pooled, not 14.4x

What changed, and when

On 15 September 2026 Abhilaksh Singh Reen, Kushal Borkar and Ritvik Mahapatra posted IMVS: Interactive Medical Volume Segmentation with Test-Time Adaptation (arXiv:2609.16775v1), accepted at the HAIC workshop at MICCAI 2026. It is a submitted manuscript, before peer review.

The contribution is a loop, not a segmentation model. A lightweight 2D Slice Mask Adapter is fine-tuned online from the annotator's own scribbles during the session. A frozen Volume Mask Tracker, a ViT-B pretrained on YouTube-VOS 2019 and fine-tuned as a mask propagator, carries the corrected mask across adjacent slices. A soft teacher-student alignment limits forgetting. Existing 3D and video-propagation tools (nnInteractive, MedSAM2, PRISM) use a propagator that never adapts to the volume in front of it, so it drifts on low-contrast and pathological structures and has to be re-prompted.

What makes the paper worth reading is the measurement protocol rather than the architecture. Times come from a user study with nine residents, all evaluating every method under one acceptance rule: advance when the segmentation is qualitatively satisfactory. The accepted masks define the automatic stopping threshold used everywhere else in the paper, DSC 0.885, NSD 0.894, HD95 4 mm. Inter-annotator agreement was a mean pairwise DSC of 0.89 plus or minus 0.04, and acceptance thresholds were consistent across raters at 0.885 plus or minus 0.03. Training used only BraTS, LiTS and MSD Pancreas splits; the other five datasets are zero-shot.

The evidence table

Annotation time in minutes per volume, with the number of user interactions in parentheses. Manual baselines have no discrete interactions. "Manual with copy" is the proficient copy-and-edit workflow and is the fair manual reference. The final column is ours: minutes for manual-with-copy divided by minutes for IMVS.

Per-dataset minutes and interaction counts from Table 1 of arXiv:2609.16775v1. The ratio column, the summed row and the pooled ratio are computed by BLOMEGA from those values.

The abstract's two speedups reproduce exactly as medians. The eight per-dataset ratios against naive manual annotation are 22.51, 22.16, 22.74, 18.15, 22.69, 9.00, 17.63 and 23.02; their median is 22.34, and the paper says 22.3x. Against manual-with-copy the ratios are 14.49, 14.10, 14.86, 11.49, 14.68, 5.78, 11.43 and 14.80; their median is 14.30, and the paper says 14.4x. The arithmetic mean is 12.7x and the pooled ratio, total minutes over total minutes, is 10.4x.

Why pooled and median diverge: MSD HepaticVessel alone is 61.85 of the 240.89 manual-with-copy minutes, a quarter of the total work, and it is the dataset where the method helps least. A median gives it one vote out of eight. A procurement decision does not. Both numbers are defensible; only one of them answers "how many annotator hours will this save me".

The gain comes from adapting the small model, and it stops where the anatomy stops being continuous

The ablation isolates it. Without online adaptation of the slice adapter, the loop needs 18 interactions per volume at DSC 0.86 and the tracker holds for 3 slices before a correction is needed. With adaptation, 4 interactions at DSC 0.92 and a 12-slice span. Adapting the propagator as well gets the same 0.92 Dice at 8 interactions, but costs 20.7 GB of VRAM against 3.23 GB and 1857 ms of GPU time against 129 ms, because the propagator drifts temporally once it is allowed to move.

The propagation length K = 17 came from a sweep over 5 to 25. At K = 5 the loop needs 7.2 interactions; at K = 17, 4.1. Push to K = 25 and the propagation failure rate rises from 29% to 55%. On tortuous hepatic vessels the tracker holds for only 4 to 6 slices against up to 17 on well-defined organs, which is the whole of the 5.8x row in the table above. Every method degrades there: the slice-wise baselines need 121 to 126 interactions on that dataset.

Interaction count at matched quality is the cleaner comparison, because it removes the confound that a scribble carries more information than a click. To reach DSC 0.90, IMVS needs 7.1 interactions on average against 10.1 for MedSAM2 and 13.1 for slice-wise iSegFormer, and it is lowest on every one of the seven datasets in that table.

What it means for anyone costing a medical annotation contract

A foundation model may already be enough for the easy half of your corpus. MedSAM2 is faster than IMVS on CHAOS-CT, CHAOS-MRI and AMOS-CT, and the paper reports the per-dataset numbers rather than averaging them away. A paired Wilcoxon signed-rank test per volume against MedSAM2 gives p = 0.02 over the four challenging datasets and p = 0.26 over the well-delineated ones. Read plainly: on healthy organ contours, there is no measured advantage here, and an off-the-shelf tool with no session state is the cheaper operational choice.

Quote the hard structures separately. Pooled across these eight datasets the manual workload is 240.89 minutes and the assisted workload 23.11 minutes, but 46% of the remaining assisted time is one dataset. Any pricing model that assumes a flat multiplier over a mixed corpus will underquote the vessels and overquote the livers. The same logic applies to the rare-event case we covered in prevalence effects in rare-event labeling: the average across a corpus is not the unit anyone actually works in.

Nine residents with a shared acceptance rule is a stronger protocol than most. The acceptance threshold was derived from the annotators rather than asserted, and the agreement between them is reported (mean pairwise DSC 0.89 plus or minus 0.04). That is rare in interactive-segmentation papers and it is what makes the minute counts comparable across methods at all. Our judgement: nine annotators from one population is still a small study, and the paper does not report how the nine were distributed across the 8 datasets and 9 methods, so treat the per-cell minutes as one measurement rather than a distribution.

Freeze what propagates, adapt what corrects. The result generalises past radiology. In any propagate-and-correct annotation loop (video object tracking, document layout, long-form audio), adapting the small corrector online is worth 4.5 times fewer interactions here, while adapting the propagator buys nothing and costs 6x the memory. That is a cheap thing to test on your own pipeline.

Check it yourself

Table 1 of the arXiv HTML has every minute count. The recomputation below is the whole of our finding.

open https://arxiv.org/html/2609.16775v1     # Tables 1, 2, 3

python3 - <<'PY'
import statistics as st
ds   = ["CHAOS-CT","CHAOS-MRI","AMOS-CT","Prostate","LiTS","HepVessel","Pancreas","BraTS"]
man  = [44.79, 21.5, 65.5, 8.53, 73.53, 96.3, 8.11, 55.26]   # naive manual
copy = [28.83, 13.68, 42.79, 5.4, 47.56, 61.85, 5.26, 35.52] # copy-and-edit manual
imvs = [1.99, 0.97, 2.88, 0.47, 3.24, 10.7, 0.46, 2.4]
rc = [c/i for c,i in zip(copy, imvs)]
rm = [m/i for m,i in zip(man,  imvs)]
print("vs copy  median %.2f  mean %.2f  pooled %.2f" % (st.median(rc), st.mean(rc), sum(copy)/sum(imvs)))
print("vs naive median %.2f  mean %.2f  pooled %.2f" % (st.median(rm), st.mean(rm), sum(man)/sum(imvs)))
print("worst dataset:", ds[rc.index(min(rc))], "%.2fx" % min(rc))
print("HepVessel share of manual-with-copy time: %.0f%%" % (100*copy[5]/sum(copy)))
PY
# vs copy  median 14.30  mean 12.70  pooled 10.42
# vs naive median 22.34  mean 19.74  pooled 16.16
# worst dataset: HepVessel 5.78x
# HepVessel share of manual-with-copy time: 26%

Source code and the demo video are linked from the paper at github.com/AbhilakshSinghReen/imvs; the repository states that weights will be released after publication and that the code is an anonymised review release, so the loop cannot be re-run end to end yet. The eight datasets are public (CHAOS, AMOS, MSD Prostate / HepaticVessel / Pancreas, LiTS, BraTS), so the stopping criterion, DSC 0.885, can be applied to any other tool on the same volumes.

What would prove this wrong

Our claim is that the reported multiplier is a median that overstates the workload saving on a mixed corpus, and that the advantage is confined to structures where slices stay continuous. It would be wrong if the authors report a pooled time comparison that lands near 14x, which would mean Table 1 does not represent the volumes the study timed, or if a replication on a corpus dominated by vessel-like structures reproduced a ratio above 10x.

A dated prediction: the peer-reviewed MICCAI 2026 version of this paper, due in the Lecture Notes in Computer Science satellite-events volume, will report the same per-dataset Table 1 and will not add a pooled or total-time figure. If the camera-ready adds a pooled comparison above 12x against manual-with-copy, this reading is wrong. Deadline: 30 June 2027.

Sources

1. Reen, A. S., Borkar, K., Mahapatra, R. IMVS: Interactive Medical Volume Segmentation with Test-Time Adaptation - A New Method for Annotating Radiology Datasets. arXiv:2609.16775v1, 15 September 2026. Tables 1 to 3, Sections 3.5 to 4.4; HTML version. Accepted at the HAIC workshop, MICCAI 2026, submitted manuscript prior to peer review.
2. Random Researcher 0611. IMVS Demo on AMOS CT. YouTube, 11 July 2026. The authors' anonymised review account, linked from the paper's repository.
3. github.com/AbhilakshSinghReen/imvs, the source release referenced in the paper's abstract.
4. BLOMEGA. Rare-event labeling and the prevalence effect.
5. BLOMEGA. Data annotation research: the latest.