Abstract

Multi-cancer segmentation in computed tomography (CT) is fundamentally limited by the scarcity of tumor masks across different organs. We present Merlin Plus, the first large-scale CT dataset with radiologist-created tumor masks across 9 organs. Merlin Plus extends the Merlin dataset by adding 1,153 per-voxel tumor masks and longitudinal metadata. To create these tumor masks, we developed a report-based active-learning framework in which radiology reports identify tumor cases for annotation and support training of a tumor segmentation model. The model generates initial masks, which radiologists review and correct to produce the final masks, reducing annotation burden while maintaining high-quality annotations. Besides tumor masks, the longitudinal metadata in Merlin Plus enables temporal modeling of cancer progression. By directly addressing the major bottleneck of limited multi-cancer segmentation masks, Merlin Plus supports scalable multi-organ cancer detection, segmentation, and longitudinal analysis in CT. Dataset is available at: https://github.com/MrGiovanni/MerlinPlus

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/4063_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/MrGiovanni/MerlinPlus

Link to the Dataset(s)

https://github.com/MrGiovanni/MerlinPlus

BibTex

@InProceedings{BasPed_Merlin_MICCAI2026,
        author = { Bassi, Pedro R. A. S. AND Li, Wenxuan AND Płotka, Szymon AND Honjol, Ruby AND Prządo, Jakub AND Zhou, Xinze AND Wang, Kang AND Yang, Yang AND Jensen, Malte AND Chaudhari, Akshay S. AND Langlotz, Curtis P. AND Yuille, Alan L. AND Zhou, Zongwei},
        title = { { Merlin Plus: A Large-Scale, Multi-cancer, Image-Mask-Report Dataset } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16895},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    This paper introduces a large dataset of abdominal CT from multiple cancers with refined segmentation masks obtained via active learning under guidance of radiologists. The dataset adds to existing public CT datasets of tumors with new additional tumor types including spleen, adrenal glands, bladder, gallbladder, esophagus, stomach, duodenum, prostate, and uterus. It also includes longitudinal metadata enabling temporal modeling of cancer progression. Radiologist refined AI-generated masks in an active learning fashion obtains high quality masks with reduced human burden.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Large-scale CT (25,494) dataset with radiologist validated lesion-level masks spanning 9 organs
    • This dataset complements existing tumor datasets with additional organs and more detailed masks compared to previous datasets, eg. ULS.
    • The proposed active learning for segmentation from reports is novel and can be generalized to other applications
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • The quality of final refined masks is not discussed. For example, a limited study on reproducibility, and inter-observer variably could assist in gaining confidence about the quality of final masks.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (6) Strong Accept — must be accepted due to excellence

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Paper described release of a large dataset of CT and tumor mask pairs that can greatly assist the community in developing relevant AI models. To create reliable high quality mask, they introduce an active learning refinement process, making mask creation under often busy radiologist schedules feasible. This methodology alone is highly valuable tool for similar tasks and is justifiably submitted as another paper.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    All concerns were addressed.



Review #2

  • Please describe the contribution of the paper

    The paper presents Merlin Plus, a large-scale CT dataset of 25K scans with voxel-wise lesion masks covering nine tumor types mostly absent from existing public segmentation datasets. It addresses a bottleneck that publicly available annotated data is largely restricted to liver/kidney/pancreatic/lung tumors. The dataset is derived from the original Merlin cohort (Stanford Hospital) and is enriched with lesion masks and malignant/benign labels derived from cross-referencing ICD codes with radiology reports.

    The second contribution is a report-based active learning annotation pipeline that allegedly substantially reduces annotation time. The pipeline bootstraps from the R-Super report supervision framework (Bassi et al., MICCAI 2025) to address the “cold-start problem” when no existing segmentation model covers the target tumor types. Subsequent active learning cycles combine automatic mask refinement using report-extracted tumor attributes (location, size, count) with a “report-confidence score” (Dice between the soft model output and the report-refined binary mask) to identify and prioritize uncertain cases for radiologist correction.

    Finally, Merlin Plus provides longitudinal metadata, enabling temporal modeling of cancer progression. This is claimed by the authors as a contribution but no further details are given and the (longitudinal) CT images and IDC codes fully stem from the original Merlin dataset.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.Clinically relevant gap addressed with meaningful scale. The focus on 9 tumor types absent from public segmentation datasets addresses a genuine and important gap. Although there partially exist datasets with some of the annotated lesion subtypes, having ONE comprehensive dataset is a major advancement.

    2.Clever use of existing reports to initialize the annotation pipeline. The approach of leveraging existing radiology reports as weak supervision (i.e. starting from R-Super to produce reasonable initial proposals) rather than only requiring fully manual masks from scratch is elegant.

    3.Intuitive active learning criterion. The report-confidence score (Dice between the soft model output and the report-refined binary mask) is a cheap and sensible proxy for uncertainty: poor model predictions require large corrections in Step 3, resulting in low Dice and flagging the case for radiologist review.

    4.Longitudinal metadata and multi-modal annotations. The inclusion of anonymized CT dates and patient IDs enabling longitudinal tracking, combined with ICD-coded malignancy labels, makes Merlin Plus significantly richer than a standard segmentation benchmark. If the authors’ claims hold and this is indeed novel, this opens doors for temporal modeling and survival analysis that most datasets in this space do not support.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.(Critical) Baseline comparison is insufficient and potentially flawed The only public baseline is ULS23 (de Grauw et al., Medical Image Analysis 2025). ULS is a prompt-based universal lesion segmentation model trained with center-cropped patches, assuming a lesion is ALWAYS present in the patch center. Running it as a fully automatic sliding-window model would introduce a fundamental evaluation mismatch. The paper does not clarify whether the evaluation was prompt-based or automatic.

    • If prompt-based: comparison should include e.g. LesionLocator (Rokuss et al., CVPR 2025), a zero-shot universal tumor segmentation and tracking model or a general-purpose prompt-based segmentation model e.g. MedSAM2 (Ma et al., arXiv 2025).
    • If fully automatic: relevant baselines would include GF-Screen (Wu et al., ICLR 2026), a reinforcement-based pan-cancer segmentation model covering many lesion types, or general-purpose models such as VISTA3D (He et al., CVPR 2025) and SegVol (Du et al., NeurIPS 2024),VoxTell (arXiv 2025) or TotalSegmentator (Wassertal et. al.) for lesions.

    I acknowledge that running all of these may not be feasible, but at minimum the ULS evaluation mode should be clarified and at least one stronger baseline must be included for proper evaluation.

    2.(Critical) Evaluation metrics are insufficient The primary metric in Table 1 is detection F1-score (organ-level, volume-thresholded) with sensitivity and specificity. While detection performance is relevant and consistent with the R-Super protocol, DSC is the standard metric for segmentation and is almost entirely absent from the main results. DSC values are only buried in the table caption for a subset of models (“Average DSC: 17 for MedFormer, 25 for R-Super with masks, 10 for R-Super reports only”) without a proper per-organ breakdown. This raises two concerns:

    • The reported DSC values are very low. Averages of ~17–25 are far below what lesion segmentation methods typically achieve on established benchmarks (e.g., LiTS, KiTS, PanTS: DSC 40–70+). Values in this range warrant explicit discussion: are they driven by small/early-stage lesions, annotation quality, or model underfitting? The paper is silent on this.

    • A dataset paper validated through segmentation models must report segmentation quality. Detection F1 alone does not demonstrate the value of voxel-wise masks, which are the primary contribution. Without per-organ DSC, downstream users cannot meaningfully assess model quality. Finally, no standard deviation is reported anywhere in Table 1, making it difficult to assess result reliability across runs.

    3.Authors claim “For per-voxel annotations, we focused on malignant tumors for 9 organs missing masks in public datasets.” however several public datasets actually encompass large parts of these lesion types e.g. “Q. Chen et. al. Scaling Tumor Segmentation: Best Lessons from Real and Synthetic Data”.

    4.Figure 1 issues The caption for panel (a) describes an organ-wise distribution reporting “the number of CT scans with lesions and the number of small lesions (< 2 cm)”, but the actual panel (a) appears to show qualitative CT/mask examples of an adrenal lesion, i.e. caption and figure content do not match. Only one number per organ appears in the distribution plot, not the two numbers (# CTs and # small lesions) described in the caption. Panel (b) shows longitudinal time point distribution but is missing a Y-axis, making it impossible to assess what fraction of patients actually have more than one scan. Or how many patients had 3+ examinations. Is this longitudinal data novel or already existent in the original Merlin dataset?

    5.(Minor) Pipeline novelty is slightly overstated Steps 1–3 (including sub-steps 3A–3D: report attribute extraction, organ-constrained masking, ball convolution-based tumor localization, and volume-matched binarization) closely follow the R-Super framework (Bassi et al., MICCAI 2025), which is also the primary baseline in the paper. The genuine novelty seems to be the report-confidence score (Step 4) and the active learning loop around it. The authors should clearly delineate what is directly adapted from R-Super vs. what is new, rather than presenting the full pipeline as their contribution.

    6.No ablation on the active learning selection criterion The report-confidence score is presented as a key design choice, but there is no experiment showing it actually outperforms alternative selection strategies (e.g., random selection, entropy-based uncertainty). While I am generally convinced of the idea, even a simple annotation efficiency curve would substantially strengthen this claim.

    7.Missing annotation statistics Step 5 mentions “e.g., the 50 masks with the lowest score per tumor type” leaving the actual number ambiguous. The total number of active learning cycles is not reported. It is unclear how many masks were radiologist-refined vs. automatically accepted as high-quality. Were in the end all masks at least visually inspected?

    8.(clarity and organization) Manuscript appears incomplete Several issues suggest the paper was submitted prematurely: Keywords section still reads “First keyword · Second keyword · Another keyword” Section 2.1 contains an unfilled placeholder: “we create # multi-tumor masks in just 5 months”. Howerver, the total mask count is arguably the most important statistic for a tumor lesion dataset paper and is missing The sentence “R-Super teacher segmentation models to segment tumors” is syntactically broken The paper does not fill the allotted page limit; space could have been used for additional dataset statistics or a qualitative figure

  • Please rate the clarity and organization of this paper

    Poor

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (2) Reject — should be rejected, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The dataset motivation is generally strong, i.e. bootstrapping multi-cancer segmentation from existing radiology reports, an is well-motivated in the paper. The 9 tumor types covered are genuinely underserved in public benchmarks. However, the submission has issues that go beyond minor revisions: (1) The Dice score, the standard metric for segmentation, is almost entirely absent from the results table, and the reported average values (~17–25) are very low without any discussion or explanation; (2) the sole external baseline (ULS) is used in a potentially inappropriate evaluation mode, and the comparison set is narrow given recent work such as GF-Screen (ICLR 2026), LesionLocator (CVPR 2025), and VISTA3D (CVPR 2025), (3) the manuscript is incomplete, with a critical placeholder for the total mask count in Section 2.1 and multiple figure errors; (4) the novelty of the annotation pipeline is partially overstated given its close overlap with R-Super. For the rebuttal, I would particularly like to see: the total mask count (manually revised by radiologists), a clarification of the ULS evaluation setup, a clearer scoping of what is novel beyond R-Super, and a proper per-organ DSC table with discussion of the low absolute values. With these addressed, this work could be competitive for acceptance.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    After considering the rebuttal, I lean slightly toward weak reject but acknowledge that the authors have made a serious effort to address the core concerns raised by all three reviewers. The rebuttal provides the missing inter-rater reliability numbers (mean DSC ~52 across 9 tumor types), the previously-absent total mask count (1,153, all radiologist-revised), clarifies the ULS evaluation setup, adds 5 stronger baselines including an external evaluation, and offers a reasonable explanation for the low absolute DSC values via the inter-reader ceiling argument. The report-confidence score is also validated with concrete precision numbers.

    Residual concerns: per-organ DSC remains low for several organs (duodenum, prostate, stomach), the dataset scale (1,153 masks) is more modest than the “large-scale” (25K) framing suggests, the manuscript was clearly rushed (multiple placeholders, broken sentences, inconsistent counts), the single-site Stanford-only limitation, and substantial methodological overlap with the concurrent TR-Distillation submission. However, the dataset addresses a genuine, clinically important gap in public CT segmentation resources (9 underserved tumor types), and the report-based active learning framework is a legitimate methodological contribution. On balance, the community value of the released resource outweighs the remaining evidentiary gaps, provided the camera-ready incorporates the rebuttal’s added analyses and fixes the manuscript issues.



Review #3

  • Please describe the contribution of the paper

    The paper presents Merlin Plus, a dataset extension of the existing Merlin cohort (Stanford Hospital, 25,494 CT–report pairs), augmenting it with radiologist-validated voxel-wise tumor masks for 9 abdominal tumor types currently absent from public segmentation datasets: spleen, adrenal glands, bladder, gallbladder, esophagus, stomach, duodenum, prostate, and uterus. The contribution has three components:(1) The dataset itself i.e voxel-wise masks for ~8,000+ lesion occurrences across 9 organs (with per-organ counts ranging from 108 for esophagus to 2,085 for spleen), augmented with clinical metadata (age, sex, race, scanner type, anonymized IDs, scan dates), malignant/benign labels derived from ICD-10 cross-referenced with reports, and longitudinal structure (multiple time points per patient).(2) A report-based active learning pipeline that addresses the cold-start problem by initializing the annotation loop with R-Super-trained segmentation (no masks required), followed by a six-step cycle: R-Super inference -> report-constrained mask refinement (organ dilation, Ball-Convolution-based tumor localization, volume matching) -> report-confidence scoring (Dice between soft mask and report-refined mask) -> radiologist review of lowest-confidence cases → fine-tuning on accumulated masks and reports. Claimed ~5× speedup (30 min -> 5 min per scan).(3) Empirical demonstration that models trained on Merlin Plus (MedFormer, R-Super with masks+reports) substantially outperform both public multi-tumor models (ULS) and models trained on the original Merlin (report-only supervision) across the 9 tumor types, with +10% F1-Score improvement on average.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Major Strengths1.This is a genuinely impactful dataset for an under-served clinical problem. The motivation is compelling and accurate: public CT tumor segmentation datasets are dominated by liver, kidney, pancreas, colon, adrenal, and lung tumors, while cancers in the other organs collectively account for roughly half of global cancer mortality. Releasing voxel-wise masks for spleen, adrenal, bladder, gallbladder, esophagus, stomach, duodenum, prostate, and uterus tumors (particularly rare types like esophagus (n=108) and duodenum (n=249)) represents substantial community value. If released as promised, this fills a real gap that cannot be addressed by methodological improvements alone.2.Report-based active learning genuinely solves the cold-start problem. The core insight i.e that traditional active learning fails when no adequate seed model exists for the target tumor types, and that R-Super can bootstrap reasonable initial masks from report supervision alone, is well-motivated and non-obvious. The design choice to couple Ball-Convolution-based tumor localization with report-derived counts and diameters is a clever way to inject weak structural priors that reduce radiologist correction burden. This is a legitimately novel workflow contribution, not just “LLM + active learning.”3.The report-confidence score is a principled triage mechanism. Using the Dice between the soft (pre-refinement) mask and the report-refined mask as a proxy for “how much did the refinement have to do” is a sensible signal for flagging unreliable cases. Low Dice implies that the model’s confidence was misaligned with the report description, which is precisely the condition under which radiologist review adds the most value. The intuition that refinement magnitude correlates with unreliability is well-argued.4.Comprehensive multimodal integration. Beyond masks, the dataset ties together CT, radiology reports, ICD-9/ICD-10 codes (2.9M+ assigned codes, 16,553 unique values), patient demographics, scanner metadata, and longitudinal timestamps. This richness enables research beyond segmentation (longitudinal progression modeling, demographic fairness studies, scanner-invariance analysis) that narrower datasets cannot support.5.Evaluation against both public models and within-dataset baselines. Table 1 compares against ULS (the strongest public multi-tumor baseline), a classification baseline, and R-Super trained on reports alone. The within-dataset comparison (Merlin vs. Merlin Plus) isolates the value of the masks themselves, while the ULS comparison isolates the value versus the state of the public art. Both comparisons are meaningful and the gap (+10% F1 on average) is substantial.6.Small-lesion coverage. Explicit reporting of lesions under 2 cm is important for the early-detection use case the paper motivates. Early-stage tumors are precisely the small-diameter cases that most benefit from mask-level supervision.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    Major Weaknesses

    1. Four radiologists created/corrected the masks, yet there is no reported measure of inter-rater agreement (Dice, Hausdorff, surface distance), no double-reading subset, and no analysis of which tumor types showed high versus low annotator concordance. For small lesions (<2 cm) across rare tumor types (esophagus n=108, duodenum n=249), inter-rater variability is typically substantial and directly affects downstream model ceiling performance. Without this, the claim of “high-quality segmentations” is an assertion rather than a demonstration. This is a critical omission for a dataset paper. 2.A quantitative validation of the active learning pipeline is missing. The paper claims a 5× speedup (30 min -> 5 min per scan) but provides no data supporting this: no time-motion study, no breakdown by tumor type or lesion size, no comparison against a from-scratch annotation arm on a matched subset, no reporting of how many cases required full redrawing versus minor correction. Similarly, the number of active learning cycles run, the number of masks corrected at each cycle, and the per-cycle performance improvement are absent. For a paper whose second core contribution is the annotation pipeline, these metrics are essential. 3.The Dice scores are surprisingly low and under-discussed. Table 1’s caption reports average Dice of 17 (MedFormer), 25 (R-Super with reports+masks), and 10 (R-Super reports only). A DSC of 25 for a “state-of-the-art” segmentation model trained on “radiologist-validated” masks is remarkably poor and it implies that the masks are either too inconsistent to learn from, the tumors are genuinely too hard (likely in some organs), or both. This deserves substantial discussion and per-organ breakdown, but the number appears only in the caption and is never addressed in the text. A dataset paper claiming segmentation value cannot leave a DSC of 17–25 unexplained. 4.The report-confidence score has an unexamined failure mode. The scoring mechanism (Dice between soft and report-refined mask) flags cases where the refinement did substantial work. But this conflates two distinct failure modes: (a) the soft mask was wrong and the refinement correctly fixed it (flagged as unreliable, appropriately), and (b) the soft mask was right but the refinement mechanically moved it (e.g., due to imprecise organ mask dilation or a tumor slightly outside the reported slice), also flagged as unreliable, but incorrectly. There is no validation that radiologist-flagged masks actually had errors versus cases where the score fired spuriously. Without a precision/recall analysis of the flagging mechanism, we don’t know whether the radiologist time is well-spent. 5.The single-site bias is unaddressed. All CT scans are from Stanford Hospital. Scanner distribution, patient demographics, and imaging protocols are not compared to other healthcare systems. Models trained on Merlin Plus may exhibit substantial performance degradation on data from hospitals with different scanner vendors, contrast protocols, or patient populations. The paper does not evaluate on an external test set, does not report scanner or demographic confounders in performance, and does not discuss this limitation. For a dataset paper intended to serve the community, external validation is a reasonable expectation. 6.There is a methodological ambiguity in Step 3(D) volume matching. The report-derived volume estimation uses “sphere/ellipsoid volume approximations” from 1D diameters, with a 30% agreement tolerance. The choice of sphere vs. ellipsoid is not specified per tumor type, and the 30% threshold is unjustified; tumors are rarely spherical, and report diameters often capture only the long axis. This procedural step materially affects ground-truth quality and warrants more scrutiny. 7.Overlap with concurrent submission is not disclosed in-paper. The companion paper (TR-Distillation, 4063 concurrent submission) shares substantial machinery (R-Super, Ball Convolutions, the same LLM extraction pipeline, the same tumor-type focus). While the program chairs’ note handles the formal disclosure, from a reviewer’s standpoint the two papers read as two angles on the same research program. This paper would benefit from clearer articulation of what is novel in Merlin Plus beyond applying the (previously published) R-Super framework to a new cohort. 8.Ethics, consent, and release mechanism are completely absent. The paper claims Merlin Plus will be “publicly available” but does not describe the data use agreement, IRB approval, de-identification protocol beyond the phrase “anonymized patient IDs,” or the license under which masks will be released. Given that this is a Stanford Hospital dataset with demographic metadata including race, the paper must address these points. Stanford’s Merlin dataset itself has specific access controls that Merlin Plus inherits, and this should be explicit. This is particularly important because Section 2.2 describes ICD code integration from the same patient encounter — additional PHI that needs explicit disclosure of de-identification. 9.”ICD-10 codes identify malignancy” claim is clinically imprecise. ICD-10 codes record encounter-level diagnoses but do not reliably identify whether a specific imaging finding on a specific scan is malignant. A patient may have a historical cancer diagnosis (recorded in ICD-10) but the lesion visible on the current scan may be benign, treated, or unrelated. The paper states malignancy is “cross-checked with reports,” but the cross-checking procedure is not described. Mislabeling of benign lesions as malignant (or vice versa) at scale would materially compromise the dataset. 10.THere are many inconsistencies and errors in the text. The abstract states 25,494 scans; Section 2.3 states 25,528 scans. The keywords are placeholders (“First keyword · Second keyword · Another keyword”). Section 2.1 contains the unfilled token “create # multi-tumor masks.” Reference [13] cites arXiv:2601.22134 with year 2026 — this appears to be a non-existent/future arXiv ID. Footnote 2 refers to “these 8 tumor types” while the main text consistently claims 9 tumor types. These errors in a submission suggest rushed preparation and reduce confidence in the numerical precision throughout. 11.The ULS is a weak comparator that the paper over-attributes meaning to. ULS is a universal lesion segmentation model, it was designed for lesion agnosticism, not organ-specific cancer segmentation. Showing that a model trained on in-distribution tumor-specific data outperforms an out-of-distribution universal lesion model at F1 score is expected. The more meaningful comparison ( against organ-specific segmenters fine-tuned on ULS subsets or on related cohorts ) is not conducted.
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    The paper introduces voxel-wise masks, ICD-9/ICD-10 codes, patient demographics (age, sex, race), scanner metadata, and dated longitudinal imaging timestamps to an existing dataset. While the original Merlin cohort presumably has appropriate IRB approval and de-identification, Merlin Plus adds categories (race, scan dates, demographic metadata) that require explicit statements about: (a) HIPAA-compliant de-identification (in particular, dates require shifting, not just removal of year); (b) IRB approval extending to the augmented dataset and mask release; (c) the data use agreement and access controls under which the dataset will be released. The authors should address these explicitly in rebuttal and in the camera-ready.

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    I encourage the authors to consider the following for the camera-ready / a revision: (a) Most important: Report inter-rater reliability on a double-read subset (e.g., 50 cases per tumor type, dual-read by two of the four radiologists), and report Dice/surface distance/Hausdorff agreement. This substantially strengthens the “high-quality” claim. (b) Quantify the annotation-time claim. A small time-motion study (even n=20 scans timed with and without the pipeline) would turn “5× faster” from an assertion into evidence. (c) Add a per-organ Dice table (not just F1) to the main text. The surprisingly low average Dice of 17–25 (caption of Table 1) deserves explanation and per-organ breakdown in the main body. (d) Validate the report-confidence score as a flagging mechanism: on a held-out subset, measure the precision/recall of “flagged masks actually needed correction” vs. “flagged masks were acceptable.” (e) Fix textual errors: placeholder keywords, the “# multi-tumor masks” token in Section 2.1, inconsistent scan counts (25,494 vs. 25,528), and the possibly-invalid arXiv ID in reference [13]. (f) Add an external validation experiment (even a small one on a public dataset containing one of the 9 tumor types (e.g., stomach or esophagus from the Medical Segmentation Decathlon)) would substantially strengthen generalization claims. (g) Provide explicit statements on IRB approval, HIPAA de-identification of dates and demographics, and the intended data use agreement. (h) Clarify the relationship to the companion TR-Distillation submission and what, specifically, is the contribution of this paper versus that one. (i) Consider a failure-case figure: for which tumor types did the active learning loop struggle most? Where did radiologist correction patterns suggest systematic model failure?

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This is a submission with high potential community value but meaningful evidentiary gaps. Dataset papers live or die on two axes: (1) does this dataset fill a real gap, and (2) is the annotation quality credible enough that the community should trust it? On axis (1), the answer is clearly yes. Tumor types representing roughly half of global cancer mortality are under-served by public CT segmentation datasets, and Merlin Plus directly addresses this. If released as promised, it fills a gap that no methodological contribution could substitute for, and the companion experiments demonstrate that models trained on it outperform public alternatives by substantial margins. On axis (2), the paper is weaker. The claim of “radiologist-validated” masks rests on no reported inter-rater analysis, no time-motion validation of the efficiency claim, no validation of the report-confidence triage mechanism, and average Dice scores (17–25) that are surprisingly low and unexplained in the main text. The active learning pipeline is cleverly designed but the paper does not quantify how much each component contributes or where the remaining errors concentrate. Combined with textual issues (placeholder keywords, unfilled tokens, inconsistent scan counts, a likely-invalid reference), the paper reads as valuable work in a somewhat rushed state. The report-based active learning framework itself (particularly the cold-start solution via R-Super and the report-confidence scoring) is a genuine methodological contribution worth disseminating, independently of the dataset release. The Ball-Convolution–based tumor localization within reported organs is a neat concrete use of weak supervision priors. On balance, the dataset’s community value and the novelty of the report-first active learning framework outweigh the evidentiary gaps, which are fixable in rebuttal/camera-ready. A convincing rebuttal providing (i) inter-rater reliability numbers, (ii) an explanation or fix for the low Dice, and (iii) explicit IRB/ethics statements would move this to a clear accept. Failure to address these points would move it to a weak reject, particularly the inter-rater question, which is foundational for any dataset paper claiming “radiologist-validated” annotations.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    My pre-rebuttal recommendation was Weak Accept (4), with the explicit condition: “A convincing rebuttal providing (i) inter-rater reliability numbers, (ii) an explanation or fix for the low Dice, and (iii) explicit IRB/ethics statements would move this to a clear accept. The rebuttal directly addresses all three of these conditional requirements with concrete data. My personal recommendation is Accept for two reasons: 1.The community benefits substantially from this dataset, and rejecting on policy technicalities (rather than scientific merit) when the evidence exists is the wrong tradeoff for an under-served clinical problem 2.The submitted manuscript’s failures were preparation issues, not scientific ones . the work itself appears solid and the rebuttal demonstrates the team has the data to support their claims I therefore move to Accept.



Author Feedback

Thanks for your time, Strong and Weak Accept, and for saying an “impactful dataset for an under-served clinical problem”

All rev., AC:

Inter-reader eval, 2 radiol., 90 masks (10/tumor type, more in camera ready): Tumors: adr bla duo eso gal pro spl sto ute DSC:58,48,47,49,52,54,63,46,58 NSD:62,48,47,46,51,54,66,47,58 HD95:12,18,23,21,16,14,10,23,12

5 new baselines+external eval: FLARE23 top-1/2 (F23-1/F23-2); F24-1;FLARE’25 Baseline (F25b); Voxtell Ours: Best model trained on Merlin Plus (R-Super) Macro-avg (was micro in paper) of DSC±std and F1±½(95%CI) Merlin-DSC Merlin-F1 External-DSC External-F1 F23-1 3±11 19±11 1±7 15±4 F23-2 5±15 22±8 3±11 24±6 F24-1 5±16 26±13 3±10 19±6 F25b 5±15 26±13 4±13 25±7 Ours 32±30 59±11 22±25 68±9 Voxtell (adrenal only) 20±29 62±7 5±11 62±5

DSC: Tumor: adr bla duo eso gal pro spl sto ute External DSC: masks 39 32 84 23 10 106 31 71 17 F25b 1 4 2 2 6 5 12 2 13 Ours 50 32 10 32 19 12 53 13 38 Merlin DSC: masks 16 19 18 10 20 17 29 35 22 F25b 6 0 0 0 1 8 4 2 21 Ours 41 29 9 33 43 51 25 16 47 External F1: F25b 3 15 31 13 8 33 31 24 64 Ours 72 71 62 74 53 68 75 82 52 For space, per-organ eval w/ ours & best new baseline (F25b). All in Camera ready

DSC analysis: sto/duo/eso are hard to segment (lowest inter-reader), lowering the mean. Easier tumors reach 40+ DSC, matching public benchmarks (PanTS top-1: 54)

Active learning: 3 cycles of ~450 masks, 1153 total, ALL radiologist-revised. Covers all Merlin malignant tumors for the 9 types + 352 benigns (focus on malignants, p.4)

R1: Thanks, inter-reader eval is above

R2: questions 7/2 on top

1-“One stronger baseline must be included”: We added 5 (top), including voxtell. Others lack public weights or don’t cover these tumors. ULS was run automatically, the camera-ready now notes this limitation

3-Public datasets: thanks, Q. Chen et al. does cover 2 of our 9 tumor types (esophagus, uterus); our related work now cites it. Yet, masks are few: 17 esophagus, 39 uterus, vs 54 and 115 in ours

4-Figure 1 & caption corrected. Small lesions (<2cm): adr 785, spl 645, ute 274, gal 158, bla 109, sto 66, duo 41, pros 25, eso 14.Longitudinal: 3.8K patients with 2+ scans, 1.5K with 3+; images are from Merlin, patient IDs & dates are our contribution

5-“(Minor) Pipeline novelty is slightly overstated”. Our main contribution is a dataset. Methodologically: the report-confidence score, and using R-Super to solve the cold-start problem of active learning (a first). The camera-ready now flags the pipeline components coming from R-Super

6-Our/random/entropy active learning cycle: +20/+15/+17 DSC (1st cycle, 450 masks)

8-We carefully proof-read the paper (#: 1153)

R3: questions 1/3/5 on top

2-Annotation time (20 masks, 2 per tumor, 4 stomach): w/ pipeline vs w/o Small tumors: 3 vs 16 min (avg) Large: 18 vs 52 Full redraw: 53% Requested active learning details on top

4-Report-confidence score reliably selects masks w/ errors: At cycle 2, only 18.7% of selected masks needed minor correction (DSC>70 vs revised masks). Bottom 20% masks by score: DSC 21 vs revised. Top 20%: DSC 59, close to inter-reader variability

6-Sphere/ellipsoid volume approximations: sphere if the report gives 1 tumor diameter, ellipsoid if 2 or 3.The 30% tolerance was tuned qualitatively w/ radiologists (not a critical choice as all masks were revised)

7-TR-Dis.: architecture (no dataset, no active learn); Merlin Plus: dataset, active learning (no architecture, no method from TR-Dis.)

8-DUA/ethics/license/de-identification follow Merlin. Study covered by Merlin’s IRB. Masks: HuggingFace; all data: Stanford AIMI

9-Malignancy label: report says malignant AND matching cancer ICD-10.Benign: report says benign AND no malignant ICD-10 in the organ. Else: unknown

10-Typos corrected (#=1153)

11-OOD vs IID eva & lesion agnosticism: we now evaluate all models OOD (top) and acknowledge the ULS comparison limitation. We added Voxtell as adrenal-specific baseline (top), a gap Merlin Plus fills




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The reviewers’ opinions on this paper diverged considerably: One Strong Accept and one Reject. The other reviewer who recommended Weak Accept also raised quite a few questions. Thus, I would like to see the authors’ clarification on points such as the Dice score (R2 and R3), baseline comparison (R2), incomplete paper (R2 R3), inter-rater reliability (R3), etc.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The authors have made considerable efforts in the rebuttal to address concerns from reviewers such as Dice score, baseline comparison, inter-rater reliability, etc. Since the released dataset is of great community values, I recommend acceptance, in line with all reviewers. The paper is warranted acceptance even without the new results. It has received strong and weak accept pre-rebuttal. The presented large dataset of CT and tumor mask pairs can greatly assist the community, if released. The new comparison and reader study is a plus but the existing contribution is already sufficient.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The reviewers like the dataset contribution and found the authors’ rebuttal satisfactory including the approach for “cold start problem” when encountering unseen tumors by leveraging clinical text.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The authors have adequately addressed all of the reviewers’ concerns. Accordingly, I recommend acceptance of the paper. Several details clarified in the rebuttal should be included in the final version.



back to top