Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Segmentation models for lung cancer on CT can achieve strong performance yet remain brittle to clinically relevant morphological and textural variations, with failures more prevalent in certain phenotypes. We introduce a counterfactual stress-testing framework that makes this brittleness measurable and actionable. We learn a conditional 3D tumour generator that produces identity-preserving counterfactual CT crops and corresponding pseudo-label masks by intervening on a compact set of radiomic “parent” features. Sweeping each feature across its empirical quantiles produces Counterfactual Robustness Curves (CRCs) for a baseline segmentation model, highlighting the feature regimes in which performance collapses. We define hard regimes as CRC regions where performance falls below a user-defined threshold and use them to select counterfactuals for targeted fine-tuning. On a publicly available CT cohort, CRCs expose consistent failure regions for small volumes, low sphericity, and high fractal dimension. Counterfactual fidelity analysis confirms that interventions achieve the intended feature changes with limited drift in non-target features. Targeted counterfactual fine-tuning guided by CRCs improves test segmentation performance compared with classical data augmentation and unconditional generative sampling.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/5911_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/AinkaranSanthi/counterfactual-stress-testing-lung-segmentation

Link to the Dataset(s)

N/A

BibTex

@InProceedings{SanAin_Counterfactual_MICCAI2026,
        author = { Santhirasekaram, Ainkaran AND Johnson, Charlie AND Chen, Mitchell},
        title = { { Counterfactual Stress-Testing for Fixing Lung Cancer Segmentation Models } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16883},
        month = {September},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper presents a Counterfactual Stress-Testing framework for lung cancer segmentation on CT. The core idea is to generate identity-preserving counterfactual tumor crops by intervening on interpretable radiomics attributes, then using these counterfactuals to measure phenotype-specific robustness through Counterfactual Robustness Curves (CRCs), and then perform targeted fine-tuning on the hardest counterfactual regimes. The paper shows that this framework not only reveals clinically meaningful failure modes, but also improves segmentation performance more effectively than standard augmentation or untargeted generative sampling.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • This paper addresses an important and clinically meaningful robustness problem in lung tumor segmentation tasks. It operates beyond average test-set performance and generic augmentation, and instead asks which specific tumor phenotypes can lead to segmentation failure.
    • The paper proposes a practical way to intervene on interpretable tumor-specific radiomics factors, generate corresponding counterfactual image-mask pairs, and visualize how segmentation performance changes across controlled phenotype shifts. This is visualized well in the shown images as well. This approach is further compared against strong augmentations and generative models.
    • The experimental section is very brief and thoroughly explanative for a 9 page paper.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    There are only two points this reviewer would like to point out:

    1.The counterfactual interventions are defined mainly through tumor-intrinsic radiomics descriptors extracted from the tumor segmented regions themselves. Small question: what are the statistics of the segmentations? is the approach viable across the entire corpus of tumor sizes or is it susceptible to big/small clear/unclear tumors? The current framing makes the interventions interpretable, but it also narrows the stress-testing space. Failure modes driven by tumor-context interactions may not be fully captured by the current set of intervention variables, which rely on the the radiomics features that capture the tumor characteristics itself but not of the composite scan (which is imperative to a proper nnU-Net segmentation). 2.The framework depends on pseudo-label masks from the generator, so generator bias may propagate into both stress testing and repair. This is partly similar to recent self-supervised learning frameworks that leak their position bias into the learning.

    Not so important, but this reviewer also found the presentation of the flow-based generator and OT machinery somewhat dense relative to the main paper contribution, and some of the causal or identifiability language may be stronger than necessary in practice. A significant portion of the time was spent on analyzing and drawing the paper flow graphs on OT and flow matching than the overall holistic approach at hand.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    The paper is very well written, just a small pinch. Its nnU-Net everywhere in the paper, except one place where it is written nnUNet: “We evaluate the nnUNet on each counterfactual image and compute Dice against the corresponding generator mask pseudo-label. “

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper proposes a well-motivated and practically useful framework for understanding and improving segmentation robustness in a way that is more interpretable than standard augmentation-based approaches. It focuses on what kinds of tumors can cause a problem and on less of scanner/acquisition parameter variations or other noise sources (a more common setting studied across the board). The methodology is thoughtful, the empirical comparisons are strong, and the paper tells a compelling end-to-end story from diagnosis to targeted model improvement. Overall, a well-written paper except for the two weaknesses that can be very well accounted for in the revised discussion.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The rebuttal satisfactorily addresses my concerns, clarifying the SCM/abduction formulation and its approximate implementation while providing solid evidence of identity preservation outside the tumor. The added HD95 analysis further strengthens confidence by confirming boundary robustness alongside Dice improvements. This reviewer also appreciates the clearer positioning as a practical framework for counterfactual robustness.



Review #2

  • Please describe the contribution of the paper

    This paper proposes a counterfactual stress-testing and repair framework for lung tumor segmentation, which leverages flow-matching generative networks conditioned on radiomics parents to inpaint tumors on patient-specific exogenous factors. The idea itself is oif certain novelty and the flow of the paper is easy to understand.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The paper presents extensive analysis of the generated counterfactual examples including their effect on downstream segmentation, their fidelity wrt. an in-house validation set and the evolution of segmentation performance over intervention deciles.
    • The idea of leveraging hard counterfactual example is a popular way of enhancing downstream segmentation performance and the authors’ implementaton is of novelty.
    • The paper is generally well-written and easy to understand.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    – The result of this paper is only tested on one dataset (NSCLC Radiogenomics), while there exists more datasets on lung cancer segmentation in general (MSD-Lung, etc). The author should provide reasons why they choose not to test their method on them. Also, the results are only evaluated on dice, evaluating them with other widely used metrics such as HD95 can be better.

    • While complementing segmentation dataset with counterfactual samples is of significance intuitively, there exists a line of work that does tumor inpainting that also boosts downstream tumor segmentation performance, some even utilize textual prompts to be of more versatility (LesionDiffusion, GuideGen, DiffTumor, FreeTumor, etc). Discussing the differences or comparing with them can make the proposed counterfactual method more convincing.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The method itself is acceptable, but it is only tested on one dataset with only one metric, and the reported performance gain is not that significant (+0.03 dice). I also would like the author to discuss the advantages (if any) of their counterfactual design with other tumor inpainting/genesis pipelines.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    the authors’ rebuttal solves most of my problems, i recommmend the authors add the analysis they provided in their rebuttal in the final version.



Review #3

  • Please describe the contribution of the paper

    This paper proposes a counterfactual stress-testing framework for CT lung tumour segmentation. The core is a conditional 3D flow-matching generator trained with minibatch OT coupling, conditioned on six radiomic parents. The authors use reverse-time ODE integration to abduct the exogenous factors from a factual crop, then swap one parent to a value at a chosen decile and regenerate the counterfactual. Sweeping across deciles gives Counterfactual Robustness Curves for a baseline nnU-Net. Counterfactuals where the baseline scores below Dice 0.7 are mined and used to fine-tune the segmenter (CF-hard). On an independent NSCLC test cohort, CF-hard reports the best mean Dice (0.84) among BigAug, GAN-Aug, Rad-OS, and a CF-rand ablation, compared to a baseline of 0.77.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    • The motivation is clinically relevant. The paper correctly points out that current robustness work measures sensitivity to corruptions like noise or motion, but rarely isolates which tumour attributes drive failure. Framing robustness as sensitivity to interpretable radiomic attributes is a reasonable response to this gap.

    • Individual-level CRCs are a distinctive angle. Most prior robustness analyses operate at the cohort level. Asking, within a single patient, how the prediction would change if the tumour were larger or more spherical is a different kind of question, and one that could be useful for deployment auditing.

    • The ablation study demonstrates the importance of the fine-tuning sample selection strategy. Comparing CF-hard to CF-rand with the same number of added samples makes it clear that the gain comes from targeting failure regimes rather than from more training data.

    • The experimental setup uses an in-house cohort for generator training and two public NSCLC datasets for segmentation training and independent testing, which is a reasonable and diverse data arrangement.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The SCM assumption that an intervention on one parent changes only that parent is not met in practice. Markovian abduction–intervention–prediction requires the parents to be mutually independent and independent of U. Table 1’s non-target drift (0.11 to 0.26 z-score, highest for fractal dimension) is direct evidence that, on this data, interventions on one parent move the others. Section 4.3 acknowledges this but does not resolve it. Without clean target isolation, the framework sits closer to conditional regeneration than to a causal intervention.

    2.The identity-preservation claim is never empirically validated. The whole framework relies on abduction recovering a non-tumour latent so that intervening on r leaves the surrounding anatomy intact. Table 1 only measures radiomic drift inside the tumour mask, which says nothing about what happens outside it. Basic checks are absent: background SSIM or difference maps, cycle consistency of abduction then regeneration, or cross-patient swap tests. Without these, CRC drops at extreme deciles cannot be separated from unintended background drift.

    3.The chosen parents do not represent the failure modes the intro emphasises. The introduction frames the problem around small lesions, poorly defined boundaries, GGO, and adjacent anatomy with similar attenuation. Of the six parents, only volume directly targets small lesions. Fractal dimension captures geometric complexity but not partial-volume or GGO-induced fuzziness, mean HU partially reflects GGO but not its spatial pattern, and attenuation similarity with adjacent tissue (a contrast property rather than a tumour-internal one) is not covered at all.

    4.Dice is the sole evaluation metric despite the paper’s own warning. The opening notes that mean Dice can hide brittle behaviour, yet CRC construction, hard-sample selection at τ = 0.7, and the final test-set comparison all use Dice alone. HD95 or surface Dice would directly probe the boundary-complexity failures the method claims to address, and could plausibly change which counterfactuals get labelled hard.

    5.Figure 4’s method comparison is reported for only one feature, and that feature is the most favourable to the narrative. The left and middle panels show baseline-only CRCs for wavelet entropy and volume; only the right panel compares all methods, and only under fractal-dimension intervention. Fractal dimension is the feature where the baseline collapses most sharply in Table 2 and where generator image fidelity is lowest among shape-related parents (ρ = 0.85, drift = 0.26). Its extracted value also depends on the exact mask used, which makes it particularly hard to separate true segmentation brittleness from generator-mask artefacts at extreme deciles. A method-comparison grid across all six features, or test-set Dice stratified by quintile, would be needed to show the gain generalises.

    6.The dataset description is internally inconsistent and does not match the cited sources. Reference [1] (Aerts et al.) is NSCLC-Radiomics (Lung1, 422 subjects with manual GTV), and [2] (Bakr et al., Sci. Data 5:180202, 2018) is NSCLC-Radiogenomics (211 subjects, masks released for only 144). Section 4.1 assigns the names in reverse. Even after correcting the swap, the 207-case test split does not align with 144 available Radiogenomics masks, and the provenance of the test masks is not stated. Table 1 and Figure 4 also report results on an “in-house validation set” that is never defined: the 100 public validation cases from Section 4.1, a slice of the 1,472 generator-training crops, or a third cohort. Since the headline rests on a test-set Dice improvement, these need to be stated precisely.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    1.”Two A100 GPUs (48 GB)” does not match any real A100 variant. The A100 ships as 40 GB or 80 GB. 2.Figure 4’s y-axis is labelled “CRC response” but the term is never defined. Figure 2 uses “Dice” on the same type of plot. 3.Figures and tables sit far from where they are first referenced. 4.The hardness threshold τ = 0.7 is applied uniformly across features, but Table 2 shows baseline Dice differs substantially between them (volume Q1 at 0.62 vs fractal Q1 at 0.84). A uniform threshold will pull in very different fractions of counterfactuals per feature and may bias CF-hard toward features with the weakest baseline.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The motivation and the shape of the contribution (patient-level CRCs, hard-mining failure regimes) are appealing. The main concern is that the SCM framing the whole framework rests on is not backed up by the evidence in the paper. The independence assumptions are hard to accept for this feature set, Table 1’s drift shows directly that interventions leak across parents, and identity preservation outside the tumour mask, the whole reason abduction is invoked, is never tested. The overall impression is less a new causal method than an analysis pipeline built on a conditional generator, dressed in SCM language. The dataset description also does not match the citations and leaves the test-set provenance unclear, which weakens the reported Dice improvement.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Reject

  • [Post rebuttal] Please justify your final decision from above.

    The central issue is unchanged. The rebuttal confirms that the theoretical conditions for counterfactual identifiability (exact dynamic-OT coupling) are not met in practice, yet continues to invoke Ribeiro et al. (2025) as justification. This does not close the gap between the SCM framing and the experimental evidence. Several other concerns raised in the review were acknowledged but not resolved. The mismatch between the failure modes motivating the work (poorly defined boundaries, similar-attenuation adjacent structures) and the chosen radiomic parents was conceded as a limitation. Non-target drift in Table 1 was attributed to finite-sample error, but this explanation does not resolve the concern. I do not believe the current manuscript meets the acceptance threshold for MICCAI and would benefit from revision and resubmission.



Author Feedback

1.Identity preservation and SCM framing [R1, R4]. The SCM/abduction angle is central to the method. Our goal is not to generate another plausible tumour, but to answer an individual-level counterfactual question: what would this same case look like if one radiomic parent were changed? This requires abduction–intervention–prediction: without this step, generative augmentation changes both tumour phenotype and patient-specific anatomy. This is motivated by the identifiability theory in [1], which shows that, under dynamic-OT/flow-matching assumptions, the monotonic transport map supports unique abduction for high-dimensional counterfactual prediction. Thus, the Markovian SCM formulation is important because it separates patient-specific exogenous variation from the intervened radiomic parent and makes the CRC an individual-level counterfactual robustness curve rather than an unconditional synthesis curve. This is precisely the property needed for clinically meaningful stress testing of the same patient. We agree that our implementation is approximate. The non-target drift in Table 1 reflects finite-sample/finite-capacity error rather than perfect parent isolation. In particular, minibatch OT with a batch size of only 100 due to memory requirements cannot exactly recover the population dynamic-OT coupling required, so leakage between correlated radiomic features is expected and will be explained in the paper. We also agree that identity preservation should be checked outside the tumour. We have now computed background preservation on the overlapping non-tumour region after removing both factual and counterfactual masks: background MSE is 0.03 ± 0.01 and SSIM is 0.89.These new results support the claim that the patient-specific context is largely preserved. 2.Radiomics scope [R2, R4]. We agree that the current parents are tumour-intrinsic and do not capture all context-driven failures. We will state this limitation and add validation-set ranges showing that CRCs span the tumour distribution in the validation set: volume 46–129 cm³, sphericity 0.33–0.80, fractal dimension 2.38–2.78, mean HU −20 to 118, HU SD 35–199, and wavelet entropy 2.1–6.9.Table 2 already shows failures across multiple parents/quintiles. Fig. 4 shows one method-comparison CRC for illustration. Generalisation is assessed on the independent test set in Table 3, where CF-hard outperforms all baselines. 3.Boundary metrics [R2, R4]: Dice was used for CRCs and τ=0.7 mining because it is the standard nnU-Net selection metric and gives a stable failure criterion across parents. We now add HD95: baseline 8.7±5.5 mm, BigAug 7.2±4.6, GAN-Aug 7.5±4.8, Rad-OS 8.1±5.1, CF-rand 6.9±4.7, and CF-hard 5.6±3.8.Thus, CF-hard improves boundary accuracy as well as overlap. We will also explore HD95/ASD-based mining as future work. 4.Pseudo-labels, related work and minor errors [R1, R2, R4]. We agree pseudo-label bias is possible; we do not treat generator masks as perfect, and support them with fidelity, drift, and added background-preservation checks. Since CF-hard and CF-rand use the same generator, pseudo-labels, and sample budget, the gain isolates CRC-guided failure selection. MSD-Lung is valuable future validation. We do not use LesionDiffusion, GuideGen, DiffTumor, and FreeTumor as direct baselines as we already control for augmentation benefit with BigAug and GAN-Aug under the same sample budget but will include them in our related work. Our distinct contribution compared to these methods is to diagnosing and repairing named radiomic failure regimes. We will correct the swapped NSCLC-Radiomics/Radiogenomics citations, state cohort provenance, exclusions, splits, and mask sources, and define the in-house validation set. We will also fix the A100 typo, relabel Fig. 4’s y-axis as Dice, and improve figure placement. [1] Ribeiro, F.D.S., Santhirasekaram, A., Glocker, B.: Counterfactual identifiability via dynamic optimal transport. arXiv preprint arXiv:2510.08294 (2025)



Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    A counterfactual stress testing framework to identify the failure modes in tumor segmentation is presented, which then is used to improve segmentation performance. The counterfactual generation using the 3D flow matching with minibatch coupling and radiomic features conditioning is seen as novel. The paper is well-written and experimental evaluation are reasonable to establish downstream utility. Concerns were raised regarding certain claims including identity preservation as well as several minor concerns that should be addressed carefully in the rebuttal.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The authors addressed most of the reviewers’ concerns and the updated results satisfied reviewers’ concerns. However, as pointed out by reviewer 2, please acknowledge the limitations and position the work clearly taking in to account its limitations and what it can address.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    This paper targets a sharp angle in lung cancer segmentation through counterfactual stress testing and is sufficiently validated. Essentially, it asks: for the same patient, and also at the population level, which tumor attributes (among selected radiomic features) would cause the segmentation model to fail? An additional practical outcome is that the generated failure-mode samples can be used to augment training and improve the original segmentation network.

    The remaining concerns are: (1) identity preservation is a key component and is partially validated, but changing one radiomic feature may also affect other features, which needs further exploration; (2) the generated masks may introduce additional pseudo-label errors; (3) validation on additional datasets with broader comparisons would further strengthen the paper.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    All reviewers agree that the paper has merits on how it presents an interpretable robustness check allowing to identify difficult tumors. The rebuttal touches on most concerns, and two out of three reviewers believe their points have been sufficiently addressed.



back to top