Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Visual Counterfactual Explanations (VCEs) aim to explain image classifiers by generating minimally edited and realistic versions of an input image that change the classifier’s prediction. Existing VCE methods are inherently classifier-dependent and therefore susceptible to classifier biases and failure modes, such as sensitivity to shortcut features and calibration errors. In this paper, we propose a classifier-free approach for visual counterfactual generation based on Contrastive Analysis (CA). Given two datasets corresponding to different classes (e.g., healthy and patients), we disentangle the generative factors that are common across the two datasets from those that are salient to each dataset, and generate counterfactual images by swapping only the salient factors. By operating directly on data distributions rather than decision boundaries, our method provides model-agnostic VCEs that are less sensitive to classifier biases. Our approach leverages the high-quality synthesis and well-structured latent space of StyleGAN2. We use the feature space F, instead than the usual W-space, to improve detail preservation. Unlike conventional CA approaches, which typically assume salient factors in only one dataset, we introduce an adapted framework and loss functions for VCE that allow multiple salient factors in each dataset. We evaluate our method on three medical imaging datasets and demonstrate superior counterfactual generation quality compared to existing approaches.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/3698_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/BioMedTP/CF_Contrastive_Analysis

Link to the Dataset(s)

N/A

BibTex

@InProceedings{HeYun_Counterfactual_MICCAI2026,
        author = { He, Yunlong AND Gori, Pietro},
        title = { { Counterfactual Contrastive Analysis } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16882},
        month = {September},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper introduces a classifier-free method for visual counterfactual generation built on contrastive analysis. Rather than modifying an image to force a specific classifier to change its prediction, the approach learns to separate images from two class-defined datasets into shared and class-specific generative components, and then produces counterfactuals by exchanging the salient components while keeping the shared structure fixed. The method is implemented within StyleGAN2, combining a latent-space disentanglement module with a subsequent F-space refinement step to better preserve image details. Experiments on three medical imaging datasets show improved reconstruction, cleaner factor separation, and stronger counterfactual image quality than prior contrastive-analysis methods and several diffusion-based VCE baselines.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The methodology is novel. I think its an interesting approach which generates counterfactual edits from distributional differences between classes, rather than from a classifier objective. This reduces sensitivity to classifier-specific artifacts and shortcut features.

    2.The method is coherent and well motivated and the use of StyleGAN2 plus feature-space refinement is technically sensible.

    3.The paper is well written and easy to follow.

    4.The paper includes three medical imaging datasets, comparisons with established baselines, and a dedicated counterfactual comparison against several diffusion-based methods. The qualitative figures are easy to read, and the reported runtime advantage is substantial.

    5.The results against CA baselines are strong. The proposed method appears to improve both reconstruction and swap quality, and the latent separation results suggest that salient informationis better concentrated in the salient variables than in prior approaches.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The main concern is how the paper is frames. The authors use the language of counterfactuals with no notion of identifiability or using structural causal models to derived counterfactuals under interventions. I think the paper paper should present itself more carefully as a generative editing framework rather than as a causal counterfactual method.

    2.The paper is motivated by the claim that classifier-guided VCE methods can reproduce the biases and failure modes of the underlying classifier, but the experimental section does not directly substantiate this point. In particular, the paper does not include a controlled shortcut-feature or biased-classifier stress test demonstrating that classifier-based approaches produce misleading edits while the proposed method instead isolates the intended semantic factor.

    3.Table 3,4 and 5 can be better organised and seperated rather than clumped together

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall i think the paper is a strong and novel contibution with generally robust validation of their claims. I still think the framing of their contribtuion as a counterfactual framework is misleading and reads more as a generative editing framework. The conclusion itself acknowledges that the method is not identifiable, does not enable causal discovery, and does not support strong causal interpretation from observational single-modal data alone. Despite this, the methodology and results are significant and deserves to be presented at MICCAI.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The author have clarified most of my concerns and I recommend acceptance of this paper. This is despite it reading more as a generative editing framework as opposed to a counterfactual framework.



Review #2

  • Please describe the contribution of the paper

    This work presents a framework aimed at generating visual counterfactual explanations (VCE) in medical imaging. The authors utilize a StyleGAN2 backbone combined with a Contrastive Analysis objective, that operates without reference to a specific downstream classifier, to attempt the disentanglement of shared anatomical structures from class-discriminative features across two neighbouring image distributions. Based on this factored representation, the proposed model seeks to synthesize counterfactual images to reveal morphological features driving diagnostic distinctions. The method is demonstrated on three datasets (BraTS, BloodMNIST, and OCTMNIST) and compared against Contrastive Analysis and diffusion VCE baselines.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The authors address Visual Counterfactual Explanations for medical image classifiers, a direction with demonstrated clinical value for model interpretability, well-suited to the broader MICCAI community. The framing as classifier-free contrastive disentanglement as a means to counterfactual generation is a conceptually interesting alternative to the dominant classifier-guided paradigm and merits investigation.

    The paper is clearly written, well-structured with clearly motivated design choices, that makes the method and comparisons easy to follow.

    The choice of StyleGAN2 is well-motivated by its inherently disentangled StyleSpace, which offers per-attribute interpretability that current diffusion-based VCE methods largely lack. This is a meaningful architectural argument that deserves attention, even as diffusion models dominate the generative landscape.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    Conceptual The authors frame the paper as Contrastive Analysis for Visual Counterfactual Explanations. The latter is generally understood as a means to understand black-box classifier behaviour [1], and as such VCE are tied to a given classifier. By reframing the problem to the domain level (a la “What would a healthy image of x look like?”) the authors answer a related but distinctively different question than their choice of focus on VCE implies (“Why did the classifier say that?”). While in itself potentially interesting, this mismatch raises questions about the clinical relevance, and the adequacy of the VCE framing for this contribution, thereby the adequacy for MICCAI.

    This conceptual gap also complicates the evaluation. VCE methods are typically assessed on their faithfulness to the underlying classifier [1,2]. Since the proposed method operates at the domain level rather than in the classifier’s weight-space, the choice to evaluate against a pretrained model and report accuracy or flip-rate metrics is ambiguous as observed changes might not reflect causal (i.e. the classifier’s actual) behaviour [1,2].

    Technical Among others the method is evaluated on BloodMNIST, OCTMNIST. As the work references, both are subset of MedMNISTv2, and as such of 28x28 dimension. While the authors describe images where rescaled to 256x256, this original resolution seems insufficient to evaluate a generative method in a medical / clinical setting, and compare its quality with SOTA methods, rendering 2/3 of the evaluations ambiguous and the StyleGAN2 choice irrelevant.

    For clinical relevance, a domain level generative method needs to be assessed qualitatively by medical professionals [cf. 3]. The proposed method does not provide such insight.

    In general, the evaluation setup doesn’t seem to control for training stochasticity, reports appear to be on single runs, limiting the robustness of the conclusions.

    Novelty While of merit, StyleGAN2 approaches for counterfactual generation is a popular approach [4,5,6,7], as such the work is an application paper of established concepts with limited expansion on the topic, therefore providing ambiguous novelty for the MICCAI community.

    [1] Chowdhury et al, “Looking in the mirror: A faithful counterfactual explanation method for interpreting deep image classification models”, ICCV 2025

    [2] Wachter et al., “Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR”, Harvard Journal of Law & Technology, 2018

    [3] Cohen et al., “Gifsplanation via Latent Shift”, MIDL 2021

    [4] Munoz et al., “HistoXplaining in Style: Counterfactual Explanations for Histopathology Images using StyleGAN2-ADA”,. Inf Syst Front 2026

    [5] Schutte et al., “Using StyleGAN for Visual Interpretability of Deep Learning Models on Medical Images”, NeurIPS workshop 2020

    [6] Lang et al, “Training a GAN To Explain a Classifier in StyleSpace”, ICCV 2021

    [7] Narayanaswamy et al., “Using generative AI to investigate medical imagery models and datasets”, EBioMedicine, 2024

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (2) Reject — should be rejected, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper presents a well-structured and clearly articulated approach to domain level counterfactual generation in medical imaging. However, significant concerns regarding the framing of the paper as a Visual Counterfactual Explanation contribution, and significant concerns regarding the evaluation setup, together with a limited degree of methodological novelty, limit the strength of the proposed work. In its current form, the contribution therefore does not meet the acceptance criteria for MICCAI.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Reject

  • [Post rebuttal] Please justify your final decision from above.

    In its current form, the work presents generative editing decoupled from any classifier, despite being framed as a VCE setup. The authors acknowledge this and propose clarifying the framing in revision. However, such a reframing would require aligning the work with the standards of state-of-the-art generative medical imaging, which among other things entails (i) a thorough qualitative assessment by clinical experts via a reader study [1], and (ii) acknowledging and addressing the known limitations of FID in medical contexts [2]. Both of these are missing and beyond the scope of this rebuttal phase.

    Beyond the framing concerns, the clinical relevance of the work remains unclear even after the rebuttal. The proposed evaluations leveraging a pretrained classifier remain inconclusive as authors didn’t control for biases and failure modes rendering any conclusion ambiguous (albeit part of their motivation). Generally, the work fails to connect motivation and results, as it falls short on evaluating claims like the needed minimality and interpretability. Similarly, while the authors admit that causal discovery is beyond the works scope, without the original interpretability aspect the suitability for any downstream task remains undiscussed.

    For these reasons, I do not consider the work, in its current state, to meet the criteria for publication at MICCAI, and I maintain my assessment

    Note: for comparing two methods statistical tests are more informative and should be favoured over reporting averages as indicated by the authors within this rebuttal

    [1] Yoon et al., Insights and Considerations in Development and Performance Evaluation of Generative Adversarial Networks (GANs): What Radiologists Need to Know. Diagnostics, 14(16), 1756, 2024

    [2] Konz et al., Fréchet radiomic distance (FRD): A versatile metric for comparing medical imaging datasets, Medical Image Analysis 2026



Review #3

  • Please describe the contribution of the paper
    • Classifier-free visual counterfactual explanations (VCE) via contrastive analysis (CA): counterfactuals are generated by operating on the data distribution rather than the potentially biased decision boundary of a classifier
    • Latent space disentanglement in StyleGAN’s W-space via contrastive learning: separates generative factors shared across two datasets (common factors) from those specific to each dataset (salient factors, e.g., pathology-specific features)
    • F-space refinement module: leverages StyleGAN’s higher-dimensional F-space (over W-space) to capture fine-grained details in the generated
    • Evaluated on three medical imaging datasets: outperforms CA and diffusion-based VCE baselines in disentanglement quality, image fidelity, and editing speed.
  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Extensive evaluation: experiments cover reconstruction quality, latent space disentanglement, and counterfactual generation; includes an ablation study; compared against 9 baseline methods in total
    • Generalizability demonstrated across three diverse medical imaging datasets (brain MRI, blood cells, retinal OCT).
    • Strong quantitative and qualitative results over related work
    • Reproducibility: code is publicly available and the method is described in sufficient detail to re-implement.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • Multi-stage training: (1) W-space disentanglement of common and salient factors, (2) F-space refinement. The motivation for splitting these two stages is not sufficiently justified.
    • Hyperparameter selection (e.g., loss weights) is not described; given the multiple competing objectives and the constraint of not disrupting the pretrained GAN’s image generation, it is unclear how stable and sensitive the optimization is and whether the same settings transfer across the three datasets or require per-dataset tuning.
    • Only single runs are reported with no standard deviations, making it difficult to assess result reliability.
    • The F-space is not adequately explained: the choice of layer 8 is not justified, the distinction from W-space and W+-space is not made clear, and known limitations of F-space regarding editability are not discussed.
    • The claim of faster editing is stated in the introduction but never substantiated in the results section.
    • Blood-MNIST and OCT-MNIST have a native resolution of 28x28, yet the model is trained at 256x256; the impact of this large upscaling on results is not discussed.
    • Table 4 uses a different set of comparison methods than Tables 1 and 2, making it impossible to draw consistent conclusions across experiments; the difference between the “swap” operation in Table 1 and the counterfactual generation in Table 4 is also unclear.
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
    • Abstract: The phrase “Given two datasets corresponding to different classes (e.g., healthy and patients)” lacks clarity: could the authors specify what characterizes the “patients” class? Is it defined by a particular disease or set of pathological features?
    • The StyleGAN2 generator is pretrained, but it is not stated on what data.
    • Equation 1: The choice of summation over concatenation for feature fusion is not motivated. Summation is a lossy operation that may discard complementary information encoded in the individual feature vectors; please justify this design choice.
    • Table 3 is included but only mentioned in the text.
    • Table 1: The model variant names (“pSp-cs”, “pSp-cs Ref”, “BT/MS Ass.”, “BT/MS Ass. Ref”) are opaque without reading the surrounding text. Please use more descriptive labels or add a caption-level legend, so the table is self-contained.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper raises several concerns (detailed in the weaknesses section) that I would like the authors to address in their rebuttal. Should these be resolved satisfactorily, I am willing to raise my score to accept.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The authors have satisfactorily addressed all my points in the rebuttal, and I trust that these will be updated or clarified in the final version of the paper.



Author Feedback

We thank all reviewers for their insightful feedback. We address all comments below

Conceptual concern (R1,R2): We acknowledge that our method is not a classical VCE approach, as it does not explain a specific classifier. Instead, it is a generative editing framework that can produce counterfactual examples, which we evaluate using standard VCE criteria with a pre-trained classifier. We will clarify this point in the article. Our work builds on the classical VCE definition from [9]: how an image from class c can be minimally and realistically modified so that a system predicts a different target class. Existing VCE methods rely on discriminative features learned by a pre-trained classifier. In contrast, we avoid classifier gradients and instead learn salient and common features at the distribution level to perform controlled edits, such as swapping and interpolation. Although our approach is classifier-free, we evaluate the generated (swapped) images with a pre-trained classifier, following prior VCE work, to verify whether the counterfactual edits induce the expected class changes. The fact that swapped images are mostly classified as the target class (Table 4) suggests that the learned salient factors capture meaningful discriminative patterns As stated in the conclusion, our method does not enable causal discovery, similarly to recent VCE approaches

Novelty (R2): The methodological novelty does not lie in using StyleGAN2 as a backbone, but in the formulation of counterfactual generation. Unlike prior methods that search for directions in the W+ space maximizing classifier prediction changes, we decompose the W+ space into common and salient factors, generating counterfactual (swap) from distributional differences between classes rather than classifier objectives, as noted by R1 and R3

Datasets resolution (R2,R3): Datasets from MedMNIST+ are provided at 224×224 resolution and were resized to 256×256 to match the input requirements of StyleGAN2.We will clarify this in the revised manuscript

Classifier failure modes (R1): We acknowledge that we did not include experiments on bias reproduction in classifier-guided VCE methods, as this has already been shown in prior work (e.g., [35]). We leave evaluation of our method on controlled bias datasets for future work

Clinical relevance (R2): Qualitative results were assessed by expert medical doctors and biologists, but no formal multi-expert reader study was conducted. The focus was methodological development on MedMNIST+, with a future journal extension planned to include a full reader study

Training stochasticity (R2, R3): We will update Table 4 to report mean and std over 5 training. Rankings and performance differences between methods remain very similar. For example, FR values are 0.955 (±0.02) for our method and 0.976 (±0.02) for ACE

Clarifications (R3): We will clarify that we follow a two-stage training similar to [5], learning first common and salient factors in the W+ space, which is more regularized and editable, and refining details in the F space (layer 8 as in [5]) without changing the semantic edit direction learnt in the W+ space. We only use W+ and F space, with StyleGAN2 generators trained per dataset and then kept frozen. Hyperparameters are selected on validation sets (and not tuned on the test sets) and are the same for all datasets. Inference time is reported in Table 4.Similarly to previous CA works [1,7,10,22,34], we use summation instead of concatenation to avoid extra parameters (to map back to the W+ dimension)

Tables (R1,R3): Tables will be reorganized with clearer, self-contained captions. Table 1 compares our method with contrastive baselines, while Table 4 evaluates counterfactuals against VCE-specific methods. We will clarify that the swap operation in Table 1 is equivalent to the counterfactual generation in Table 4.FID differences arise because Table 1 uses random X,Y pairs, whereas Table 4 fixes either Y or X using the images of Fig. 3



Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper was reviewed by three experts in the field and received the following recommendations: two Weak Accepts and one Reject. While the reviewers noted several strengths, including the novelty of the classifier-free design and clear motivation, they also raised several concerns, including unclear claims regarding causal counterfactual framing and clinical value, insufficient evaluation with only single runs, and resolution issues in Blood-MNIST and OCT-MNIST. The authors are encouraged to address these concerns in the rebuttal.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    After the rebuttal, two reviewers recommended acceptance, while one reviewer maintained a Reject recommendation. While the paper has some limitations, including the lack of a multi-expert reader study, it also demonstrates several strengths, including the novelty of the classifier-free design and clear motivation. Therefore, the paper is recommended for acceptance to MICCAI 2026.The authors are encouraged to incorporate the necessary changes discussed during the rebuttal process into the final camera-ready version.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The paper has interesting points and innovation; R2’s concerns are valid, and the paper is borderline because of concerns of clinical relevance.

    If there were absolutely no chance of relevance, I feel it would be a reject; I will mark this as accept, acknowledging reviewer majority, but I agree that clinical relevance is questionable.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Reject

  • Please justify your recommendation.

    After going through the reviews as well as the authors’ rebuttal, I agree that, as acknowledged by the authors, there is a gap in the claim that requires reframing. I align with the reviewer’s post-rebuttal comment, and the paper needs substantial revision to appropriately revise its framing.



back to top