List of Papers Browse by Subject Areas Author List
Abstract
Pathology foundation models (PFMs) provide strong tissue representations and have become central to digital pathology. However, deployment in disease-specific settings is limited by 1) the high computational cost of billion-parameter PFMs and 2) distribution mismatch and non-biological bias inherited from pan-cancer, multi-centre pre-training, including site-specific signatures and imbalanced disease prevalence. These factors can encourage shortcut learning and under-emphasise subtle morphology required for reliable modelling of a specific cancer type. We present SmartStu (a Smart Student), a framework to customise compact, breast-cancer-specific PFMs via distillation whilst mitigating confounding. SmartStu distils representations from multiple teacher PFMs into a lightweight student backbone. Crucially, we introduce adversarial distillation that leverages a dedicated noise model trained to predict nuisance, edge-dominated cues on the distillation set. Using this noise model as a counterexample, the adversarial objective encourages the student to recognise, yet suppress, features predictive of nuisance targets. We further incorporate multi-teacher ensemble distillation and an auxiliary self-supervised objective with artefact injection. We validate SmartStu on three external cohorts (Yale HER2, SLN-Breast, and BRACS) with multiple tiny backbones. SmartStu yields breast-cancer-specific PFMs that are over 30× smaller than general PFMs whilst largely preserving, and sometimes improving, downstream performance measured by balanced accuracy (bAcc) and AUC. Code is available at https://github.com/zwchen03/advDistall.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0753_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/zwchen03/advDistall
Link to the Dataset(s)
TCGA-BRCA dataset: https://portal.gdc.cancer.gov/projects/TCGA-BRCA
Yale-HER2 dataset: https://www.cancerimagingarchive.net/collection/her2-tumor-rois/
SLN-Breast dataset: https://www.cancerimagingarchive.net/collection/sln-breast/
BRACS dataset: https://pubmed.ncbi.nlm.nih.gov/36251776/
BibTex
@InProceedings{CheZhi_Harnessing_MICCAI2026,
author = { Chen, Zhiwei AND Hu, Yang AND Xiao, Yuxiang AND Ju, Yakun AND Zhang, Tianyang AND Xu, Yingxue AND Li, Wei AND Chen, Hao AND Rittscher, Jens AND Yang, Kaixiang},
title = { { Harnessing Adversarial Distillation to Customise Debiased, Disease-Specific Pathology Foundation Models for Breast Cancer } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16891},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes SmartStu, a framework for compressing large pathology foundation models into compact and breast-cancer-specific student models via multi-teacher feature distillation, adversarial distillation, and auxiliary self-supervised learning, while actively suppressing site-related biases inherited from multi-centre pretraining.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Well-motivated problem: Deploying billion-parameter PFMs and robust models in clinical pathology workflows is a real bottleneck. Jointly addressing model compression and bias mitigation in a single framework is a meaningful contribution, these two problems are usually tackled independently.
- Principled debiasing mechanism: The adversarial distillation idea is good, train a dedicated model to capture exactly what you want to remove, then use gradient reversal to penalise the student for encoding those features. This is more targeted than generic stain normalisation, which can destroy diagnostically relevant variation.
- Thorough experimental design: The paper evaluates across 3 external cohorts, 8 student architectures spanning CNNs and ViTs, single- and multi-teacher settings, and ablates each component (distillation → adversarial → full objective). The consistent stepwise improvement across these axes is convincing.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- BRACS performance gap remains large: On the 7-class BRACS task, even the best SmartStu variants reach ~0.51 bAcc vs. teacher baselines at ~0.51 as well, but this is still low. More critically, the gap between student and teacher is relatively small on easy binary tasks but the absolute numbers on the harder multiclass task suggest the representations may struggle with fine-grained subtyping, which is arguably the most clinically important scenario.
- Statistical reporting is weak: Five-run means are reported but no standard deviations or confidence intervals. Provide statistical test would make it better.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
It appears the authors may have uploaded an incorrect file as the supplementary material, as it seems to correspond to a different MICCAI submission.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is methodologically sound, though the concerns raised in the weaknesses section should be addressed in the final version.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
This paper proposes a knowledge distillation method that actively suppresses site-specific confounding factors. By pre-training a dedicated noise model to capture site-specific cues (such as staining and scanning variations), and then using these as counterexamples during the distillation process, the method employs a gradient reversal layer to force the student model to recognize yet suppress these interfering features, rather than passively inheriting the teacher model’s biases. The authors developed a comprehensive custom workflow to distill a general, large-scale pan-cancer PFM into a compact, breast cancer-specific student model, while stabilizing training through multi-teacher ensemble and auxiliary self-supervised objectives. Validated on three independent external datasets (Yale HER2, SLN-Breast, and BRACS), SmartStu not only preserves the downstream performance of the teacher model but even outperforms the original teacher in certain configurations, while its attention heatmaps focus more on true tumor regions, reducing false activations.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The bias-removal mechanism designed in this paper is quite innovative. By explicitly modeling and suppressing site-specific features, adversarial distillation breaks away from the passive approach of traditional distillation. Compared to the domain classifiers commonly used in domain generalization, the noise model actively captures edge-dominated interference cues (such as scanner grating and staining batch effects) rather than generic domain labels. This work achieves over 30-fold parameter compression and supports a wide range of deployment scenarios, from MobileNet to ResNet. By simulating real-world clinical variations, it enhances the model’s stability on heterogeneous data, making it highly practical for engineering applications.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Adversarial models require pre-training on multi-center data and rely on site labels. This may result in prohibitively high training costs for new diseases or rare cancers where metadata annotations are lacking. Although the attention heatmap shows that SmartStu focuses more on tumor regions, there is a lack of fine-grained visualization or ablation analysis to determine which feature channels the adversarial distillation specifically suppresses and which disease-related patterns it preserves. The entire model remains a black-box process. There are too many training hyperparameters, and since the paper doesn’t list them all, it’s very difficult to reproduce the results. Finally, is this supplementary material submission compliant with the guidelines? It seems that supplementary materials of a non-pathology demonstration type are not permitted.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Solid experimental work.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
- Lightweight PFM distillation framework for breast cancer application designed to actively suppress site-related bias & confounding factors
- Authors employ an adversarial distillation mechanism using a dedicated noise model to identify nuisance targets & capture site-related signs.
- Integrates robustness components including multi-teacher ensemble distillation & auxiliary self-supervision with multi-type artifact injection to stabilize representation learning under realistic acquisition variability
- Strong evaluation by validating method on 3 external breast cancer datasets, Yale HER2, SLN Breast & BRACS using ABMIL, often leading to improvement in downstream performance. Further flexibility demonstrated by testing with multiple diverse student backbones such as ResNet, TinyVit, MobileNet-v3
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Unlike standard knowledge distillation that propagates useful features from teacher including site-related bias, here in SmartStu, the use of a dedicated noise model as a counter-example (via gradient reversal) during distillation seems like a novel way to address the distribution mismatch & non-biological site-specific bias inherent in tissue agnostic pre-training
- Interpretability analysis is also a standout feature of the work. Authors demonstrate that adversarial debiasing leads to more biologically relevant features. Visualizations using attention heatmaps & Eigen-CAM show that SmartStu suppresses spurious background activations & concentrates its focus on actual tumor ROI, outperforming even its larger teacher models in localization
- Author also address the deployment bottleneck of massive PFMs in resource-constrained clinical workflows, achieving over 30x model compression while preserving or even improving downstream performance.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- The current validation of the framework’s disease-specific claims is limited to breast cancer. To demonstrate broad applicability, it would be important to validate the framework on another more prevalent cancer types, such as lung or prostate cancer.
- Another concern is that the adversarial debiasing process may over-clean the learned representations, for eg the adversarial teacher, trained to detect site-specific signs, risks eliminating important biological variations, such as site-specific patient morphology, potentially mistaking them for nuisance noise. The paper seems to miss out a discussion or maybe a sensitivity analysis to demonstrate that this debiasing does not also remove clinically important features merely correlated with collection sites.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the individual components of this work, such as knowledge distillation & gradient reversal, are somewhat incremental applications of established techniques, the overall framework seems to be novel & offers good clinical utility. The 30x compression of PFMs, coupled with improvement in out-of-distribution performance through active debiasing, represents a good practical achievement for real world deployment. This is all effectively demonstrated by empirical validation across 3 external cohorts & with multiple student backbones.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #4
- Please describe the contribution of the paper
The authors propose SmartStu, a framework to remove confounders from pathology foundation models and distill their knowledge into smaller backbones, without degrading downstream performance significantly. SmartStu also introduces adversarial distillation which uses a noise model to predict and suppress site-related biases embedding within the FMs. In this setup the authors pretrain a separate model including site labels for the data and use this noise model to teach the student model to suppress site-specific signals. The authors also experiment with multi-teacher distillation variants and demonstrate that smaller, parameter-efficient student models are successfully able to retain and often improve performance in comparison with the parent FMs. Evaluation across three external breast cancer cohorts and various student backbones outlines the effectiveness of SmartStu in preserving knowledge across model size.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper has several major strengths:
Significance of motivation: The retention of performance without using hundreds of millions of parameters in foundation models is a crucial selling point of SmartStu and it demonstrates that knowledge from rich pathology FMs can be transferred and utilized in resource constrained environments.
Standardization: The use of TRIDENT in preprocessing, the choice of datasets for both pretraining and external validation are all standard practices. It enables fair and accurate comparison across model backbones, experimental setups and cohorts.
Experiments back claims about knowledge retention: Extensive experiments using various student backbones and clear performance metrics demonstrate that the authors’ claims about retaining and often improving performance using the distilled student model are supported by evidence.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The paper has a few major weaknesses that need to be addressed:
Fair comparison with mini FMs: There are plenty of mini pathology FMs that have now been introduced that employ distillation to transfer knowledge into smaller deployable architectures or take other approaches for downsizing the FMs. Models like H0-mini (~20M), Atlas2S (~7M), Virchow2G Mini (~22M) are parameter efficient versions of their larger counterparts that show equally impressive performance on downstream tasks. The authors must position themselves fairly in the midst of these smaller variations of the FMs to ensure meaningful comparison.
Robustness Index not reported: It is now pretty standard to report the robustness index (RI) for evaluating foundation models in pathology. It quantifies the representational robustness of encoders across scanners and staining pairs. Without RI it becomes difficult to understand whether the distilled embeddings are truly stable across these scenarios. Authors should use the PathoROB benchmark to report RI.
The central claim about debiasing: The central claim about debiasing is well motivated (a recent study by Dawood et al, link: https://www.nature.com/articles/s41551-026-01616-8 can be a strong supporting evidence for the motivation, although I believe this is too recent since it was published on March 2, 2026). However, there is no experimental evidence demonstrating that the debiasing has actually worked. The manuscript does not prove that the noise model actually models site bias as opposed to a blend of site, stain, scanner biases alongside meaningful morphological signals. The artefact injection is likely to cover some nuisance factors but not the entire set of confounding factors. The technical strength of the paper would be improved many fold if the authors reported quantitative measures of site-bias reduction.
Multi-teacher distillation precedence: GPFM (https://www.nature.com/articles/s41551-025-01488-4) proposes a generalizable pathology FM and outperforms many of the established pathology FMs across many tasks. It uses expert KD from UNI/Phikon/CONCH plus self-distillation, so it already distills knowledge from other FMs and improves upon their performance. I believe it would be prudent to distill GPFM into student backbones to see how much SmartStu can retain performance and possibly even improve further on external validation.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The authors do not present a fair comparison between SmartStu distilled students and mini FMs that are distilled from their larger counterparts. It is essential for exhaustiveness.
Robustness index is not reported.
The central claim surrounding debiasing the model from site-specific signals is not supported by evidence. The authors should validate the claim by presenting metrics.
GPFM is a crucial distillation source given that it already distills from other pathology FMs and improves performance on many tasks.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank the reviewers and meta-reviewer for their thoughtful comments and for recognising the novelty, practical relevance, and strong evaluation of SmartStu. We just would like to clarify a procedural issue: the supplementary PDF was not uploaded in error. It was an anonymised version of a related concurrent MICCAI submission, provided in accordance with the MICCAI 2026 guidance for authors submitting closely related papers. We also appreciate the constructive suggestions on statistical reporting, broader disease validation, more quantitative assessment of debiasing, and further positioning against emerging compact pathology foundation models and robustness benchmarks. These comments are valuable and will inform the camera-ready version and our future work.
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes a framework for compressing large pathology foundation models into debiased, disease-specific pathology foundation models. The bias removal algorithm is novel and the parameter compression makes it very promising for real-world deployment. Most reviewers agree that the evluation is strong.