List of Papers Browse by Subject Areas Author List
Abstract
Pathology Foundation Models (PFMs) offer powerful Whole Slide Image (WSI) representations but suffer from massive computational costs. While Knowledge Distillation (KD) can create efficient student models, existing multi-teacher methods often use suboptimal uniform weighting that ignores tissue heterogeneity. We propose LaGuadia (Language-Guided Adaptive DistillAtion), a framework that develops a compact pathology image encoder by dynamically integrating expertise from multiple PFMs under clinical linguistic guidance. Our approach utilizes a multi-stage pipeline: first, extracting visually observable clinical keywords from pathology reports; second, aligning visual features with these keywords via a Vision-Language meta-teacher (MedSigLIP) to provide dense semantic guidance; and finally, performing adaptive KD where teacher contributions are weighted based on their semantic alignment with the clinical narrative. Experiments on WSI captioning, visual question answering, and slide-level classification tasks demonstrate that an 87M parameter LaGuadia student model matches or exceeds foundation-scale models such as GigaPath and UNI, achieving strong factual consistency and robust generalization. These results highlight clinical language as an effective semantic anchor for building efficient and reliable digital pathology systems. Code is available at https://github.com/hvcl/LaGuadia.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/6364_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/hvcl/LaGuadia
Link to the Dataset(s)
N/A
BibTex
@InProceedings{KimGan_LaGuadia_MICCAI2026,
author = { Kim, Gangsu AND Jeong, Won-Ki},
title = { { LaGuadia: Language-Guided Adaptive Distillation from Pathology Foundation Models } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16879},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes LaGuardia, a language-guided adaptive multi-teacher distillation framework to train a compact pathology image encoder. Across WSI captioning, VQA, and MIL classification on TCGA cohorts, the 87M parameter student reportedly matches or exceeds substantially larger PFMs, with notable gains in factual consistency and competitive performance on slide-level tasks.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The idea of language-guided, per-patch adaptive teacher weighting is compelling and well-motivated for handling spatial/semantic heterogeneity in WSIs. 2.Evaluates on diverse downstream tasks spanning generation (captioning, VQA) and conventional slide-level MIL (survival, subtyping, N-stage), using five-fold CV.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Heavy reliance on LLM-based keyword extraction (ChatGPT-4o-mini) raises reproducibility concerns; prompts and controls are not fully specified and LLM outputs can be non-deterministic. 2.Pseudo-labeling depends on a single nearest keyword in an embedding space that itself is learned from a meta-teacher; robustness to noisy or incomplete report-derived keywords is not quantified. 3.Distillation uses only CLS/class tokens; fine-grained patch- or region-level signals are not leveraged, which may limit gains on morphology-centric tasks (acknowledged as a limitation). 4.On captioning, VQA ,and Survival, average improvements are small (often within noise margins), with no statistical significance testing or human evaluation; claims of superiority should be tempered. 5.The training corpus for distillation includes the same TCGA cohorts used in downstream evaluation; without external validation, potential leakage or overfitting to cohort-specific language patterns cannot be ruled out. 6.Recent works on multi-teacher fusion/distillation and multimodal alignment are under-discussed, e. g. , Shazam (online MoE fusion), FuseCPath (collaborative distillation with heterogeneous PFMs), mSTAR (multimodal distillation to patches), and slide-level contrastive frameworks like DRE-SLCL. Positioning LaGuardia relative to these,especially regarding efficiency vs accuracy trade-offs,would strengthen the context.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
None
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper presents a timely and interesting approach to leveraging clinical language for adaptive multi-teacher distillation in computational pathology. However, improvements on captioning/VQA are modest, and the experimental section would benefit from stronger fairness controls, broader ablations, and external validation. The dependence on LLM-based keyword extraction also raises reproducibility questions that should be mitigated by releasing prompts and alternatives. Overall, I view LaGuardia as a promising and practical distillation framework with clear clinical relevance and a sensible path to deployment; with clarifications and expanded evaluation, it would be a solid contribution. I lean toward weak acceptance, contingent on addressing fairness and reproducibility concerns.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors’ response has addressed my concerns to some extent; however, I believe there is still room for further improvement in the manuscript.
Review #2
- Please describe the contribution of the paper
The paper introduces LaGuadia, a language-guided adaptive distillation framework that leverages report-derived clinical keywords and a vision-language meta-teacher to dynamically combine multiple pathology foundation models, enabling a compact student encoder to retain clinically relevant knowledge and achieve competitive performance across several downstream pathology tasks.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The work focuses on distilling large pathology foundation models into a much smaller student encoder, which is an important direction given the computational cost of current whole-slide image foundation models and the need for more deployable pathology systems. The motivation is clinically and technically well justified. 2.The paper includes evaluation of factual consistency for generative tasks. In addition to standard text-generation metrics such as METEOR, BLEU-4, and ROUGE-L, the authors also report FactEnt for captioning and VQA-style generation. This is a sensible choice in a clinical setting, where factual correctness matters more than fluent-sounding text, and it strengthens the paper’s empirical framing.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The methodological novelty is limited, and the paper somewhat overstates the originality of its core idea. Several ingredients of the proposed framework have clear precedents in prior work. GPFM already studies unified multi-expert knowledge distillation for pathology foundation model pretraining. On the multimodal side, CONCH and MUSK have already established pathology image-text alignment as an important foundation-model direction. Importantly, KEEP further shows that explicit disease knowledge can be injected into pathology vision-language pretraining through a disease knowledge graph and hierarchical semantic grouping, indicating that the broader idea of introducing structured clinical semantics into pathology representation learning is not new. 2.A core part of the pipeline depends on a closed LLM, yet that component is not validated. The paper uses ChatGPT-4o-mini to extract clinically observable keywords from pathology reports, and this keyword extraction step is foundational to the whole method. However, there is no evaluation of keyword quality, no prompt-sensitivity analysis, no comparison with simpler baselines such as rule-based extraction or pathology ontologies, and no discussion of failure cases. Because the proposed adaptive weighting mechanism is only as reliable as the extracted keywords, this omission hurts both reproducibility and interpretability. 3.The experimental protocol leaves important ambiguity about leakage. The manuscript states that the model is distilled using 1,910 WSIs and 1,801 pathology reports from TCGA BRCA, STAD, and THCA, and that keywords extracted across all cohorts are aggregated into a global Keyword Bank. Downstream evaluation is then performed on tasks drawn from the same three TCGA disease cohorts using 5-fold cross-validation. What is not clearly stated is whether report processing, keyword-bank construction, and encoder training are repeated separately inside each fold, or whether report-derived information from held-out folds can enter the distillation pipeline. Since the method is explicitly language-guided, this ambiguity is more serious than in a standard visual pretraining setup. 4.The headline improvements on the main generative tasks are numerically tiny, yet the claims are much stronger than the evidence. On WSI captioning, the overall score is 0.2461, compared with 0.2458 for UNI and 0.2457 for GPFM. On WSI VQA, the overall score is 0.4184, compared with 0.4182 for Virchow2.These are extremely small margins. The paper reports fold means, but no standard deviations, confidence intervals, or significance tests. Under those conditions, claims such as “outperforms foundation-scale models” and “demonstrates robust generalization” are too strong for what the tables actually show. Moreover, the absolute values on the survival-related tasks also appear unexpectedly low and are insufficiently contextualized with respect to recent pathology survival literature; although direct cross-paper comparison is imperfect because cohorts and splitting strategies vary, recent pathology foundation-model studies have reported substantially higher mean C-index values on TCGA survival benchmarks, which makes it difficult to judge whether the gains reported here are practically meaningful or whether the present evaluation setting is unusually challenging. 5.The paper also does not convincingly establish that the proposed multi-teacher distillation framework provides clear value beyond a strong initialization. The student model is already a DINOv3-initialized ViT-B, which is a strong generic visual backbone, yet the additional pathology-specific distillation machinery yields only marginal gains on the main generative benchmarks. In particular, the overall improvement is only 0.0003 on WSI captioning compared with UNI and 0.0002 on WSI VQA compared with Virchow2.Moreover, the authors’ own ablation shows that naive KD can even degrade key clinical metrics, with FactEnt dropping from 0.5017 to 0.4988 and survival performance dropping from 0.5425 to 0.5180.Taken together, these results make it difficult to conclude that the proposed distillation strategy works in a robust and clearly beneficial manner; rather, the current evidence suggests a complicated pipeline whose empirical advantage over a strong off-the-shelf initialization remains insufficiently demonstrated.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(2) Reject — should be rejected, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The main issues are limited methodological novelty relative to prior report-guided and multi-teacher distillation work, extremely small gains on the headline captioning and VQA results, and insufficient experimental support for the paper’s strongest claims about superiority and generalization. In addition, the paper does not convincingly show that the complex language-guided distillation pipeline provides clear benefit beyond a strong DINOv3-initialized ViT-B student. Taken together, the current version feels incremental and overclaimed relative to the empirical evidence.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
I appreciate the authors’ clarifications, especially the additional statistical tests and the efficiency framing of the 87M student model. However, the rebuttal does not resolve my main concerns. The data leakage issue remains unclear and potentially more serious: the encoder is described as being trained once on a single fold’s train/val split and then frozen for all downstream folds, which may contaminate other fold test sets unless a fully independent pretraining split is documented. Excluding only test-only keywords is also insufficient; the entire keyword extraction, keyword-bank construction, pseudo-labeling, and encoder training pipeline should be performed strictly within each training fold. The novelty argument is also only partially convincing. The rebuttal narrows the contribution to language-guided per-patch teacher weighting, but multi-expert pathology distillation, pathology vision-language pretraining, structured clinical knowledge injection, and heterogeneous PFM fusion all have strong precedents. Finally, the headline improvements on captioning and VQA remain numerically tiny, and the newly reported significance tests are selective. The work may be an efficient student model with some promising task-specific gains, but the evidence is not sufficient to support the stronger claims or to overcome the methodological and protocol concerns. Therefore, my recommendation remains reject.
Review #3
- Please describe the contribution of the paper
Aiming at the problems that the calculation cost of pathological basic model is too high, the existing multi-teacher distillation can’t adapt to organizational heterogeneity, and it is difficult to judge the quality of teachers only by vision, LaGuadia framework is proposed: firstly, visually observable clinical keywords are extracted from pathological reports as semantic anchor points, then visual features are aligned with clinical semantics through MedSigLIP visual-linguistic meta-teachers, and finally, teachers’ weights are dynamically allocated based on semantic similarity to realize adaptive knowledge distillation.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Put forward a brand-new language guidance paradigm of multi-teacher distillation in pathology, break through the traditional uniformity/confidence weighting, judge teachers’ expertise with clinical keywords in pathological reports as objective semantic benchmarks, and solve the key problem of “who should teach and what to teach” under organizational heterogeneity; At the same time, the visual-language alignment is embedded in the distillation process, so that students can not only learn morphological characteristics, but also learn semantic knowledge consistent with diagnosis, and greatly improve the factual consistency of the generation task.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Teachers choose UNI, GigaPath and Virchow2, which fails to verify the universality and robustness of the teacher combination. 2.Keyword extraction relies on GPT-4o-mini, which lacks the robustness analysis of extraction quality and noise keywords. 3.The training process is complicated in three stages, which makes end-to-end training and engineering deployment difficult, and does not provide quantitative data of efficiency such as reasoning speed and memory occupation.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Please refer to weaknesses.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank all reviewers for their constructive feedback.
Data Leakage [R1Q5,R2Q3] The encoder is trained once on a single fold’s train/val split and frozen for all downstream evaluation. In the global Keyword Bank used in Stages 2 and 3, keywords existing only in each fold’s test split are explicitly excluded. Extracted keywords are universal pathology terminology based on WHO Classification, not TCGA-specific expressions, avoiding cohort-specific overfitting.
Contributions [R2Q5] We agree with the reviewer that the DINOv3 initialized ViT-B is a strong visual backbone. However, the reviewer’s observation that naive KD degrades FactEnt (0.5017 to 0.4988) and Survival (0.5425 to 0.5180) actually demonstrates that uniform distillation harms clinical metrics even from this strong baseline. This does not imply that our strategy is not beneficial, but rather suggests that naive uniform distillation fails and adaptive distillation is necessary. LaGuardia recovers this degradation and improves Survival by +4.18% over DINOv3, succeeding where uniform distillation fails.
Keyword Extraction [R1Q1,R1Q2,R2Q2,R3Q2] Prompt and code will be released upon acceptance. API parameters were set without constraints (temperature/top_p=1, frequency/presence penalty=0). Essential diagnostic elements are clearly described in pathology reports, so core keywords are extracted preferentially, limiting LLM non-determinism impact. Rule-based systems are sensitive to linguistic variations and limited in generalization [10.1016/j.labinv.2025.104272]. Pathology ontologies such as UMLS provide standardized terminology but cannot infer H&E-observable morphological findings from diagnosis names. Our extraction of ‘findings observable in 20x H&E images’ is only possible through LLM. Keyword extraction is limited to 10 observable findings per report to mitigate excessive extraction. Pseudo-labeling selects from these 10 keywords, not the entire bank, providing robustness against keyword noise. GPT-4o-mini is a misstatement of GPT-5-mini, to be corrected in camera-ready.
Novelty [R1Q6,R2Q1] CONCH, MUSK, and KEEP pre-train FMs from scratch on large-scale datasets, whereas our approach distills existing PFMs into a lightweight model. mSTAR self-distills from its own slide aggregator to a patch extractor using WSIs, reports, and gene expression data, and DRE-SLCL performs WSI-level contrastive learning. Among distillation methods, Shazam requires multiple teachers at inference. GPFM uses pre-fixed teacher weights, and FuseCPath, the most similar to ours, fuses heterogeneous FMs via collaborative distillation but without language-guided per-patch teacher selection. In contrast, our methodology adaptively assigns per-patch teacher weights based on clinical keywords, dynamically determining which teacher is most relevant for each patch’s morphological characteristics, a mechanism not addressed in prior work.
Performances [R1Q4,R2Q4] We performed paired t-tests on existing fold results as suggested. For captioning, NLG metric differences are marginal as noted, but the 87M model performs comparably to models up to 13x larger (UNI 675M, Virchow2 631M, GigaPath 1.1B). In VQA, the highest FactEnt was achieved in most cohorts. In MIL, significant improvements include +9.5% over UNI in BRCA Subtype (p=0.031), +10.3% over GigaPath in STAD LAUREN (p=0.030), and +12.0% over MedSigLIP (p=0.002). In Survival, +5.5% over H0-mini (86M) in STAD (p=0.002) demonstrates effectiveness at a comparable scale. No significant inferiority was observed across all MIL tasks. Lower C-index reflects evaluating only patients with available reports (BRCA 973/1098, STAD 315/443, THCA 325/507). Relative comparison under identical conditions is appropriate.
[R3Q1] Teachers were selected from different institutions for diversity in distribution [R3Q3] 3-stage complexity occurs only during training; at inference, only the 87M student is used (4-12x lighter) [R1Q3] CLS token limitations are future work
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Due to the mixed reviews, I recommend inviting the paper for rebuttal. The authors are encouraged to address the major concerns raised by the reviewers. If these issues can be adequately resolved during the rebuttal process, I would consider acceptance, even if the reviewers do not update their scores afterward.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
This paper proposes LaGuardia, a language-guided adaptive distillation framework for compressing pathology foundation models through clinically informed teacher selection. Reviewers appreciated the practical significance of developing lightweight pathology encoders, the novel use of clinical language to guide adaptive multi-teacher distillation, and the comprehensive evaluation across captioning, VQA, survival prediction, and classification tasks. I agree with Reviewers #1 and #3 that the proposed framework is well motivated and demonstrates promising performance while substantially reducing model size. The rebuttal clarified the role of keyword extraction, the distinction from existing pathology-language pretraining and distillation approaches, and provided additional statistical analyses supporting the reported gains. While some concerns remain regarding the complexity of the training pipeline and the evaluation protocol, I believe these issues do not outweigh the overall contribution. Therefore, I find the work technically sound, practically relevant, and worthy of acceptance.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The reviewers generally agreed that the paper addresses an important practical problem in computational pathology, namely distilling large pathology foundation models into a compact and deployable encoder. The proposed language-guided adaptive multi-teacher distillation framework was viewed as technically interesting and clinically meaningful, particularly the use of report-derived semantic guidance for adaptive teacher weighting. Although concerns remain regarding limited novelty relative to prior multimodal distillation work, dependence on LLM-based keyword extraction, and modest gains on some generative tasks, the post-rebuttal discussion overall supports the value of the proposed framework and its practical efficiency advantages. I therefore recommend acceptance.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation.
The most serious concern—potential cross-fold data leakage—was not resolved by the rebuttal and was arguably reinforced by it: training the encoder once on a single fold’s split and freezing it for all downstream evaluation risks contaminating the test splits of other folds, and excluding only test-only keywords does not address this. Combined with the narrowed novelty claim relative to existing multi-expert distillation and clinical-knowledge-injection work, and the marginal, selectively-tested gains on captioning and VQA, the current evidence does not establish that the reported results are reliable. I therefore recommend rejection