Abstract

Organ segmentation from PET/CT is critical for quantitative analysis and radiotherapy planning in oncology. To ease the high annotation cost for PET/CT annotation, semi-supervised learning provides a practical and effective solution for developing deep models with limited labeled data. Recent developments in visual foundation models have demonstrated remarkable adaptability with improved efficiency. In this work, we propose a mutual distillation framework to seamlessly exploit both structural and functional foundation models, which act as modality-specific generalists for distillation of knowledge from structural CT and metabolic PET imaging. By bridging the gap between the task-specific precision of specialist models and the segmentation priors of different generalist foundation models, we propose a mutual distillation framework MuDuo that synergistically exploits structural and functional foundation models SAM-Med3D for CT and SegAnyPET for PET to mutually distill knowledge to a lightweight student network. Our approach eliminates the need for manual prompts while maximizing the utility of unlabeled data for automatic segmentation, achieving state-of-the-art performance on the AutoPET dataset with only 5 labeled cases.Our source code is available at https://anonymous.4open.science/r/MuDuo/

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0353_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/Wubeining/MuDuo

Link to the Dataset(s)

AutoPET: https://www.autopet.org/

BibTex

@InProceedings{MaoFuy_Mutual_MICCAI2026,
        author = { Mao, Fuyou AND Wu, Beining AND Jiang, Yanfeng AND Xu, Bohan AND Lin, Lixin AND Ji, Naye AND Zhang, Hao AND Tang, Yan},
        title = { { Mutual Distillation of Dual-Foundation Models for Semi-supervised PET/CT Segmentation } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16880},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    This paper proposes a mutual distillation framework (MuDuo) that synergistically distill both structural and functional knowledge from structural CT and metabolic PET images via two different foundation models to a lightweight student network for PET/CT organ segmentation. In addition, an IoU-based consensus filtering mechanism was introduced to exclude unreliable predictions and ensure high-quality pseudo supervision.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper is well-organized and easy to follow. The research problem of the anatomical-functional mismatch and cross-modal information fusion in semi-supervised segmentation is well-motivated. The mutual distillation framework that synergistically exploits structural and functional foundation models is somewhat innovative. Segmentation experiments contained the proposed MuDuo and several existing semi-supervised methods in three different data configurations. Ablation studies were implemented to validate the effectiveness of multiple components in the proposed MuDuo.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    It is natural that the combination of multiple foundation models yields superior segmentation performance, as demonstated by previous semi-supervised learning studies. Therefore, the paper has limited technical novelty. However, demonstrating that the combination of structural and functional foundation models leads to a better model, as shown in their experiments, provides valuable insights to the community. Segmentation experiments are conducted solely on an outdated U-Net backbone, leaving it unclear whether the observed improvements would hold for other architectures, such as transformers. The majority of the semi-supervised methods adopted are relatively dated, and only one is derived from the foundation model SAM.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper addresses the organ segmentation in whole-body PET/CT leveraging both structural and functional foundation model priors to guide learning of a lightweight student network. However, given that both the U-Net backbone and the comparative semi-supervised methods are outdated, the demonstrated effectiveness of the proposed method remains unconvincing. The paper could significantly benefit from experiments showcasing the effectiveness of the technique across different segmentation models, particularly in comparison to more recent foundation model-based methods.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    I believe the authors’ rebuttal is acceptable, and the paper can be published in the MICCAI conference.



Review #2

  • Please describe the contribution of the paper

    This paper proposes MuDuo, a mutual distillation framework for semi-supervised PET/CT organ segmentation. Two frozen foundation models, SAM-Med3D for CT and SegAnyPET for PET, serve as modality-specific teachers that distill knowledge into a lightweight student U-Net. The student’s predictions serve as prompts to generate pseudo-labels, which are filtered by IoU-based consensus and fused via dynamic mixing for consistency regularization. Extensive evaluation on AutoPET with 5/10/20 labeled cases.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.Strong motivation for dual-modality foundation models. Exploiting modality-specific foundation models (SAM-Med3D for structural CT, SegAnyPET for functional PET) is a natural and well-justified design for PET/CT segmentation. This goes beyond single-modality SAM-guided SSL. 2.The cross-modality IoU filtering is a principled quality control mechanism. The ablation confirms its importance. 3.The method outperforms all baselines across all label settings. The anonymous code link is provided, supporting reproducibility.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The title should be condensed to “PET/CT Organ Segmentation”, given that this article focuses on organ segmentation rather than the original tumor segmentation task of AutoPET. The corresponding statement of the article should also be clarified 2.Even the best result (49.56% Dice with 20 labels) is relatively low for organ segmentation. The paper should discuss why PET organ segmentation is so challenging. 3.The author state that “We allocate N cases for the training set and the remaining100 - N cases for the test set.” It is not reasonable to put the experiments on these three different sets of data in the table together. Besides, the results are reported as averages across all organs. Per-organ performance would reveal whether the method benefits uniformly or is biased toward certain organs. 4.Ablation study is not comprehensive. How about K=2 or more than 3? It is suggested to make a supplement to reflect the trade-off between performance and prompting efficiency.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    A well-structured framework that naturally extends foundation model-guided SSL to dual-modality PET/CT. The cross-modality consensus mechanism is effective. Despite there are still some limitations refer to weakness, they could be strengthened through rebuttal revision.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #3

  • Please describe the contribution of the paper

    The paper targets semi-supervised 3D organ segmentation on PET/CT, a clinically important task with high annotation cost. The authors propose MuDuo, a mutual distillation framework that uses two frozen modality-specialized foundation models: SAM-Med3D for structural CT (mask prompts) and SegAnyPET for functional PET (multi-point prompts). The method generates high-quality pseudo-labels via cross-modality IoU consensus filtering and fuses them to supervise a lightweight student U-Net. Experiments on AutoPET show state-of-the-art performance under extremely low-label regimes (5/10/20 labeled cases), outperforming SemiSAM+, MT, CPS, and fully-supervised baselines.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The task of PET/CT segmentation is clinically meaningful and important. 2.The method is built upon existing state-of-the-art foundation models.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The main contribution lies in extending foundation-model-assisted semi-supervised segmentation from a single-modality to a dual-modality PET/CT setting, incorporating dual teachers, IoU-based filtering, and pseudo-label fusion. However, the methodological novelty remains moderate. 2.The paper lacks a clear description of how the prompt generator is trained. Specifically, what data and supervision signals are used, and whether the generator is frozen or jointly optimized during training. As this module is central to pseudo-label generation for both teachers, this omission undermines reproducibility. 3.Several hyper-parameters are introduced, such as the IoU threshold for pseudo-label filtering and the number of point prompts. The manuscript does not clearly explain how these values were determined, which limits the practical applicability of the method. 4.The experimental validation is not thorough. Additional analyses (e.g., sensitivity studies, ablation on key components, or evaluation on more datasets) would be needed to support the claims.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall, the methodological contribution is limited. The experimental results are promising but not comprehensive.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The authors have made detailed responses to my raised comments. I believe they have done their best to improve the work.



Author Feedback

We sincerely thank all reviewers for their constructive comments. We especially appreciate R3’s concerns on novelty, reproducibility, hyper-parameter justification, and validation scope. We agree that several key designs should be clarified and will revise the manuscript accordingly.

Regarding novelty, MuDuo should not be positioned as a new generic SSL paradigm. Its precise contribution is a PET/CT-specific dual-teacher semi-supervised framework for organ segmentation. Rather than simply combining two foundation models, MuDuo queries modality-specialized teachers in complementary ways: SAM-Med3D uses CT-side mask prompts to exploit anatomical priors, while SegAnyPET uses PET-side point prompts to exploit metabolic uptake priors. Their outputs are selected by cross-modal IoU consensus and fused as pseudo labels for consistency learning. We will delimit this contribution and avoid over-generalized claims.

We apologize for the unclear description of the prompt generator. In our implementation, it is a lightweight U-Net trained on the labeled subset with organ annotations before the semi-supervised stage. During mutual distillation, the prompt generator, SAM-Med3D, and SegAnyPET are frozen, and only the student model is optimized. For CT, it produces class probability maps as mask prompts for SAM-Med3D. For PET, it produces coarse logits; each predicted organ region is smoothed, and the top-K foreground locations are sampled as point prompts for SegAnyPET. We will add details on its supervision, freezing strategy, and calling order.

Regarding hyper-parameters, MuDuo does not rely on a manually tuned absolute IoU threshold. The fixed tau=0.5 is only an ablation baseline. Our final strategy is adaptive top-50% IoU filtering, where CT/PET predictions are ranked by cross-modal agreement and only the most reliable half is retained. This is more robust to organ size and case difficulty. Table 2 supports this design: adaptive top-50% improves Dice and reduces HD95 over both no filtering and fixed tau=0.5.K=3 is motivated by heterogeneous PET uptake, where one point may cover only the high-uptake core. Table 2 also shows that multi-point prompting improves both Dice and HD95 over K=1. Regarding validation, we agree that broader evaluation would further strengthen the work. Still, the manuscript includes 5/10/20 low-label settings, PET-only and PET/CT baselines, four metrics, qualitative comparisons, and ablations on teacher configuration, IoU filtering, and prompting. We will clarify that these ablations verify structural-functional complementarity, improved pseudo-label reliability, and better PET-teacher guidance. We will also state the evidence scope: the experiments are based on an organ-annotated AutoPET subset and a 3D U-Net student. Although MuDuo is conceptually compatible with other backbones, we will not claim empirical generalization beyond the reported experiments.

We thank R2 for the comments on title, task description, data split, and absolute Dice. We will revise the title and task description to focus on PET/CT organ segmentation rather than the original AutoPET lesion segmentation task. We will clarify that methods are compared under the same protocol within each label setting, and avoid implying identical test sets across 5/10/20 settings if this is not the case. We will also discuss why the absolute Dice is modest: whole-body PET organ segmentation is challenging due to low PET resolution, blurred boundaries, heterogeneous uptake, uneven organ visibility, and PET/CT spatial mismatch. Thus, our main evidence lies in consistent relative improvements and boundary-metric gains under low-label regimes.

Finally, we accept R1’s concern regarding backbone and baseline scope. We will moderate the related claims: MuDuo shows that cross-modal foundation-model consensus improves semi-supervised PET/CT organ segmentation under the current U-Net student setting, but we will not overstate gains across all architectures.




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The paper is borderline (scores 3–5), with reviewers agreeing the idea is solid and results are promising, but pointing to a few fixable issues. The rebuttal should clarify the true novelty beyond combining two foundation models, strengthen the experimental evidence (e.g., generalization beyond U-Net and comparison to stronger baselines), address evaluation gaps (limited ablations, missing per-organ results, dataset setup), and improve method clarity (prompt generator, hyperparameters, training details). Overall, the feedback is constructive, and since at least one reviewer conditions their decision on the rebuttal, a focused response could tip the paper toward acceptance.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    Following the rebuttal, all reviewers support acceptance of the paper. The Area Chair concurs with this recommendation and supports the paper’s acceptance.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    All reviewers recommended a clear acceptance of this work after the rebuttal. The major contribution lies in the integration of both structural and functional foundation models. Experiments have demonstrated the effectiveness of model designs, and the rebuttal response clarifies some issues. The authors are encouraged to address all reviewer comments in the camera-ready version.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The proposed dual-modality mutual distillation framework (MuDuo) is recognized as a clinically meaningful and technically sound approach to semi-supervised PET/CT organ segmentation. While initial concerns were raised regarding the moderate technical novelty, backbone generalization, and specific parameter settings, consensus among reviewers post-rebuttal highlights that the authors have successfully resolved these queries through detailed empirical clarifications.



back to top