List of Papers Browse by Subject Areas Author List
Abstract
Automated surgical phase recognition is challenging across decentralized centers due to domain shifts in equipment and technique, along with strict privacy constraints. While pre-trained Vision–Language Models (VLMs) provide strong semantic priors, existing federated VLM adaptation methods remain ineffective because they treat surgical phases as flat categories, mixing center-specific noise with true phase semantics.
We propose Hi-FL, a Federated Learning framework for Hierarchical Adaptation (HA) of VLMs, which splits procedural logic into three levels: (i) procedure-level global context over the procedure, (ii) phase-level dynamics across stages, and (iii) action-level frame-wise cues. To handle heterogeneity across centers, we also introduce a Class-Conditional Targeted Alignment and Repulsion (TAR) loss that aligns local features to global semantics while minimizing cross-institutional interference.
Evaluated on Cholecystectomy, Colorectal, and Gastric-Bypass procedures, Hi-FL consistently outperforms federated and few-shot baselines, improving accuracy by up to 10% in moderate-data regimes (5%–50% videos per center) and in low-shot settings (16–64 frames). Additionally, it preserves semantic phase structure and merges multi-center distributions, overcoming center-specific noise, making it a practical framework for privacy-preserving, cross-site deployment of VLMs in surgery.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/4826_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/CAMMA-public/Hi-FL
Link to the Dataset(s)
N/A
BibTex
@InProceedings{AleJul_HiFL_MICCAI2026,
author = { Alekseenko, Julia AND Quero, Giuseppe AND Laracca, Giovanni Guglielmo AND Baldari, Ludovica AND Morales-Conde, Salvador AND Mascagni, Pietro AND Mutter, Didier AND Padoy, Nicolas},
title = { { Hi-FL: Hierarchical Federated Learning Adaptation of Vision-Language Models for Multi-Institutional Surgical Phase Recognition } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16892},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This work presents a strategy for surgical phase recognition during laparoscopic procedures based on vision-language models. The proposed approach introduces a novel hierarchical learning adaptation combined with federated optimization, enabling the development of center-specific models within a multi-institutional setting.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
A key strength of this work is the adaptation of vision-language models not only to capture global variability across surgical procedures, but also to enable the development of center-specific models.
Additionally, the paper presents an extensive experimental validation in different type of procedures.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The introduction is relatively long, limiting the space available for presenting experimental results or discuss.
Regarding the datasets, the number of videos per dataset appears to be relatively limited. Probably, dataset with frame-wise temporal labels are scarce, but the authors didn’t mentioned it.
Additionally, Figures 3 and 4 are difficult to interpret due to their small size and low resolution. Some elements of the legends also appear to be cropped, which further reduces their clarity.
An example may be beneficial to illustrate the model’s performance with and without the proposed hierarchical adaptation and federated learning components.
Despite the observed improvements over the baseline, the reported results exhibit noticeable variability. The manuscript does not provide sufficient analysis to explain this behavior.
Finally, the paper lacks a dedicated discussion of its limitations.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The work addresses an important problem and proposes a meaningful approach by combining hierarchical federated learning with the adaptation of vision-language models for multi-institutional surgical phase recognition. In particular, the ability to account for both global and center-specific variability represents a relevant strength.
However, the experimental validation, although extensive, relies on datasets with a relatively small number of videos, which may affect the robustness of the conclusions. Although the proposed approach demonstrates improvements in performance with respect to the baseline, the results appear to be somewhat noisy. Therefore, a more deep discussion is needed to explain the underlying causes of this variability. Furthermore, the manuscript lacks of an explicit discussion of limitations, as well as the absence of commentary on the scarcity of frame-wise annotated datasets.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors have addressed my main concerns by proposing to add a discussion, improve the quality of the plots, and analyze the variability of the results.
Review #2
- Please describe the contribution of the paper
This paper focuses on the multi-institutional surgical phase recognition and proposes the Hi-FL (Hierarchical Federated Learning) framework. By hierarchically adapting Vision-Language Models (VLMs) and introducing a semantic alignment mechanism, it achieves high-precision generalization of cross-institutional surgical phase recognition while protecting data privacy, with particularly outstanding performance in data-scarce scenarios.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper focuses on cross-institutional domain shift. The problem originates from real-world clinical scenarios, and the proposed solution has clear practical value. The main strengths of this work can be summaried from two aspects: a) Innovation of the hierarchical framework. Surgical phase understanding is decomposed into a three-level structure of “Procedure-Phase-Action”, which accurately distinguishes between universal surgical semantics and local institutional noise. This framework solves the limitation of flat categories in traditional methods. b) Solid experimental analysis. Comprehensive comparative experiments and ablation studies are conducted against state-of-the-art federated learning methods.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Although this work holds several major strengths as mentioned above, there also exist several concerns that need to be addressed in the revised version. a) The methodology part is insufficiently clear. The introduction to the core hierarchical mechanism is simply too brief, and specific details of the multi-level structure are not adequately shown in Fig. 1 and Section 3.2.Especially in Fig. 1, which requires careful explanation, as it corresponds to the components in the ablation experiments of Section 5.4.Currently, it is somewhat confusing. b) Insufficient quantitative validation of hierarchical disentanglement. Hierarchical disentanglement is only achieved through the indep loss, but there is no evaluation to confirm whether the three-level features well capture semantics of different granularities. The effectiveness is questionable. c) Comparisons with state-of-the-art federated learning methods for surgical phase recognition are lacking. d) The figures and tables need to be improved. The description of qualitative results in Section 5.2, including the legends, lacks clarity. The paper should emphasize the presentation of its core components, and the writing style can be further adjusted.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper has a complete overall structure with clear main innovations and contributions to federated learning. However, the experimental validation and methodological description as well as writing flows can be further improved.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
Main part of my raised comments is well discussed in the rebuttal, and the authors claim that they will add more details and analysis of methodologies, figures and results. I agree that this work can be accepted in the main conference. Thanks for the efforts of all the authors again.
Review #3
- Please describe the contribution of the paper
Due to the limited amount of data available for learning a generalized surgical understanding model, this paper proposes a federated learning framework for surgical phase recognition that combines hierarchical temporal modeling with semantic alignment across institutions. The core idea is to decompose surgical context into multiple levels of abstraction, such as procedure, phase, and action, to guide representation learning in a more structured manner. The method further incorporates dynamic semantic alignment between visual and textual representations and introduces a centroid-based alignment loss to reduce inter-center variability in the action classes without sharing raw data. Together, these components aim to preserve discriminative local features while improving cross-institutional consistency.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper addresses an inherent problem during the decentralized federated learning framework to improve the consistency across hospitals without violating any data sharing constraints.
Major strengths:
- The centroid component provides a class level alignment signal across centers without sharing raw videos.
- Class-wise centroids are simple proxies for each surgical phase at each hospital. Each site shares one average embedding per class instead of full features or raw data.
- This is a genuinely federated-specific mechanism. Unlike the hierarchical module, the centroid exchange is directly aimed at cross-center heterogeneity.
- The TAR loss is conceptually clean: it pulls a frame embedding toward its correct class centroid and pushes it away from other class centroids.
- This gives more structure than plain cross-entropy, because it encourages class-consistent geometry across institutions, not just correct local labels.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- The contribution of the federated alignment component is not clearly isolated. Although the paper compares against prior federated methods, it does not include a direct ablation of TAR or centroid exchange within the proposed framework. Therefore, it remains unclear how much of the gain comes specifically from the federated semantic alignment term.
- The method description contains several tightly coupled components, and some formulations are not presented clearly enough for easy interpretation. In addition, parts of the notation appear ambiguous or insufficiently explained such as equation (4) in Methodology. Improving the clarity of the method section would strengthen the paper considerably.
- The paper focuses mainly on positive results, but provides little discussion of limitations, such as when the method may fail, how sensitive it is to hyperparameter choices in broader settings, or how well it may generalize beyond the evaluated datasets. A short discussion of limitations and future work would make the paper more balanced.
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, The paper tackles the challenge of surgical phase recognition in a practical multicenter setting, where data are limited and cannot be centrally shared. The idea of aligning frame embeddings with class specific centroids is intuitive and appealing, as it encourages semantic consistency across institutions while preserving privacy, but I believe that the the clarity of presentation could be improved.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
The authors’ response helps clarify the role and potential advantages of TAR within their hierarchical design. However, the additional ablation results mentioned in the rebuttal were not included in the submitted manuscript, so they are difficult to weigh directly in the final assessment based on the rebuttal guidelines. This makes this part of the response somewhat less convincing. In addition, including a discussion of the method’s limitations would make the work feel more balanced and credible.
Author Feedback
We thank the reviewers for their constructive feedback. We will improve Figs. 3&4 for better readability and add FedAvg & non-FL examples (R1Q3,4). Fig.1 will be revised to map each equation/component from Sec. 3.2 & 5.4 (R2Qa,d). We will add a Discussion section (R1Q6, R3Q3) on: dataset scale limits from costly surgical annotation; sensitivity, noting the method assumes primarily phase-distributional heterogeneity; and reliance on expert labels with future directions toward weakly/self-supervised methods. We commit to releasing our codebase. Bypass is public; for the other datasets we will provide preprocessing scripts. R1 Q2: We will shorten Sec.1-2 to create space for Discussion. On dataset scarcity: dense frame-wise surgical annotation requires expert surgeons to label long procedures frame by frame (Maier-Hein et al., MedIA 2022). MultiCholec and Bypass represent the practical upper bound of surgical FL benchmarks, motivating our 5%-50% regime. Q5: Variability arises from multi-center heterogeneity (e.g., Bypass (Lavanchy et al., IJCARS 2024): omentum division routine in Strasbourg but not Bern) and the data-scarce regime (5% data, 16-shot), where small frame changes affect optimization. Federation is a stabilizer: removing it collapses ACC and inflates variance: MultiCholec: Local ~34.1±4.5; FedAvg 40.9±4.8; Hi-FL 51.9±3.6 Colorectal: Local ~25.0±10.2; FedAvg 33.2±2.4; Hi-FL 42.5±2.0 Bypass: Local ~32.1±5.3; FedAvg 43.7±5.2; Hi-FL 47.9±4.8 We will add this analysis. R2 Qa: We will expand Sec.3.2 and redesign Fig.1 linking each stream to its ablation. Four levels (L) operate on xt. (L1, ablated by A2) Global Procedural: causal running mean via bottleneck MLP produces bounded scale vector st. (L2, ablated by A1) Temporal Phase: parallel causal Conv1D capture phase transitions at increasing timescales, yielding additive shift vt. (L3) Action: at = Wactxt, identity-initialized; at fusion at is the operand L1 and L2 act upon (atst + vt), gradients flow through scale/shift jointly, driving Wact to amplify relevant dimensions and suppress irrelevant VLM features. (Ablated by A3) Dynamic Text: meta_net shifts CLIP prototypes conditioned on causal running mean on the classifier side. All levels are combined as zt (defined in Sec.3.2), with gate G controls how strongly the hierarchical correction overrides the frozen VLM. Qb: Disentanglement is quantified per-dataset. Pearson correlation (lower is better) between L1 and L2 (L3 being dependent on them) is lowest for Hi-FL (Colorectal 0.017, MultiCholec 0.025, Bypass 0.024) and rises when components are ablated (up to ~0.1 on average for the datasets), suggests reduced linear dependence between L1 and L2.Removing the independence loss also drops accuracy: 42.5→41.0 on Colorectal, 51.9→50.2 on MultiCholec, 47.9→46.8 on Bypass. Qc: Table 1 compares Hi-FL against five VLM-SOTA methods highlighting our contributions. Most other phase-based FL methods, like FedCy, use vision-only encoders without semantic grounding, unlike VLM-based methods. R3 Q1: TAR contribution is most clearly seen on Bypass, where removing it drops accuracy by 4.1pp (47.9→43.8) and raises Pearson from 0.024 to 0.043.On MultiCholec: Hi-FL 51.9 vs. No-TAR 49.2 (2.7pp drop, Pearson 0.025→0.041). On Colorectal: 42.5 vs. 42.0 (Pearson rises 0.017→0.064), showing TAR improves cross-center representation consistency beyond accuracy. Q2: We will revise the methodology presentation, see R2Qa. We will rewrite Eq.4 with all terms defined: the independence loss penalizes absolute cosine similarity between mean-centered L1 and L2 activations, minimizing this term forces the optimizer to find two solutions that are orthogonal. Q3: Hyperparameter sensitivity (S1-S3) confirms robustness, with <1pp accuracy variation on Colorectal and <2pp on MultiCholec/Bypass across a 10×knowledge-guard range (wkg = 0.01→0.1) and 0.1-0.7 repulsion margin. Extreme settings/procedures remain future work.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper has received 3 bordeline positive reviews. All reviewers agree on the novelties introduces by the hierarchical framework and the broad validation. However all 3 identify key areas for improvement in terms of:
1) Clarifty on methodology details: - Rev #2, 3
2) Further focused analysis of results including a discussion on limitations- all
3) Ablations to highlight the contribution of the key introduced components - Rev#1
4) Unclear reproducability without a commitment for public release of code/models/data. - all
My recommendation is for the paper to be invited for rebuttal
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The authors have done a good job with the rebuttal clarifying all questions posed. 2 reviewers recommend “accept” and 1 reduced their mark to “reject” I believe this is because of the inclusion of the ablation results being deemed inappropriate.
It is my opinion that the ablation studies represent variants of the executed experiments and not new ones.
My recommendation is for the paper to be accepted.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
After rebuttal, two reviewers suggest accepting this paper and one reviewer suggest rejecting this paper. Based on the rebuttal as well as the recommendations of the reviewers, the paper can be accepted.
It is suggested that the authors sufficiently consider the comments raised by reviewers and further improve the paper in the final version.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
All reviewers were satistfied with the rebuttal. The authors should address the remaining concerns in the camera-ready version.