List of Papers Browse by Subject Areas Author List
Abstract
Understanding surgical workflow in real time is fundamental for intelligent surgical embodiment, where AI systems continuously perceive and respond as surgery proceeds. In the operating room, critical decisions depend on subtle, moment-to-moment changes, such as fine instrument movements and evolving tissue states, where even slight perceptual delays can limit assistance or compromise safety. Yet existing methods remain offline or operate at coarse temporal scales, generating descriptions only after processing clips, preventing immediate reaction. We address this by proposing SurgOnAir, a streaming vision-language model that processes frames sequentially without future access and progressively generates narration tokens as visual input arrives.
SurgOnAir achieves fine-grained frame-to-token generation, enabling instant responsiveness to evolving surgical dynamics.
Built upon our curated hierarchical dataset SurgOnAir-11k spanning action-, step-, and phase-level supervision, the model is trained to produce multi-level textual responses that reflect the inherent hierarchy of surgical procedures. Furthermore, special transition tokens are generated to explicitly mark state changes, allowing SurgOnAir to capture and signal key workflow transitions as they occur.
Experiments show that SurgOnAir enables real-time understanding through a single vision-language model that unifies streaming across multiple hierarchies of the surgical workflow, generating superior and hierarchy-aware narrations. The project is publicly available at \url{https://github.com/KarolineZhy/SurgOnAir}.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1882_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: https://papers.miccai.org/miccai-2026/supp/1882_supp.zip
Link to the Code Repository
https://github.com/KarolineZhy/SurgOnAir
Link to the Dataset(s)
N/A
BibTex
@InProceedings{HeJin_SurgOnAir_MICCAI2026,
author = { He, Jingyi AND Zhou, Yue AND Bai, Long AND Yuan, Kun AND Navab, Nassir AND Bi, Yuan},
title = { { SurgOnAir: Hierarchy-Aware Real-Time Surgical Video Commentary } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16892},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The main contribution of this paper is a streaming surgical video narration model that operates sequentially and generates commentary in real time, without relying on future frames or retrospective access to full clips. This is an important shift from the dominant offline setting, since the authors are explicitly targeting temporally grounded understanding for live surgical environments rather than post hoc video interpretation.
A second key contribution is the explicit incorporation of hierarchical surgical workflow structure into the narration process. Instead of treating narration as a flat sequence generation problem, the model is designed to reflect the inherent organization of surgical procedures across phase, step, and action levels. This is conceptually relevant, because surgical workflow understanding is naturally hierarchical and the paper’s framing is that such structure should help real time narration remain contextually coherent.
The paper also contributes a curated dataset, SurgOnAir-11K, constructed through a temporal grounding pipeline that uses ASR, domain aware correction, action aware filtering, and metadata driven hierarchical organization. This dataset creation effort is a meaningful part of the contribution, since the paper’s training setup depends on aligning word level narration with higher level procedural structure.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
A major strength of the paper is that it addresses a timely and important problem, namely real time surgical video understanding, rather than only offline captioning or retrospective analysis. This is relevant for the broader direction of intelligent surgical assistance, where temporally grounded interpretation is necessary if an AI system is expected to respond meaningfully as the procedure unfolds.
Another strength is the dataset curation effort. The construction of SurgOnAir-11K through temporal grounding, ASR correction, action aware filtering, and hierarchical phase and step organization appears substantial. Even if one sets aside the proposed model, this kind of curated hierarchical dataset is itself useful, because temporally aligned supervision is often the bottleneck in training and evaluating real time narration systems in surgery.
A further strength is the modeling idea of using explicit state or transition tokens for hierarchy aware narration. This is a sensible design choice because it attempts to couple continuous language generation with explicit workflow state tracking, rather than relying entirely on the model to recover phase and step structure implicitly from free text.
The experimental design is also a positive aspect. The paper compares against both offline and streaming baselines and includes ablations on hierarchy and the state formulation itself. This helps support the authors’ claim that the hierarchy aware design contributes meaningfully beyond simply fine tuning a streaming backbone on surgical data.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
A major weakness of the paper is the literature review and positioning of related work. Although the manuscript cites both surgical and generic streaming vision language literature, the discussion remains somewhat compressed, especially given the broad novelty framing. The paper would benefit from a more dedicated and balanced related work section that more clearly distinguishes prior surgical captioning, hierarchical surgical video understanding, and general streaming video language models.
Related to this, the novelty should be framed more carefully. The problem is important, and the hierarchical perspective is sensible, but the method itself is built on top of an existing streaming backbone, LiveCC-7B-Base, with the main additions being hierarchical supervision, state transition tokens, and dataset curation. That is still valuable, but it reads more as a task specific extension and adaptation than as a fundamentally new streaming architecture.
Another important weakness is the evaluation design. The primary reported outcome is an LLM as judge pairwise win rate, which on its own is not a sufficiently rigorous assessment of factual correctness, temporal precision, or clinical usefulness. While this kind of evaluation can be informative, the paper would be much stronger if it also included more direct and objective measures of narration quality, temporal alignment, and hierarchical accuracy, ideally complemented by human expert assessment.
The paper also lacks real world validation. Although the stated motivation is real time surgical understanding for intelligent assistance, the evaluation does not demonstrate whether the generated commentary is actually helpful, actionable, or safe in a real operating room setting. As a result, the work currently supports feasibility of the narration framework, but not its practical utility for surgeons, trainees, or robotic systems in the real world.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I lean toward weak accept because the paper addresses a timely and important problem, namely real time surgical video understanding for future intelligent surgical assistance. This is a meaningful direction for MICCAI, and the paper moves beyond standard offline captioning by explicitly targeting streaming narration without future access, which is better aligned with how surgical AI systems would need to function in practice.
The strongest aspects of the paper are the dataset curation effort and the hierarchy aware streaming formulation. The construction of SurgOnAir-11K through temporal grounding, ASR correction, action aware filtering, and hierarchical organization appears substantial and is an important contribution on its own. In addition, the use of explicit state transition tokens to mark phase and step changes is a sensible modeling idea that ties continuous narration to surgical workflow structure in a more principled way than flat caption generation. I also view the experimental section as reasonably supportive. The paper compares against both offline and streaming baselines and includes ablations on hierarchy and state formulation, which helps show that the proposed hierarchy aware design contributes beyond simply fine tuning a generic streaming model on surgical data. This makes the paper more complete than a pure proof of concept.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors have addressed my concern. I think the paper is worth becoming publicly available to the community.
Review #2
- Please describe the contribution of the paper
This work presents a streaming surgical video narration framework based on a multimodal large language model. The method interleaves visual and text tokens over time for real-time generation and uses hierarchy-aware state tokens to represent workflow transitions. In addition, the paper introduces a hierarchical dataset creation pipeline and the resulting dataset, which augment fine-grained action-level narration with higher-level procedural labels. The approach is evaluated against both offline and streaming baselines.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper addresses an important and practically relevant problem, namely real-time surgical video narration rather than retrospective offline analysis.
- A further strength is the hierarchical data construction effort, which may provide a useful resource for future work when released publicly.
- the reported results show clear improvements over the compared baselines, suggesting the value of in-domain surgical data for this task
- The paper also includes comparisons with offline and streaming baselines, along with ablation experiments.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- The main weakness of the paper is limited methodological novelty. The core streaming formulation appears closely related to prior streaming video-language work already cited in the paper as [6], rather than constituting a fundamentally new modeling framework. The hierarchy-related contribution is also less convincing than claimed. Hierarchical supervision, LLM-assisted augmentation, and automated hierarchical data construction for surgical vision-language learning have already been explored in prior works. Therefore, this aspect of the paper appears more incremental than fundamentally new.
- In addition, the proposed higher-level labels appear to be partly LLM-generated and are not sufficiently validated, which weakens the claim of introducing a reliable hierarchical dataset and suggests that this component is closer to weak supervision or pseudo-labeling
- Although the ablation study shows some improvement, the paper does not sufficiently explain why the hierarchy/state-token mechanism is effective or provide deeper analysis of how it improves model behavior beyond the reported gains.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(2) Reject — should be rejected, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper addresses an important and practically relevant problem, namely real-time surgical video narration, and includes a reasonably broad empirical evaluation with baseline comparisons and ablation studies. However, my main concern is limited methodological novelty. The core streaming formulation appears to build substantially on existing streaming video-language approaches, while the hierarchy-aware component is not sufficiently convincing as a major contribution. In particular, the higher-level labels appear to be partly LLM-generated and are not rigorously validated, which weakens the claim of a strong hierarchical contribution. Moreover, although ablation results show some improvement, the paper does not adequately justify why the proposed hierarchy/state-token mechanism is effective beyond the reported gains. For these reasons, I find the paper technically interesting but not sufficiently strong for acceptance in its current form.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
I thank the authors for the clarifications, particularly that phase labels come from expert metadata and only step labels are LLM-generated. That said, my main concerns stand. The methodological novelty still reduces largely to the state-token mechanism on top of LiveCC, and validating LLM-generated labels via agreement with other LLMs is not convincing. The problem and dataset are valuable, but I don’t find the contribution strong enough for acceptance.
Review #3
- Please describe the contribution of the paper
This paper presents SurgOnAir, a streaming surgical vision-language model for real-time video commentary. The authors first construct a hierarchical dataset, SurgOnAir-11K, using temporally grounded ASR from WhisperX, GPT-4o-based domain-aware correction and action-aware filtering, and metadata-based phase/step organization. They then fine-tune LiveCC-7B-Base with explicit
tokens to model hierarchical workflow transitions. The proposed method outperforms LLaVA-Video, Qwen2.5-VL-7B, and LiveCC-7B on pairwise win rate. - Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper addresses an important problem. Streaming, temporally grounded surgical narration is a meaningful direction for surgical AI. 2.The pipeline is conceptually clear and easy to follow, including ASR alignment, terminology correction, action filtering, hierarchical phase/step construction, and streaming fine-tuning with state tokens. 3.The ablation results suggest that the hierarchical formulation is beneficial compared with a flat narration baseline.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The novelty is limited. The proposed framework appears largely similar to LiveCC, except for the introduction of explicit
tokens for phase and step transitions. In addition, the hierarchical aspect is not new, as HecVL has already constructed hierarchical surgical video-text pairs across multiple granularity levels. 2.The real-time/streaming claim is not sufficiently supported. The paper does not report latency, throughput, memory usage, FLOPs, or long-sequence efficiency, which are important for validating real-time capability. 3.The evaluation metric is too limited. The main results rely only on the GPT-4o-as-a-judge pairwise win rate. There is no standard captioning metrics such as BLEU, CIDEr, METEOR, and ROUGE-L, and no objective workflow metrics such as phase/step accuracy or transition localization. 4.The data construction pipeline is under-documented. Key stages, including domain-aware ASR correction, action-aware filtering, and step generation, all depend on GPT-4o, but the prompts, output formats, and post-processing details are not clearly described. 5.The effect of upstream errors is unclear. The paper does not analyze how the quality of temporally grounded ASR and ACTION/EXPLANATION/INTERACTION classification affects the final dataset and downstream performance. 6.What is the detailed distribution of samples across the different procedural categories? 7.The baseline comparison could be fairer and stronger. It would be more convincing to fine-tune LLaVA-Video-7B and Qwen2.5-VL (e.g., low-rank adaptation for their LLMs only) on the same SurgOnAir training set. If possible, compare with more recent, stronger open-source MLLMs such as Qwen3-VL. 8.There are citation inconsistencies. The paper discusses Qwen2.5-VL-7B, but the reference list points to a Qwen3-VL technical report. Similarly, the method uses GPT-4o, but the cited reference is the GPT-4 technical report. - Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I appreciate the relevance of the problem and the clarity of the proposed pipeline. The paper has a reasonable technical story, and the direction is worthwhile. However, the current version overstates its novelty relative to prior streaming video-language work and prior hierarchical video-language modeling, while the evaluation is not yet strong enough to support the central claims. The manuscript would be substantially improved by more careful positioning, stronger and more objective evaluation, and explicit reporting of real-time efficiency and upstream errors’ influence.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
I remain concerned that the proposed method is a task adaptation of LiveCC+HecVL, with limited methodological novelty beyond the introduction of explicit state tokens for phase and step transitions. The evaluation is also insufficiently objective, as it relies solely on LLM-as-a-judge metrics rather than on objective metrics, raising concerns about the reliability of the method. The explanation for not fine-tuning stronger open-source baselines is not fully convincing, since these models could be evaluated under the same causal or sliding-window streaming setting. In addition, the effect of upstream errors from ASR correction and action filtering remains under-specified. Overall, the rebuttal only partially addresses my concerns regarding novelty, evaluation reliability, baseline fairness, and error propagation.
Review #4
- Please describe the contribution of the paper
The paper proposes SurgOnAir, a model for real-time surgical video commentary. It can process video as a stream and generate text without seeing future frames. Its main contribution is a hierarchy-aware design that models surgical phases, steps, and actions together. The paper also builds a new dataset, SurgOnAir-11K, with temporally aligned and hierarchical annotations. Experiments show that this approach gives better real-time surgical narration than prior offline and streaming methods.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The dataset is very practical and matches real surgical needs. It is built for temporally aligned and real-time understanding, which makes it more meaningful than standard offline settings. 2.The method fits the task well. It processes video as a stream and does not need future frames, so it can support real-time surgical assistance. 3.The hierarchical design is a strong point. Modeling phases, steps, and actions is natural for surgery and helps the model understand workflow more clearly. 4.The transition token design is interesting. It lets the model explicitly capture workflow changes instead of only generating flat narration. 5.The evaluation is convincing. The paper compares with both offline and streaming baselines and includes ablations to show why the method works.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The method is only partly novel. It mainly builds on prior streaming video LLM frameworks such as LiveCC, VideoLLM-Online, and StreamingVLM, and the main new part is the hierarchical state modeling. 2.The evaluation is limited. The paper mainly uses GPT-4o win rate, but does not provide enough task-specific metrics such as phase accuracy, step accuracy, or real latency analysis. 3.The benefit of the hierarchical design is not fully analyzed. Although the ablation shows gains, the paper does not deeply explain which level of hierarchy contributes most, or in which surgical scenarios the hierarchy is especially helpful.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is novel in introducing a hierarchy-aware streaming formulation that fits surgical workflows well. The dataset is also a meaningful contribution, since temporally aligned and hierarchical surgical data are valuable for the community. Finally, the proposed method is well motivated and shows clear empirical improvements, so the paper is both technically interesting and application-oriented.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank reviewers for the positive feedback. We appreciate reviewers recognizing the importance of surgical video streaming. We also appreciate that R1 recognize our SurgOnAir-11K dataset as “substantial,” and that R2 note its potential as “a useful resource for future work.” Moreover, R1, R3, and R4 acknowledge our hierarchical design. Finally, we thank R1, R2, and R4 for positively noting our experiments and ablations.
Contribution clarification(R1/R2/R3/R4): Our contributions are threefold. Application: We present the first real-time surgical narration framework for intraoperative assistance, enabling online responses instead of retrospective full-video analysis (e.g., LLaVA-Surg, SurgVidLM). Methodology: We unify continuous narration and discrete hierarchical state transition via special state tokens. This achieves simultaneous narration and workflow recognition, unlike prior works (e.g., LiveCC, HecVL) that address them separately. Dataset: We introduce SurgOnAir-11K, the first real-time narration dataset, providing temporally aligned supervision for streaming surgical video narration.
State token for hierarchical modeling(R2/R3/R4): As R1 notes, surgical workflows are inherently hierarchical. Unlike HecVL’s discriminative vision-language alignment, our generative model targets live narration (R3). Flat narration may produce locally plausible but globally inconsistent descriptions over long streams. To address this, we formulate phase/step transition prediction as an auxiliary task. This injects hierarchical knowledge to boost narration coherence, yielding 20% gains over the flat baseline.
Caption metrics(R3): In surgical narration, both semantic correctness and temporal alignment are critical. To assess narration quality, we employ an LLM-as-a-judge evaluation. This approach serves as a robust alternative to metrics like BLEU, as it accommodates greater expressive diversity while better capturing clinical semantics. An analysis of our temporal alignment performance is shown below.
Temporal alignment evaluation (R1/R3): We analyze 421 evaluation clips containing tool and action transitions. A prediction is temporally correct if the new tool or action appears in the generated narration within 1.5 seconds after the transition. Under this criterion, SurgOnAir correctly identifies 68.4% of clips, compared with 27.3% for HuluMed-7B [7], which also tends to produce less specific narrations during transitions. These results suggest that hierarchy-aware modeling improves temporal adaptation by providing procedural context during streaming generation.
Fine-tune baseline models (R3) : We thank the reviewer for the suggestion. We clarify that LLaVA-Video-7B and Qwen2.5-VL are included only as reference general-purpose offline MLLMs, not as direct baselines. They are used to show that general models struggle with surgical video understanding. Since they are offline captioning models, fine-tuning them would not demonstrate live narration ability or causal streaming understanding.
LLM-Generated labels (R2): The phase annotations are from expert-provided video metadata, while only the step-level labels are LLM-generated. To provide the requested evaluation on the generated step annotations, we report high cross-model agreement rates, including 82.69% agreement with Claude Sonnet 4 and 90.5% agreement with Gemini 3.1 Pro. The generated hierarchy therefore provides higher-level procedural context for temporally grounded narration, rather than workflow labels.
Latency(R4): Under the 2 FPS streaming setting, HuluMed-7B [7] requires on average 1.94 seconds to process and generate narration for 1 second of video, whereas SurgOnAir requires only 0.95 seconds, making it substantially better aligned with real-time streaming requirements.
We will also add procedural distributions, curation prompts, and fix citation inconsistencies (R3) in revised manuscript. To our knowledge, this is the first work on real-time surgical narration.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The rebuttal should address three decision-affecting concerns. First, methodological positioning: the method is built on the LiveCC streaming backbone, with the main contributions being hierarchical supervision, state-transition tokens, and dataset curation; the rebuttal should clarify the differentiation from the LiveCC formulation and from prior hierarchical surgical video-language work such as HecVL. Second, the evaluation relies primarily on a GPT-4o pairwise win rate, with no standard captioning metrics and no objective workflow metrics such as phase or step accuracy or transition localization; the rebuttal should add these. Third, the real-time and streaming claim is not currently substantiated by latency, throughput, memory, or FLOPs analysis, and the LLaVA-Video and Qwen2.5-VL baselines are evaluated zero-shot rather than fine-tuned on SurgOnAir-11K, which weakens the comparison.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The paper proposes a streaming vision-language model for real-time surgical narration, built on the LiveCC backbone and incorporating hierarchical supervision, state-transition tokens, and a new SurgOnAir-11K dataset. The reviewers split evenly after the rebuttal. The principal concerns are methodological novelty, reliance on a pairwise LLM-as-a-judge metric without standard captioning or workflow metrics, and the streaming claim, which is not currently substantiated by efficiency reporting. The rebuttal addresses these concerns by introducing a temporal alignment metric, clarifying that phase labels come from expert metadata while only step labels are LLM-assisted, and committing to latency and FLOPs analysis. The dataset, the explicit framing of real-time narration as a research problem in surgery, and the integration of phase and step supervision with state tokens together constitute a recognizable contribution. The two post-rebuttal Accepts are well argued. The camera-ready version must include the efficiency analysis, at least one standard captioning metric, and the LLM-as-a-judge results.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Overall, I believe the paper’s strengths (timely real-time surgical narration problem, substantial dataset curation, and hierarchy-aware streaming formulation with state-transition tokens) outweigh its limitations (moderate architectural novelty, dependence on LLM-as-judge evaluation). Although, the latter critique is rather serious, to my mind. Given the solid rebuttal, I think this borderline case could become a good poster at the main track and I recommend acceptance.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation.
After the rebuttal, reviewer’s recommendation remain mixed with some key concerns unresolved.
