Abstract

Temporal grounding in surgical videos is essential for clinical applications, yet this area remains significantly underexplored. Furthermore, existing methods designed for general-domain videos struggle to generalize to the surgical domain due to two unique challenges. First, surgical procedures exhibit a complex inherent hierarchy, ranging from coarse phases to fine-grained actions. Second, processing long surgical videos in discrete chunks disrupts continuous context, leading to temporal isolation. To address these issues, we formally introduce the research problem of Surgical Video Temporal Grounding (SurgVTG) and formulate a novel task of Hierarchical Multi-Segment SurgVTG (HMS-SurgVTG), accompanied by a comprehensive benchmark. To tackle this task, we propose a Memory-Augmented Vision-Language Model framework that mitigates cross-chunk isolation through an explicit read-write memory loop. By dynamically updating and injecting structured procedural memories via lightweight heads, our framework seamlessly preserves historical context during scalable inference. Extensive experiments demonstrate that our approach enhances long-range temporal grounding quality, establishing a new benchmark for surgical video temporal grounding. Code is available at https://github.com/Mr-Kill/Surgic alVideoTemporalGrounding.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/4572_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/Mr-Kill/SurgicalVideoTemporalGrounding

Link to the Dataset(s)

N/A

BibTex

@InProceedings{LiuQix_Surgical_MICCAI2026,
        author = { Liu, Qixuan AND Qiu, Shi AND Wu, Xiwen AND Xie, Binzhu AND Fu, Chi-Wing AND Heng, Pheng-Ann},
        title = { { Surgical Video Temporal Grounding } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16892},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    1.Problem formulation: First formal definition of Surgical Video Temporal Grounding (SurgVTG), with a hierarchical extension (HMS-SurgVTG) that distinguishes Phase/Instrument/Triplet levels, clinically motivated and well-structured.

    2.Benchmark construction: A reproducible pipeline converting CholecT50 frame-level labels into hierarchical temporal segments, yielding 2,666 queries with clear evaluation protocols.

    3.Method proposal: A memory-augmented VLM framework with an explicit read-write loop to propagate procedural context across video chunks, plus a pragmatic three-stage training strategy.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.Clinically relevant problem: Surgical videos have unique hierarchical structure and long temporal dependencies. Formalizing SurgVTG fills a clear gap and provides a useful benchmark for the community.

    2.Memory loop design: The idea of writing structured state (phase/instrument/triplet) after each chunk and reading it for the next is intuitive and addresses a real challenge in long-video understanding.

    3.Honest bottleneck analysis: The oracle experiment (Table 1) transparently shows that Phase-level grounding is bottlenecked by memory prediction quality—a valuable diagnostic for future work.

    4.Reproducibility-friendly: Benchmark construction, query templates, training details, and loss design are described clearly enough for replication even without code.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.Core methodological concern: Chunk-wise processing ≠ end-to-end long-video modeling The framework processes videos in 64s chunks with an autoregressive memory loop. This is a pragmatic engineering choice, but it is not true end-to-end long-range modeling. Table 1 reveals a critical gap: with ground-truth memory, Phase mIoU reaches 67.67%; with predicted memory, it drops to 19.23%—even below the Qwen3 baseline (21.33%). This strongly suggests that memory prediction errors accumulate across chunks, corrupting the search space for later segments. Phase-level grounding, which most depends on long-horizon context, suffers the most. The current memory design (2 phase slots + boundary existence flags) may be too aggressive a compression for stable macro-state tracking.

    2.Baseline comparisons could be more comprehensive Comparisons focus on generic VLMs (Qwen3, UniVTG). Surgery-specific temporal methods (e. g. , Trans-SVNet, TecNo, Rendezvous) are not included or discussed. Even without new experiments, a brief discussion on why a VLM+memory approach is preferable to adapting established surgical pipelines would help position the contribution. Long-video VTG methods cited in related work (TimeChat, StreamingVLM, Timesuite) are not compared or contrasted architecturally.

    3.Template-dependent queries limit generalization claims Queries are generated from templates that explicitly inject parent-level context (e. g. , “During the X phase, the Y is used”). This makes the task closer to “conditioned retrieval” than “hierarchical reasoning”. It remains unclear how the model would perform on free-form queries (e. g. , “Find all grasper usage segments”). A brief discussion on expected generalization to open-vocabulary queries would strengthen the paper.

    4.Cross-chunk segment merging is underspecified How are predictions from adjacent chunks reconciled when an event spans a chunk boundary? Is there overlap handling, smoothing, or state continuity constraints? Clarifying the post-processing logic would improve reproducibility and deployment readiness.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper tackles a clinically meaningful problem and proposes a thoughtful memory-augmented architecture. The benchmark construction is rigorous, and the three-stage training strategy is pragmatic. However, I recommend Weak Reject for the following reasons:

    1.Architectural bottleneck: The chunk-wise + autoregressive memory design shows clear signs of error accumulation, especially for Phase-level grounding. Without stronger error control or a hybrid architecture, the core claim of “seamless context preservation” is not fully supported.

    2.Necessity not fully established: In a closed-vocabulary, highly structured surgical workflow, it remains unclear whether a large VLM+memory approach is preferable to a lighter pipeline (e.g., specialist models + rule-based constraints). More discussion or comparison would help justify the design choice.

    3.Claim-evidence gap: Phrases like “significantly enhances” and “mitigates hallucination” are not backed by quantitative metrics, which may lead readers to overestimate the method’s maturity.

    If the authors can:

    • Acknowledge the chunk-wise approximation and discuss error propagation more explicitly
    • Temper overstatements and align claims with empirical results
    • Add discussion on template dependency, generalization to free-form queries, and deployment considerations I would be happy to reconsider my score. In its current form, however, I believe the paper requires moderate revisions to meet MICCAI’s standards for methodological rigor and claim precision.
  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    The paper introduces Surgical Video Temporal Grounding (SurgVTG), a new research problem focused on localizing temporal segments (surgical phases, instruments, and action triplets) within long-form surgical videos based on natural-language queries. To facilitate this, the authors convert frame-level annotations from the CholecT50 dataset into a hierarchical time-segment benchmark. To address the challenges of long-range dependencies and “temporal isolation” inherent in chunk-based video processing, they propose a Memory-Augmented Vision-Language Model (VLM) framework. This framework utilizes a read-write memory loop to maintain and inject procedural context across video chunks.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1- New research problem formulation: The transition from fully supervised, closed-world recognition (phase/action classification) to a grounding-based approach is interesting. By leveraging the built-in knowledge and reasoning capabilities of Large Language Models, the paper pushes the field toward more flexible, open-world surgical video analysis, allowing for more intuitive and interactive clinical queries.

    2- Interesting insights on domain gap: The comparison between the proposed surgical model and a general-domain temporal grounding specialist (UniVTG) in a zero-shot setting provides an interesting insight into the domain gap.

    3- Insightful architectural ablations: The ablation studies regarding the memory loop are interesting. The observation that macro-level phase grounding is heavily dependent on memory, whereas fine-grained instrument and action grounding are less so, provides a genuine scientific insight into the different nature of temporal dependencies across surgical hierarchies.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1- Insufficient Motivation for the grounding task: While the paper introduces SurgVTG as a “novel research problem” in contribution 1, it fails to provide a compelling argument as to why temporal grounding is necessary or superior to existing SOTA phase and action models. The authors do not explicitly address why a natural-language grounding approach is more beneficial than simply applying timestamp filtering or post-processing to the output of a high-performing closed-world phase/action model.

    2- Limited Benchmark Contribution: The creation of the HMS-SurgVTG benchmark is essentially a data-reformatting exercise on one single dataset (CholecT50). Converting frame-level labels to segments, filtering short clips (<2s), and using templates for query generation are relatively simple preprocessing steps.

    3- Absence of Task Visualization: Despite claiming to introduce a new research problem and curating the new HMS-SurgVTG benchmark, the paper is missing a high-level concept figure illustrating the SurgVTG task itself (e.g., the flow from a natural language query to a temporal segment in a video). The only provided figure in the paper focuses on the model architecture, which serves as a baseline for the task rather than a definition of the problem.

    4- Methodological gap in baseline comparisons: There is an inconsistency in the evaluation of baselines. The authors compare their fine-tuned model against a general VLM (Qwen3) that was also fine-tuned on the HMS-SurgVTG dataset, but compare it against the temporal grounding specialist (UniVTG) in a zero-shot setting. Could the authors please clarify why the specialist UniVTG was not fine-tuned on the HMS-SurgVTG dataset as well? The zero-shot results are interesting to show the domain gap, but a fine-tuned UniVTG would show the direct comparison between the specialist architectures.

  • Please rate the clarity and organization of this paper

    Poor

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper serves as an interesting “Proof of Concept,” demonstrating that the reasoning capabilities of VLMs can be applied to surgical temporal grounding. The technical contribution, specifically the memory-augmented loop and the resulting insights into the grounding performance that degrades the longer the temporal segment (phase vs action triplet) are interesting.

    However, the paper’s presentation does not currently live up to its contribution of “introducing a novel research problem.” The motivation is underdeveloped, the benchmark is limited in scope (relying on a single reformatted dataset), and the conceptual framing is missing critical visualizations. I would recommend the authors to address the following points: 1.Explicitly motivate the shift from closed-world classification to open-world grounding. 2.Provide a conceptual overview of the SurgVTG task. 3.Clarify the missing fine-tuned UniVTG baseline results

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    I appreciate the clarification provided in the rebuttal. Following the authors commitment to enhancing the presentation, specifically by clarifying the motivation for temporal grounding and detailing how this addresses the ‘novel research problem’ with an improved overview figure, I recommend acceptance. The inclusion of fine-tuned UniVTG results enhances the transparency of the findings. Overall, the paper provides new insights, open sourcing the code and annotations will facilitate a comparison with established closed-world vision models. Additionally, the utilization of temporal grounding metrics that consider top-1 grounded segments provides an interesting evaluation comparison to standard multi-label metrics.



Review #3

  • Please describe the contribution of the paper

    The paper introduces a new hierarchical temporal grounding task for surgical videos, builds a dedicated benchmark, and proposes a memory-augmented VLM framework to model cross-chunk temporal dependencies.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.Construct a new benchmark for surgical video, and hierarchical query generation is a meaningful addition. 2.The results show the introduced method achieves good performance on fine-grained tasks.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The motivation of this paper is not convincing. The claim that hierarchical workflow structure is unique to surgical videos is not convincing. Similar multi-level structures (e.g., stage → tool → action) are common in many procedural domains such as cooking, daily activities, and instructional videos. Therefore, the proposed hierarchy appears to be a general property of procedural videos rather than a surgical-specific characteristic. 2.Limited modeling novelty: The method mainly combines a VLM (Qwen3-VL) with LoRA and a memory mechanism, which is conceptually similar to existing long-video/streaming VLM approaches. The paper lacks clear differentiation and positioning with respect to prior memory-based methods. 3.The benchmark section is underdeveloped. The paper does not present concrete examples, detailed construction procedures, or comparisons with existing surgical datasets. This is a significant limitation, as such information is essential for validating the quality and novelty of a newly proposed benchmark. 4.The paper lacks comparisons with recent VTG and long-video VLM methods (e.g., TimeChat, TimeSuite, StreamingVLM), as well as surgical-domain baselines, making the evaluation incomplete.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The claimed hierarchical structure is not unique to surgical videos but common in many procedural tasks. The modeling novelty is limited, as the method largely builds on existing VLMs with standard memory mechanisms. The benchmark is insufficiently presented, lacking examples, construction details, and comparisons with existing datasets. Finally, the evaluation is incomplete, with too few and insufficiently strong baselines, especially in video temporal grounding.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Reject

  • [Post rebuttal] Please justify your final decision from above.

    The rebuttal addresses several experimental concerns, but I still find the paper somewhat overclaiming its “open-world” and “open-vocabulary” capabilities, since the benchmark is fundamentally built from closed-taxonomy annotations with template-generated hierarchical queries. Therefore, the proposed dataset is closer to a reformulation of existing closed-world surgical recognition tasks into natural-language query formats, rather than a genuinely open-ended temporal grounding benchmark. In addition, the paper still lacks sufficient qualitative examples, detailed benchmark visualization, and concrete evidence regarding dataset diversity and openness, making the benchmark contribution and conceptual framing appear stronger than what is currently supported by the presented dataset construction pipeline.



Review #4

  • Please describe the contribution of the paper

    The authors redefine the recognition tasks of phases, surgical instruments, and triplets in surgical video analysis as a hierarchical video temporal grounding task, and propose a new Memory-Augmented Vision-Language Model framework to solve this task.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The authors redefine the tasks of phase, surgical instrument, and triplet recognition in surgical video analysis as a hierarchical video temporal grounding task, and propose a new Memory-Augmented Vision-Language Model framework to solve this task.
    • In their experiments, the authors compared their method with Qwen3 and UniVTG. While their method underperformed Qwen3 in phase recognition, it demonstrated higher performance than both methods in instrument recognition and triplet recognition.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • While the experimental results in this paper use metrics such as mIoU and recall, accuracy, F1 score, and mAP are generally used for phase recognition and similar tasks. Therefore, it is unclear why non-standard evaluation metrics are being used in this paper.
    • While the experiments compare the proposed method with VLM as a conventional approach, several conventional methods that use video alone have been proposed for phase recognition, instrument recognition, and triplet recognition. Therefore, without a comparison to these methods, it is impossible to determine whether the performance on these tasks is superior to that of conventional methods.
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
    • The authors redefine the tasks of phase, surgical instrument, and triplet recognition in surgical video analysis as a hierarchical video temporal grounding task, and propose a new Memory-Augmented Vision-Language Model framework to solve this task.
    • In the experiments, comparisons were made only with VLM as the baseline method; no comparisons were conducted with conventional video-based phase recognition, instrument recognition, or triplet recognition methods. Consequently, it is not possible to determine whether the performance on these tasks is superior to that of conventional methods.
  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    My primary concern was that the experimental results regarding phase recognition, instrument recognition, and triplet recognition did not include comparisons with conventional video-based methods. However, the rebuttal stated that while phase recognition and instrument recognition are inferior to video-based methods, triplet recognition is superior to them, and that these findings would be included in the revised document; therefore, I consider this concern to have been addressed.



Author Feedback

We thank AC and reviewers for constructive comments. SurgVTG is the first open-vocabulary, compositional temporal grounding framework for hour-long surgical videos—a paradigm shift beyond closed-taxonomy specialists, providing timestamped foundation that LLM-driven clinical AI needs. We will release all code and data upon publication.

[R1&R2&R3: Motivation] (1) Open-world queries: existing specialists are confined to fixed closed-world taxonomies; SurgVTG can directly query out-of-taxonomy concepts, e.g., “skeletonization of cystic pedicle,” an implicit composition of hook dissection of cystic duct+artery, achieving leading mIoU of 58.30, showing SurgVTG’s great adaptation capability to new procedures without redesigning or retraining. (2) Top-1 temporal grounding: triplet recognition is multi-label over ~100 co-occurring classes per frame, so specialists report mAP and top-N accuracy. SurgVTG outputs top-1 grounded segments, better matching clinical usage like report and surgery review. (3) Compositional queries: logical combinations (e.g., “segments during calot triangle dissection where grasper is active but hook is not”) require per-case rule engineering under specialist + timestamp pipelines, but are easy single queries under VTG with comparable or even better outcomes. (4) Grounded evidence: temporal grounding can provide timestamped evidence to reduce LLM hallucination for medical decision-making.

[R1&R2&R3&R4: Baseline comparison] Phase specialists TeCNO/Trans-SVNet achieve higher phase acc. than ours; but triplet Rendezvous is lower than ours on triplets; across tasks, these specialists are about 35%-50% worse than ours overall. Fine-tuned UniVTG’s phase acc. is higher but overall 57.9% lower than ours. We have tested all suggested models, and specific exp results will be added in Tab. 1. [R1: Memory error accumulation] Chunk-wise processing is practically feasible for hour-long surgical videos. It keeps GPU memory bounded and scales naturally to any length of procedures. It does not alter problem domain and autoregressive memory loop remains our key design for end-to-end long-video reasoning. We’ll provide this deployment discussion in revision. We acknowledge that phase-level predictions accumulate errors across chunks. Memory matters more for Phase because same low-level operations often recur in different chunks across distinct phases. Accumulated error also stems from inherent imbalanced phase lengths in surgical videos. Our light phase prediction head inevitably over-predicts the majority phases; errors then propagate through the memory loop. We’ll explicitly discuss this issue in revision. As future work, we plan to use an imbalance-aware, uncertainty-calibrated prediction head to avoid the issue.

[R1: Template robustness] On VID08, queries sampled from highly paraphrased templates yield near-stable performance (~2.1% avg relative change), showing our robustness to free-form queries given VLM’s open-vocabulary capability.

[R1: Cross-chunk merging] We use non-overlapping chunks for efficiency, but cross-chunk context is propagated through our memory-loop design. During training, boundary-spanning events are cropped to chunks; at inference, we concatenate cross-chunk predictions. We will clarify this procedure in the revision.

[R1: Overstated claims] We’ll revise overstated language to better align with our experimental evidence. Thanks for reminding.

[R2: Conceptual overview] We’ll update Fig.1 to provide a conceptual overview of our task. Thanks for this suggestion.

[R3: Hierarchy & Benchmark] Hierarchy in surgical video is uniquely demanding: (1) visually similar actions carry different clinical meaning across phases; (2) ordering is safety-critical and irreversible. Benchmark section will be more detailed and the dataset/code will be released.

[R4: Metrics] Acc/F1/mAP suit fixed-taxonomy classification or multi-predictions; mIoU and R@k are standard VTG metrics for top-1 grounded segments.




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The rebuttal should address four decision-affecting concerns. First, the motivation for shifting from established closed-world phase, instrument, and triplet recognition to natural-language grounding, with a clear argument for why this is preferable to timestamp filtering applied to strong specialist models. Second, the gap in Table 1 between oracle memory and predicted memory: Phase mIoU drops from 67.67 to 19.23 and falls below the Qwen3 baseline at 21.33, and the rebuttal must explain how cross-chunk error accumulation can be controlled. Third, the differentiation of the proposed memory loop from prior long-video and streaming VLM approaches such as TimeChat, TimeSuite, and StreamingVLM, together with comparisons to surgery-specific temporal models for phase, instrument, and triplet recognition. Fourth, the use of standard recognition metrics (accuracy, F1, mAP) alongside mIoU and recall, and a tempering of overstated claims such as “seamless context preservation,” “significantly enhances,” and “mitigates hallucination” so that they align with the empirical evidence.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The paper formalizes the surgical video temporal grounding task and its hierarchical multi-segment variant, introduces a benchmark, and proposes a memory-augmented vision-language framework. The key concerns were the motivation for natural-language grounding relative to timestamp filtering by phase or triplet specialists, the gap between oracle-memory and predicted-memory performance at the phase level, the differentiation of the memory loop from prior streaming and long-video VLM approaches, and the absence of standard recognition metrics. The rebuttal articulates a coherent motivation centered on open-vocabulary queries, top-1 grounded segments for clinical workflows, compositional queries, and grounded evidence for downstream LLMs; provides task-specific comparisons against established phase, instrument, and triplet specialists; acknowledges the cross-chunk error-accumulation problem at the phase level and discusses it as a known limitation. Two careful reviewers endorsed acceptance after the rebuttal. The remaining Reject is reasonable, but the contribution to the field is meaningful. The camera-ready should temper the strongest claims and explicitly discuss the cross-chunk error accumulation.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Reject

  • Please justify your recommendation.

    After rebuttal, three out of four reviewers update the recommendations. There are two Accept and one Reject. R1 did not update. After reading the reviews and author rebuttal, I think the current shape of the paper is not mature. Major concerns include limited novelty, insufficient baseline comaprisons, lack of visualization analysis. Therefore, I incline to Reject.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The rebuttal addressed critical concerns. The paper presents an interesting and novel methodology and has sufficient evaluation.



back to top