Abstract

Surgical Video Question Answering (VideoQA) provides a promising paradigm for dynamic intraoperative interpretation, enabling real-time decision support and context-aware retrieval in clinical environments. Nevertheless, existing approaches are predominantly restricted to images or short clips, limiting their ability to model long-range procedural dynamics and causal dependencies across extended surgical workflows. To address this challenge, we propose SurgLQA, a unified long-horizon VideoQA framework for scalable surgical reasoning. This framework incorporates Faithful Temporal Consolidation (FTC), which leverages intrinsic temporal cues to construct compact long-range representations while preserving fine-grained temporal fidelity. Further, we develop Temporally-Grounded Multi-Policy Scaling (TMS), an adaptive test-time inference paradigm that strategically adjusts policy-level reasoning capacity within temporally grounded contexts. To facilitate systematic evaluation, we restructured a long-duration colonoscopy VideoQA benchmark, Colon-LQA, and conducted extensive experiments on Colon-LQA and REAL-Colon-VQA. Experimental results demonstrate that our approach achieves consistent performance gains in long-range reasoning with temporally grounded inference. Code link: https://github.com/RascalGdd/SurgLQA.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0680_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: https://papers.miccai.org/miccai-2026/supp/0680_supp.zip

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{GuoDia_SurgLQA_MICCAI2026,
        author = { Guo, Diandian AND Yang, Xikai AND Li, Ruiyang AND Pei, Jialun AND Heng, Pheng-Ann},
        title = { { SurgLQA: Scalable Long-Horizon Surgical Video Question Answering } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16892},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    This paper proposes SurgLQA, a unified framework for long-duration surgical video question answering. It introduces the FTC (Faithful Temporal Consolidation) module, the TMS (Temporally-Grounded Multi-Policy Scaling) mechanism, and constructs the Colon-LQA benchmark for long-duration surgical video QA, thereby advancing research in this direction. These designs are well-founded and clinically meaningful, demonstrating outstanding performance particularly in handling sparse critical events and long-range dependencies.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper proposes the Colon-LQA benchmark for long-duration surgical video question answering. By stitching multiple short video clips into extended sequences and aggregating their corresponding question-answer pairs, the authors directly address the critical gap of lacking suitable evaluation datasets for the task of long-horizon surgical video reasoning.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.While the experiments in this paper are focused on the colonoscopy setting, the work positions itself as a “surgical video question answering. “ It is recommended to discuss the potential for generalizing or transferring​ the proposed method to other surgical modalities (e. g. , gastroscopy, laparoscopy, open surgery) or to acknowledge its potential limitations​ in handling more complex scenarios, such as those involving multiple views or multiple instruments. 2.The paper emphasizes its capability in long-horizon reasoning, yet the experimental analysis lacks an in-depth exploration of this “long-horizon” characteristic. It would be insightful to further investigate how the performance of the proposed method varies as the video duration increases, for instance, by analyzing performance trends across different input lengths.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The proposed FTC module and TMS mechanism demonstrate clear methodological innovation. The idea of adaptively selecting temporal sampling strategies based on queries is particularly clever and aligns well with clinical intuition. Comprehensive ablation experiments and comparisons on existing data have been conducted to validate the effectiveness of the method. However, there is a lack of analysis on how model performance changes as video duration increases.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper introduces SurgLQA, a framework for long-horizon surgical video question answering. It focuses on addressing the limitations of existing methods that rely on short clips or frame-level reasoning. The method consists of two main components: Faithful Temporal Consolidation, which compresses long videos into compact representations while preserving temporal structure, and Temporally-Grounded Multi-Policy Scaling, which adaptively selects sampling strategies based on the query and predicted temporal intervals. The paper also proposes a restructured benchmark, Colon-LQA, by concatenating multiple clips into longer sequences, and demonstrates improvements over prior methods on this and another dataset.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper tackles a relevant and underexplored problem, namely long-horizon reasoning in surgical video QA, which is important for realistic clinical scenarios. The framework is logically structured, with clear separation between temporal compression and adaptive inference. The idea of combining query-conditioned temporal grounding with adaptive sampling is interesting and reasonably novel in this context. The experimental results show consistent improvements across multiple metrics and datasets, including both in-template and out-of-template settings, which suggests some robustness. The ablation studies are fairly thorough and demonstrate the contribution of each component. The scalability analysis is also a strong point, showing improvements in memory and runtime while maintaining accuracy.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The novelty is somewhat incremental, as both temporal compression and query-conditioned temporal grounding have been explored in prior video understanding literature. The paper would benefit from a clearer distinction from related work. The constructed Colon-LQA dataset, although longer than previous settings, still uses relatively short sequences (around tens of seconds), which limits the claim of true long-horizon reasoning. The evaluation protocol, where all models are fine-tuned under the same setting, may not fairly reflect the strengths of models designed for different regimes. There is limited discussion of clinical relevance or real-world deployment, which is important for a MICCAI audience. Some methodological details are insufficiently described, such as how temporal intervals are predicted and how the policy instructor is implemented and trained. The qualitative analysis is limited, with little discussion of failure cases. The writing is generally understandable but contains some awkward phrasing and lacks clarity in parts.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    It would strengthen the paper to provide clearer comparisons with recent long-video modeling approaches and to better justify the novelty of the proposed components. Including experiments on truly long-duration videos or providing evidence that the method scales to such settings would improve the claims. More detailed explanation of the temporal grounding mechanism and policy selection would help reproducibility. Adding failure case analysis would also be valuable.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper addresses an important problem and presents a well-motivated framework with solid empirical improvements. The combination of temporal compression and adaptive sampling is effective and supported by experiments. However, the novelty is moderate, and the evaluation does not fully demonstrate true long-horizon reasoning. Some methodological details are unclear, and the clinical relevance is not deeply explored. Overall, the paper is slightly above the acceptance threshold but not strongly so.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #3

  • Please describe the contribution of the paper

    Scalable Long-Horizon Surgical Video Question Answering

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Strengths: 1.The paper presents a new VideoQA paradigm for long-term horizon in surgery, which is currently missing from AI in this field. FTC and TMS work together for scaling reasoning capability. 2.Proposes new benchmark (Colon-LQA) and achieves impressive results compared with baseline methods.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    Technical Flaws: 1.Limited novelty: FTC has similarities with temporal pooling and feature compression, whereas TMS can be compared with adaptive sampling/attention. 2.Construction of benchmark: Colon-LQA consists of artificially constructed video sequences by concatenation → not representing natural surgical temporal correlation. 3.Temporal grounding is poorly validated: no explicit tests for grounding (only evaluated via QA performance). 4.No statistical test to prove improvement in QA performance. 5.Computational cost is not discussed (no FLOPs, latency, or real-time test). Include all the above mentioned parameters. 6.Prompt-based interval prediction is unstable and biased in TMS. Clinical validation not presented.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    8/10

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Author Feedback

We thank all reviewers for their positive feedback. Here, we respond to and address the remaining concerns:

Reviewer 1: 1.Modality Extension. REAL-Colon is currently the only available dataset with both long surgical videos and fine-grained surgical CoT annotations for this task. Therefore, our current experiments focus on colonoscopy videos. Since FTC and TMS rely on temporal evidence preservation and query-guided adaptive sampling rather than colonoscopy-specific cues, SurgLQA has the potential to extend to other surgical modalities. We will clarify this point in the revision and discuss future extensions to broader modalities, such as gastroscopy and laparoscopic surgery, which will require additional long-horizon annotations.

2.Input-Length Ablation. Fig. 6 already provides an input-length ablation, including memory, runtime, and K-ACC with increasing input frames. We will further extend this analysis to longer surgical videos spanning tens of minutes to better reflect the long-horizon setting.

Reviewer 2: 1.Surgical-Specific Design. SurgLQA is tailored to sparse evidence retrieval in long surgical videos. FTC preserves temporal evidence rather than simply compressing tokens, and TMS performs query-guided interval estimation with question-adaptive multi-policy sampling rather than fixed or generic sampling. We will clarify these distinctions from existing long-video methods.

2.Extended-Duration Evaluation. To further support the long-horizon claim, we will try to evaluate longer surgical videos spanning tens of minutes and report the corresponding performance/runtime trends.

3.Baseline Protocol. By unified training protocol, we mean that all compared models were fine-tuned and evaluated under the same data splits, input resolution, and evaluation metrics. Model-specific hyperparameters followed the official implementations or recommended configurations.

4.Clinical Relevance. The supplementary material included a prototype demo illustrating how SurgLQA can be integrated into practical surgical VideoQA systems. Together with the runtime analysis in Fig. 6, it supports the potential of SurgLQA for real-time clinical use. We will further discuss clinical relevance and deployment feasibility.

5.Additional Clarifications. We will provide additional details on temporal interval prediction, policy selection, and policy instructor training to improve reproducibility. We will also add representative failure cases and further polish the manuscript for clarity.

Reviewer 3: 1.FTC/TMS Design. FTC and TMS are tailored to long-horizon surgical VideoQA. FTC preserves sparse temporal cues through timestamp-token interleaving and fidelity-preserving retrieval loss, while TMS uses query-conditioned temporal grounding and multi-policy re-sampling instead of a fixed sampling strategy. We will make this distinction clearer.

2.Temporal Continuity. Colon-LQA is constructed by concatenating temporally ordered consecutive segments, and transitional segments are preserved to maintain procedural continuity and temporal correlation.

3.Grounding Validation. We will add explicit grounding reliability analysis, such as interval-hit accuracy between predicted query-relevant intervals and evidence regions.

4.Result Stability. We will report mean ± standard deviation over multiple runs and include significance testing in the revision.

5.Efficiency Reporting. Fig. 6 demonstrates the runtime efficiency and scalability of SurgLQA with increasing input frames. We will make this analysis more explicit and further report detailed computational cost statistics, e.g., FLOPs and latency.

6.Interval-Prior Robustness. In TMS, prompt-based interval prediction is used only as a coarse temporal prior rather than hard localization, which reduces sensitivity to minor interval errors. We will evaluate robustness under different prompt settings and further discuss clinical feasibility and deployment limitations in the revision.




Meta-Review

Meta-review #1

  • Your recommendation

    Provisional Accept

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The reviewers generally agree that this paper addresses an important and timely problem in long-horizon surgical video question answering. The proposed SurgLQA framework, including Faithful Temporal Consolidation and Temporally-Grounded Multi-Policy Scaling, is well motivated and shows consistent improvements over baseline methods. The construction of Colon-LQA also provides a useful benchmark for studying longer surgical video reasoning. The experimental results, ablation studies, and scalability analysis support the effectiveness of the proposed design.

    The main concerns are that the novelty is moderate, some components are related to existing temporal compression and adaptive sampling methods, and the current benchmark is constructed by concatenating shorter clips rather than using naturally long surgical videos. Reviewers also noted that the paper would benefit from deeper analysis of performance across different video lengths, stronger validation of temporal grounding, clearer implementation details, statistical testing, and more discussion of clinical generalizability.

    Overall, despite these limitations, the paper tackles a relevant CAI problem, presents a technically sound framework, and provides solid empirical evidence. The review scores are positive overall, with two weak accepts and one accept. I therefore recommend acceptance, preferably as an early accept, while encouraging the authors to clarify the long-horizon claim, strengthen the temporal grounding analysis, and discuss limitations more explicitly in the final version.



back to top