List of Papers Browse by Subject Areas Author List
Abstract
Video polyp segmentation (VPS) is an important task in computer-aided colonoscopy, as it helps doctors accurately locate and track polyps during examinations. However, VPS remains challenging because polyps often look similar to surrounding mucosa, leading to weak semantic discrimination. In addition, large changes in polyp position and scale across video frames make stable and accurate segmentation difficult.
To address these challenges, we propose a robust VPS framework named CMSA-Net. The proposed network introduces a Causal Multi-scale Aggregation (CMA) module to effectively gather semantic information from multiple historical frames at different scales. By using causal attention, CMA ensures that temporal feature propagation follows strict time order, which helps reduce noise and improve feature reliability. Furthermore, we design a Dynamic Multi-source Reference (DMR) strategy that adaptively selects informative and reliable reference frames based on semantic separability and prediction confidence. This strategy provides strong multi-frame guidance while keeping the model efficient for real-time inference.
Extensive experiments on the SUN-SEG dataset demonstrate that CMSA-Net achieves state-of-the-art performance, offering a favorable balance between segmentation accuracy and real-time clinical applicability. The code is available at https://github.com/wangtong627/CMSA-Net.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1232_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/wangtong627/CMSA-Net
Link to the Dataset(s)
SUN-SEG: https://github.com/GewelsJI/VPS
BibTex
@InProceedings{WanTon_CMSANet_MICCAI2026,
author = { Wang, Tong AND Qi, Yaolei AND Wang, Siwen AND Razzak, Imran AND Yang, Guanyu AND Xie, Yutong},
title = { { CMSA-Net: Causal Multi-scale Aggregation with Adaptive Multi-source Reference for Video Polyp Segmentation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16883},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper’s main contribution is a real-time video polyp segmentation framework (CMSA-Net) that improves robustness under low contrast and large inter-frame motion/scale changes by combining two key ideas:
1.Causal Multi-scale Aggregation (CMA): a spatio-temporal attention module that lets the current frame aggregate information from multiple spatial scales of reference and past adjacent frames under a causal (time-ordered) attention constraint, aiming to reduce feature contamination and improve discriminability.
2.Dynamic Multi-source Reference (DMR): an efficient online reference-frame maintenance strategy that keeps a compact set of reference frames and updates them adaptively based on (i) foreground–background semantic separability (via prototype similarity) and (ii) prediction confidence (entropy), providing stronger and more reliable guidance than a fixed single reference without the overhead of large memory banks.
Together, CMA+DMR deliver SOTA accuracy on SUN-SEG while maintaining real-time inference speed, supported by ablations showing the two components are complementary.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Well-motivated problem framing: Directly targets key VPS pain points—low contrast (weak semantics) and large inter-frame motion/scale changes.
CMA is a solid technical contribution: Multi-scale feature aggregation with a causal (time-ordered) attention constraint to reduce cross-frame feature contamination.
DMR is practical and efficient: Dynamically maintains multiple reference frames using separability + confidence cues, avoiding brittle fixed references and heavy memory banks.
Convincing empirical support: Comprehensive SUN-SEG evaluation (Easy/Hard, Seen/Unseen) with clear ablations validating each design choice.
Real-time orientation: Reports complexity/speed and designs for clinical feasibility rather than purely offline accuracy.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
DMR can be brittle because it is self-scored from the model’s own predictions. Both “semantic separability” (prototype distance via masked pooling) and “confidence” (entropy) are derived from p_t. This risks selecting wrong-but-confident frames as references and amplifying errors over time; the paper lacks drift/stability analysis.
The “causal / real-time” story is not fully pinned down by the evaluation protocol. The method is presented as causal and suitable for real-time use, but experiments are described on fixed clips (e. g. , 6 frames). The paper should explicitly define a strict streaming inference setup (no future access under any sliding window) and report latency/long-sequence behavior.
Insufficient implementation detail for reproducibility. CMA’s attention tokenization and tensor shapes (how multi-scale features are concatenated/flattened, exact masks, Q/K/V formation) and DMR’s update mechanics are not specified to the level needed for easy reimplementation.
Writing/typos and notation issues reduce clarity. “Casual” appears where “Causal” is intended, and there are minor formula/notation inconsistencies (e. g. , the attention scaling term is written in a nonstandard way). These don’t change the idea, but they hurt readability and confidence in exact implementation.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Pros: The CMA (multi-scale + causal temporal attention) and DMR (dynamic multi-source reference updating) are well-motivated for VPS under low contrast and large inter-frame variation. Results on SUN-SEG are strong across Easy/Hard and Seen/Unseen, and ablations show CMA and DMR are both important. Efficiency reporting (GFLOPs/FPS) supports the real-time angle.
Cons (why not higher): DMR is scored from the model’s own predictions (prototypes/entropy), which may create a feedback loop (e.g., “wrong-but-confident” references); the paper lacks stability/drift analysis. The “causal/real-time” evaluation protocol is not fully specified as strict streaming with per-frame latency/long-sequence behavior. Minor writing/notation issues (e.g., “Casual” vs “Causal”) reduce clarity/reproducibility.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
This paper proposes CMSA-Net for video polyp segmentation, which combines a Causal Multi-scale Aggregation (CMA) module with a Dynamic Multi-source Reference (DMR) strategy. The main contribution is a temporally causal and multi-scale feature aggregation mechanism that improves current-frame segmentation using reference and adjacent frames, together with an adaptive reference update strategy based on semantic separability and prediction confidence. The method is validated on the SUN-SEG benchmark and shows consistent performance gains with competitive real-time inference efficiency.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper addresses an important and clinically relevant problem, namely video polyp segmentation under weak semantic contrast, large spatio-temporal variation, and real-time constraints. 2.The proposed method is technically coherent: CMA and DMR are complementary, with CMA enhancing temporally causal multi-scale aggregation and DMR improving the quality of reference information. 3.The experimental evaluation is relatively solid, including comparisons with multiple prior methods, ablation studies, and efficiency analysis on different SUN-SEG subsets (easy/hard, seen/unseen). 4.The paper demonstrates a favorable accuracy-efficiency trade-off, which is valuable for practical deployment in colonoscopy scenarios.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The novelty appears moderate rather than highly significant. While the method is effective, it can be viewed as a structured extension of existing VPS designs that already exploit temporal aggregation, reference frames, or long-/short-term interaction, such as PNS/PNS+, SALI, MAST, and STDDNet. 2.The empirical evidence is mainly based on a single benchmark (SUN-SEG). Therefore, claims regarding robustness and generalization should be stated more cautiously. 3.Some methodological details remain unclear and affect reproducibility, including the exact handling and number of references in DMR, whether dynamic updates are applied during training and/or inference, and several notation inconsistencies in the paper. 4.Although the method consistently improves over prior baselines, some gains over the strongest same-backbone baselines are relatively modest on certain subsets, so the empirical advantage is solid but not overwhelmingly large.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I recommend a weak accept. The paper tackles an important and clinically relevant task, and the proposed CMA + DMR design is technically coherent and supported by reasonably solid experiments, including comparisons, ablations, and efficiency analysis on SUN-SEG. The method appears effective and practically relevant due to its favorable real-time performance. My reservation is that the novelty is moderate rather than highly significant, and the empirical support for robustness/generalization is still limited by evaluation on a single benchmark. In addition, some methodological details and notation inconsistencies reduce clarity and reproducibility. Overall, I believe the paper is above the acceptance threshold, but mainly as a solid and useful contribution rather than a highly impactful one.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This paper presents CMSA-Net, a neural network architecture designed for video-based polyp segmentation in colonoscopy sequences. The authors introduce dedicated modules for multi-frame and multi-scale feature fusion, aiming to address challenges such as weak semantic discrimination and significant spatio-temporal variation in endoscopic videos. The proposed method is evaluated on two datasets and compared against 14 existing approaches. Experimental results indicate consistent improvements across all reported metrics relative to competing methods.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper provides a clear and detailed description of the proposed network architecture, including its individual modules and the rationale for their design. This level of transparency makes the method easy to follow and reproducible, and helps the reader understand how the different components contribute to addressing spatio-temporal challenges in video-based polyp segmentation.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Limited Novelty in Multi-Scale Feature Fusion The proposed multi-scale feature fusion module offers limited novelty, as similar approaches have been extensively studied in prior work. Moreover, the design appears relatively simple compared to more recent and advanced fusion strategies, some of which have demonstrated superior performance.
2.Marginal Performance Gains over Recent Methods Despite extensive comparisons with existing approaches, the reported improvements are incremental, particularly when compared to recent methods such as STDDNet (ICCV 2025). Additionally, results in Table 3 indicate that GELOP achieves higher performance than the proposed method (R) on certain metrics, which weakens the claim of consistent superiority.
3.Missing Ablation on Loss Components While the paper includes ablation studies on architectural modules, it lacks a detailed analysis of the loss design. There is no ablation isolating the contribution of individual loss components, either for the segmentation loss or for the overall objective, making it difficult to assess their effectiveness.
4.Inconsistency Between Quantitative and Qualitative Results The qualitative results do not convincingly reflect the quantitative improvements reported in the paper, particularly compared with STDDNet. The visual examples fail to clearly demonstrate the claimed performance gains, raising concerns about the consistency of the evaluation.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
An important missing component is a detailed ablation study on the loss design. In particular, the paper would benefit from an analysis isolating the contribution of each loss component, both within the segmentation loss and the overall training objective. This would help clarify the role of the optimization strategy in the reported performance gains.
I encourage the authors to include such an ablation study in the rebuttal to strengthen the paper and improve the clarity of their contributions.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We sincerely thank all reviewers for their constructive comments and for recognizing the importance of video polyp segmentation, the coherence of CMA+DMR, and the solid SUN-SEG evaluation. We address the main concerns below.
Novelty (R2,R3). We agree that temporal aggregation, reference frames, and multi-scale features have been explored in VPS. Our novelty lies in their causal and role-aware integration. Existing methods often rely on fixed/single references or aggregate frames without explicitly modeling the causal relations among reference, adjacent, and current frames, which may cause feature contamination or unreliable guidance under large variations (Fig. 1). CMSA-Net addresses this via (1) CMA, which performs multi-scale aggregation under a time-ordered attention constraint, and (2) DMR, which maintains compact multi-source references using complementary separability and confidence cues. Table 4 shows that removing CMA or DMR leads to clear degradation, especially on Hard-Unseen, supporting their complementary roles.
DMR stability (R1). We appreciate the concern about self-reinforcement. DMR includes several safeguards. First, it uses two complementary cues (separability and confidence) rather than a single score. Second, both scores incorporate temporal consistency, preventing selection based solely on entropy or prototype distance. Third, cooldown intervals limit frequent updates. Finally, DMR maintains a compact reference set instead of accumulating all historical frames, which constrains error propagation. The strong Hard-Unseen results and the “w/o DMR” ablation in Table 4 indicate that DMR improves robustness in the submitted evaluation.
Causal/real-time setting (R1). CMSA-Net is causal by design. In Sec. 2.1, an adjacent frame at time t only attends to reference frames and past frames up to t; the current frame uses only reference and past adjacent frames within the window. No future information beyond the prediction target is used. The 6-frame clip is the implementation of this causal window rather than an offline setting. Table 3 reports 38/47 FPS with PVTv2-B2/Res2Net-50, supporting real-time feasibility.
Single benchmark (R2). We agree that more datasets would strengthen generalization claims. Nevertheless, SUN-SEG is the largest and most widely used VPS benchmark, with 49,136 annotated frames and diverse clinical conditions (low contrast, fast motion, illumination changes, instruments, etc.). It is also the standard benchmark used by representative works such as PNS+, SALI, MAST, and STDDNet. Thus, we believe SUN-SEG provides a rigorous basis for evaluation.
Magnitude of gains (R2,R3). We agree that gains over strong baselines (e.g., STDDNet) are sometimes moderate. However, consistent improvements across subsets and metrics remain meaningful. More importantly, CMSA-Net shows clearer advantages in difficult settings (Hard-Seen/Hard-Unseen and challenging qualitative cases), which directly correspond to the targeted clinical challenges.
Loss ablation (R3). Our contribution focuses on architecture rather than loss design. We adopt standard segmentation losses (Dice, weighted IoU, weighted BCE), consistent with prior VPS works. Thus, improvements mainly reflect the effect of CMA and DMR, and Table 4 already isolates the main architectural components.
Reproducibility and notation (R1,R2). We acknowledge that some details and notation could be clearer. The manuscript specifies multi-scale features, Q/K/V construction, attention heads, clip length, and DMR cooldown intervals. We will correct typos (e.g., “Casual” → “Causal”) in the camera-ready version and release code and models upon acceptance.
We thank the reviewers again for their valuable feedback.
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This submission proposes CMSA-Net, a video polyp segmentation framework that leverages causal multi-scale aggregation to model temporal dependencies and improve semantic discrimination across frames. Three reviewers (R1, R2, R3) vote for weak accept. The work is above the acceptance threshold and is suitable for acceptance. That said, several important issues should be addressed in the final manuscript to strengthen clarity, rigor, and reproducibility: R1 raises a meaningful concern that DMR relies on model predictions, which may introduce a feedback loop where incorrect but confident predictions are reinforced. R2 notes that evaluation is limited to a single dataset and novelty is moderate. R3 highlights the lack of detailed ablation studies on the loss function.