List of Papers Browse by Subject Areas Author List
Abstract
Accurate instrument tracking and segmentation is essential for robot-assisted surgery, where failures pose direct threats to patient safety. Current methods focus on dealing with the large intra-class variations of different instruments, neglecting the critical process of instrument’s abrupt disappearance and reappearance, which significantly misleads the predictions of advanced models. We identify two critical challenges hinders the addressing of these issues: instrument misidentification due to abrupt positional shifts and redundant memory caused by long-term instrument absence. To address these two challenges, we present bi-directional motion reasoning with dynamic memory pruning for surgical video segmentation (BRAIN). The bi-directional motion reasoning module first detects abrupt positional shifts from forward and backward trajectories and corrects errors via a mask reselection mechanism. Then, a dynamic memory pruning module maintains a compact yet diverse memory bank by filtering out redundant frames captured during instrument absence. Extensive experiments demonstrate that our framework significantly outperforms existing state-of-the-art methods by 5.27% and 7.28% on EndoVis 2018 and 2017 benchmarks, respectively.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/5179_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/Chuanzhen-Xu/brain-training
Link to the Dataset(s)
N/A
BibTex
@InProceedings{XuChu_BRAIN_MICCAI2026,
author = { Xu, Chuanzhen AND Li, Shumeng AND Zhang, Jian AND Shi, Yinghuan},
title = { { BRAIN: Bi-directional Motion Reasoning with Dynamic Memory Pruning for Surgical Video Segmentation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16892},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper addresses two important challenges in surgical video segmentation: abrupt positional shifts and long-term instrument absence. By explicitly modeling these issues, it offers a meaningful and practically relevant perspective on improving robustness in real-world surgical scenarios.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper is motivated by realistic challenges in surgical videos, which are indeed common in clinical surgical navigation settings. Addressing these issues enhances the practical relevance of the work.
2.The proposed framework is clearly organised, with complementary modules (BMR and DM) that target different aspects of the problem. The overall design is coherent and easy to follow.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The paper does not engage sufficiently with recent related work in surgical video segmentation using large foundation models, such as EndoMamba: An Efficient Foundation Model for Endoscopic Videos via Hierarchical Pre-training, Memory-Augmented SAM2 for Training-Free Surgical Video Segmentation, and ReSurgSAM2: Referring Segment Anything in Surgical Video via Credible Long-term Tracking (all MICCAI 2025). This weak positioning makes the claimed novelty less convincing. Also, the limited experimental comparison (in Table 1) with recent related methods further reduces confidence in the paper’s contribution relative to the current state of the art.
2.The method involves several key hyperparameters (e. g. , thresholds for motion consistency, weighting factors, and memory pruning criteria), but the paper does not provide systematic ablation or sensitivity analysis. This makes it difficult to assess the robustness and reproducibility of the design choices.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Although the work addresses a relevant problem and shows promising results, the novelty would need to be better established through stronger engagement with recent related work, especially foundation-model-based methods in surgical video segmentation. The current work also lacks a systematic sensitivity analysis for several key hyperparameters, which makes the robustness of the proposed design harder to assess.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
The response is not substantial enough to change my assessment. I still think the paper does not engage sufficiently with recent work. Therefore, I will maintain my score.
Review #2
- Please describe the contribution of the paper
The paper proposes a framework (BRAIN) to address abrupt positional shifts and long-term instrument absence in surgical video segmentation. It introduces a Bidirectional Motion Reasoning (BMR) module for motion consistency modelling and a Dynamic Memory (DM) pruning module to maintain representative memory entries.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The use of bidirectional tracking (Kalman filtering combined with RTS smoothing) and memory diversity-based pruning provides a reasonable and interpretable design. The method demonstrates consistent and significant improvements over SAM2 and related baselines on EndoVis benchmarks.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The method involves multiple critical hyperparameters, including τ, γ, k, λ, L, and β. However, only β is studied in the ablation (Fig. 4), while all other parameters are fixed without sensitivity analysis. The lack of systematic analysis raises concerns about robustness and reproducibility. 2.The appearance score used in Motion-Guided Mask Reselection is not clearly compared to standard feature matching approaches in ReID literature. Its differences and potential advantages are not sufficiently explained. 3.When motion inconsistency is detected, especially under abrupt shifts or long-term absence, the memory bank itself may contain corrupted, occluded, or misidentified entries. In such cases, relying on appearance-based reselection may reinforce existing errors rather than correct them. This critical dependency is not analyzed or validated. 4.It is unclear which prompting strategy (among those described in Table 1) is used for the results reported in Table 2.5.In the BMR module, the contribution of bidirectional reasoning is not isolated. In particular, the effect of RTS backward smoothing is not independently evaluated, making it unclear whether the gains come from forward Kalman tracking or backward smoothing. 6.Although BRAIN introduces the BMR module on top of SAM2, the paper does not analyze the additional computational cost or complexity, which is important for practical deployment.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper presents a framework with improvements over SAM2.However, the method involves multiple hyperparameters without sensitivity analysis, and key components are not clearly justified. The contribution of forward and backward components not evaluated separately. In addition, some experimental details and computational cost are not clearly reported.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors have adequately addressed most of the concerns raised and improved the explanation of several technical components. Therefore, I support acceptance.
Review #3
- Please describe the contribution of the paper
This paper targets a specific failure mode in surgical video segmentation: instrument disappearance and reappearance. The authors identify two concrete problems: (1) motion inertia where models keep tracking the last known location and latch onto wrong instruments after abrupt shifts, and (2) redundant memory where memory banks fill with empty or occluded frames during long absences, degrading re-identification.
Their solution, BRAIN (Bidirectional motion Reasoning with dynAmic memory prunIng for surgical video segmentatioN), combines:
- Bidirectional Motion Reasoning (BMR): Forward Kalman filtering plus backward RTS smoothing to detect motion inconsistencies (abrupt shifts), with a mask reselection mechanism that switches to appearance-based matching when motion is unreliable
- Dynamic Memory Pruning (DM): Dissimilarity-based pruning of redundant memory frames to maintain a compact, diverse memory bank instead of SAM2’s fixed recent-frame buffer
They evaluate on EndoVis17/18, showing improvements over SAM2 and SurgSAM2 baselines, particularly for instruments with high absence rates. This is a surgical-specific adaptation of SAM2’s video segmentation framework, adding classical tracking (Kalman filter) and memory management to handle occlusion and reappearance better than generic video segmentation.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The problem formulation is precise and clinically grounded. Unlike vague claims about “surgical challenges,” the authors define concrete phenomena: motion inertia (Fig. 1) and instrument absence quantified via IAR (Instrument Absence Ratio). The safety argument—that misidentification could cause unintended robotic manipulation—is specific and credible. Fig. 2 showing baseline collapse at IAR > 60% is compelling evidence that current methods fail exactly where they matter most.
2.The bidirectional motion reasoning is elegant and well-motivated. Using RTS backward smoothing to detect inconsistencies with forward predictions is a classic signal processing technique [19], but the application to surgical instrument tracking is thoughtful. The Mahalanobis distance-based consistency score (J_t) provides a principled way to detect “teleports” or abrupt re-entries. The dynamic weighting scheme, which decays motion trust as a function of absence duration, is intuitively motivated: an instrument unseen for 50 frames should not be trusted for motion predictions.
3.The memory pruning mechanism addresses a real SAM2 limitation. SAM2’s fixed-length memory bank (initial frame + 6 recent frames) is known to struggle with long occlusions. The dissimilarity-based pruning with minimum retention policy (always keep reference frame, max 6 additional frames) is simple but effective. Fig. 6’s visualization showing memory bank composition under different settings is genuinely helpful for understanding why this matters.
4.Strong empirical results on challenging classes. Table 2 shows massive improvements on historically difficult categories: Clip Applier (+20.62%: 45.29% → 65.91%) and Ultrasound Probe (+35.63%: 25.62% → 61.25%). These are instruments that frequently disappear and reappear, so the gains directly validate the method’s design intent. The ablation in Fig. 7 showing BMR and DM are complementary.
5.Honest about building on SAM2 rather than claiming novelty from scratch. The paper clearly positions itself as extending SAM2 and SurgSAM2, not replacing them.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The core components are largely classical, not novel. Kalman filtering for object tracking dates to the 1960s, RTS smoothing is from 1965, and memory management in visual tracking has been extensively studied (e.g., SAMURAI uses motion-aware memory, MoSAM does spatial-temporal memory selection). The specific combination is new, but the individual pieces are well-established. The paper cites some related work but doesn’t clearly articulate what BRAIN does differently from these concurrent SAM2 adaptations.
2.The “dynamic” memory pruning is less dynamic than claimed. The dissimilarity-based selection (Sec. 2.2) uses a fixed threshold β=6 and fixed maximum memory size (6 frames). This is essentially static pruning with a dissimilarity heuristic, not truly adaptive to scene complexity. A more dynamic approach might adjust memory capacity based on estimated occlusion duration or instrument motion predictability. The ablation in Fig. 4 shows performance vs. β but doesn’t explain why β=6 is optimal or how to set it without validation data.
3.Missing comparison to non-SAM2 surgical trackers. Table 2 compares BRAIN to task-specific methods (ISINet, TraSeTR, S3Net, MATIS, LACOSTE) and SAM-based approaches, but these are mostly single-frame or category-level methods. There’s no comparison to modern surgical video tracking methods that explicitly handle occlusion, such as TraSeTR’s temporal consistency or LACOSTE’s stereo-temporal fusion. The claim that BRAIN “surpasses strong task-specific models” is partially true but conflates different problem settings (instance-level vs. category-level, video vs. single-frame).
4.The appearance score during reselection is underspecified. When J_t ≥ τ (motion inconsistent), BRAIN switches to appearance matching using “cosine similarity between the embedding vector extracted from M_{t,i} and that of the most confident mask in the memory bank.” But if the instrument has been absent for 100 frames, the memory bank contains only old embeddings. How does appearance matching work when the instrument reappears with different orientation, lighting, or occlusion? The paper doesn’t analyze failure modes where appearance changes drastically during absence.
5.Limited analysis of IAR-specific performance. The paper defines IAR and shows baseline collapse (Fig. 2), but doesn’t report BRAIN’s performance stratified by IAR bins. Does BRAIN maintain 80%+ accuracy at IAR > 60%? The abstract claims “significant outperformance” but without IAR-stratified results, we can’t verify that the gains come specifically from handling high-absence cases versus general improvement.
6.The evaluation protocol has gaps. They “exclude the first and last frames of each sequence” following standard protocol, but surgical videos often have the most instrument entry/exit at boundaries. This exclusion might artificially inflate performance. Also, using SurgSAM2’s instance-level annotation protocol for evaluation is fine, but the comparison to category-level methods in Table 2 is unfair—these methods weren’t designed for instance tracking.
7.No discussion of computational cost. Adding Kalman filtering, RTS smoothing, and dissimilarity computation to SAM2’s per-frame inference adds overhead. Is BRAIN real-time? SAM2 is already computationally heavy; the extra modules might push it beyond practical deployment limits for robotic surgery. Given the safety framing, latency analysis is relevant.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The architectural novelty is thin. Kalman filtering, RTS smoothing, and memory pruning are established techniques. The contribution is primarily in applying them to SAM2 for surgical video, with sensible design choices. This is valuable engineering but not a conceptual breakthrough. The paper cites SAMURAI and MoSAM but doesn’t clearly differentiate from these concurrent works, I’d like to see a direct comparison or clearer articulation of the unique aspects.
I’m also concerned about generalization beyond EndoVis. These datasets are relatively controlled (short sequences, limited instruments, static camera). Real surgical videos have smoke, blood, lens cleaning, and camera movement that might break the Kalman motion model. The paper doesn’t test robustness to these perturbations or discuss failure modes beyond the clean ablation scenarios.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #4
- Please describe the contribution of the paper
This paper addresses two critical failure modes in surgical video segmentation: abrupt positional shifts after instrument reappearance and memory pollution from prolonged instrument absence. The proposed BRAIN framework features two core modules: BMR module combining Kalman filtering and RTS smoothing to detect motion inconsistencies, and DM pruning module that removes redundant frames based on embedding dissimilarity. Experiments on EndoVis17 and EndoVis18 demonstrate significant improvements over SAM2-style methods, with ablations confirming the complementary roles of BMR and DM.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.Accurately identifies and quantifies critical real-world challenges (abrupt positional shifts and high Instrument Absence Ratio) with compelling visual data. 2.The bidirectional modeling using Kalman filtering and RTS smoothing is mathematically sound. The dynamic fusion strategy linking motion weights to absence duration is highly practical. 3.The DM module’s redundancy-aware filtering is a simple, and effective solution to prevent empty or incorrect frames from dominating the memory bank, which can be applied to other domains with BMR module.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The exact nature of the “embedding vector” used in Eq. 10 and the appearance score is unclear. The authors must clarify if this relies on fragile binary mask shapes or robust image-level semantic features (e.g., SAM2 Mask Tokens), and specify the distance metrics and normalization used. 2.Introducing RTS smoothing and pairwise memory dissimilarity inherently adds computational overhead. The total absence of runtime analysis (FPS, GPU memory) makes it impossible to compare against “real-time” baselines, which is a critical omission for latency-sensitive surgical applications. 3.The paper mixes results copied from original papers (some using category-level or prompt-free settings) with the fine-tuned, instance-augmented BRAIN. The fairness of the experimental setup should be emphasized more.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper proposes a logically sound and practically valuable framework to address critical surgical tracking failures. The reported performance gains are impressive. However, the manuscript suffers from ambiguous technical details regarding feature extraction, a critical lack of runtime efficiency analysis, and potentially uncontrolled baseline comparisons. If the authors can clarify the embedding implementation, provide hardware-specific runtime metrics, and ensure fair baseline comparisons during the rebuttal, this work will be a very strong contribution to the field.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The paper addresses a meaningful and practically important failure mode in surgical video segmentation, and the reported gains on challenging re-entry and long-absence cases are compelling. I agree that the main limitations are real, especially the somewhat incremental novelty relative to recent SAM2-style surgical adaptations and the limited evaluation scope beyond EndoVis benchmarks.
That said, the rebuttal satisfactorily addressed several decision-critical concerns, e.g., runtime and memory overhead, providing additional sensitivity analysis, isolating the contributions of Kalman filtering and RTS smoothing, specifying the embedding/similarity design, and reporting IAR-stratified results. These responses do not remove all concerns, but they increase my confidence that the method is technically sound, targeted to a real problem, and supported by meaningful empirical improvements. On balance, I support acceptance.
Author Feedback
We thank the reviewers for the constructive comments. We are encouraged that BRAIN is recognized as “well-motivated” (R3), “easy to follow” (R1), and “clearly organised” (R1,2), with “significant improvements” (R2,3) and “quantifying critical real-world challenges” (R4). We will release code upon acceptance. 1 Computational cost (R2,3,4). BRAIN (46.75 FPS, 0.02 s/frame, 3.23 GB) adds negligible overhead over SAM2 (52.18 FPS, 0.02 s/frame, 3.14 GB) on A800, since extra modules use simple matrix computation, RTS uses a short temporal window (L=15), and DM prunes redundant frames to further speed up the process. 2.1 Hyperparameter sensitivity (R1,2,3). BRAIN is insensitive to γ, k, L; τ and β are interpretable. λ=e^-n adjusts dynamically with n, not manually tuned. We evaluated τ/γ/k∈{0.1,…,0.9}, L∈{10,12,15,17,20}, and β∈{3,…,7}. BRAIN is stable to score fusion (γ), tracker-update gating (k), RTS window (L) with 0.06/0.05/0.09% IoU variation. τ and β are more influential, with 1.26/1.32% variation. τ controls motion-inconsistency detection: small τ over-triggers appearance reselection and large τ misses abrupt shifts. β controls memory pruning: small β over-prunes context, large β keeps redundant memories. Thus,τ=0.5,β=6 balance best. “Dynamic” means dynamic memory-frame selection, not capacity adjustment; dynamic capacity yields comparable Dice (86.48%) but lower efficiency (29.07 FPS). Failures mainly occur when both cues vanish, e.g., severe lens occlusion/blur, or strong camera shake, a shared open challenge. 2.2 BMR ablation (R2). We isolate BMR: Kalman improves Dice from 83.44% to 85.13%, and RTS further improves it to 86.51%, confirming both contributions. 3.1 Reselection (R2,3). Appearance reselection uses the most confident memory embedding (0.13% high-confidence errors), so corrupted, occluded, or misidentified frames do not dominate the appearance score. As analyzed in Fig. 6, DM suppresses empty, low-confidence, and redundant absent frames, enabling robust appearance matching after many absent frames. BMR improves current-frame mask quality and reduces drift; together, BMR and DM reduce repeated use of erroneous memories. When re-entry involves orientation, lighting, or occlusion but no abrupt positional shift, BRAIN selects mask using both motion and appearance cues. With IAR > 60%, including occluded re-entries, appearance matching localizes the target in 91.2% of cases. 3.2 Embedding/ReID (R4,2). Embedding is a SAM2 feature-pooled mask-level semantic vector. Appearance score is L2-normalized cosine similarity to the most confident memory embedding. BMR considers both motion and appearance scores, differing from ReID methods that recognize identities by simply fusing all frames. 4.1 Novelty (R1,3). Although SAMURAI, MoSAM, EndoMamba, Memory-Augmented SAM2, and ReSurgSAM2 improve surgical video representation or tracking, they ignore long instrument absence with abrupt positional shifts, a critical cause of performance reduction. Instead of simply combining Kalman and RTS like previous methods (SAMURAI), we innovatively propose BMR to correct mask selection under abrupt positional shifts and DM to remove redundant long absent frames, jointly enabling SOTA performance (e.g., SAMURAI, 79.69% Dice). 4.2 IAR-stratified (R3). For low/mid/high IAR (<30%, 30–60%, >60%), BRAIN obtains 87.43/81.85/93.30% IoU, compared with SAM2’s 83.23/79.85/52.15% and SurgSAM2’s 85.17/81.47/65.06%. The largest gain at high IAR confirms that BRAIN targets long-absence/re-entry failures. 5 Prompt/ Fairness (R2,3,4). Table 2 uses the full-mask prompt. Table 1 is the fair comparison, where SAM2, SurgSAM2, and BRAIN use the same instance-level protocol and prompt setting. The standard VOS first/last-frame exclusion changes performance by only 0.03%. Table 2 is a broad reference covering category-level, prompt-free setting. In the same instance-level setting, BRAIN beats occlusion-aware trackers e.g., LACOSTE/TraSeTR (85.6/81.0% Dice)
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The rebuttal should address four decision-affecting concerns. First, runtime, FPS, and GPU-memory analysis, which is critical given the safety framing and is currently absent. Second, a systematic sensitivity analysis of the multiple hyperparameters τ, γ, k, λ, L, β—of which only β is currently studied—and an isolated ablation that disentangles forward Kalman tracking from backward RTS smoothing within the BMR module. Third, an explicit specification of the embedding vector and similarity metric used in the motion-guided mask reselection, together with a discussion of long-absence re-entry behaviour, raised by R3: how appearance matching works when the instrument reappears after long absences with corrupted or stale memory entries. Fourth, IAR-stratified performance to substantiate the central claim that the method specifically improves the high-absence regime, and clearer positioning with respect to concurrent SAM2 adaptations such as SAMURAI and MoSAM, and to recent surgical-video foundation models such as EndoMamba, Memory-Augmented SAM2, and ReSurgSAM2.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The paper proposes bidirectional motion reasoning combined with dynamic memory pruning for surgical video segmentation on the EndoVis benchmarks. The reviewers raised four decision-affecting concerns: missing runtime and memory analysis given the safety framing; insufficient sensitivity analysis of multiple hyperparameters and a lack of an isolated ablation that separates forward tracking from backward smoothing; the unspecified embedding vector and similarity metric used in the motion-guided mask reselection, along with a question about behavior under long-absence re-entry; and positioning relative to concurrent SAM2 adaptations and recent surgical-video foundation models. The rebuttal substantively addresses the first three concerns, providing efficiency numbers, a hyperparameter sweep with reported sensitivities, isolating the contributions of the forward and backward components, explicitly specifying the embedding and similarity, and providing a stratified analysis under high-absence regimes. Two reviewers explicitly endorsed acceptance after the rebuttal. The remaining Reject concerns the positioning of related work, which the camera-ready version must improve.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
After rebuttal, there are two Accept and one Reject. The other reviewer did not update. After reading the paper as well as the reviews and author rebuttal, I think the paper is a good work with good performance and method design. Major concerns include limited engagement with recent related work and incremental novelty. Pros outweigh cons and therefore the recommendation is Accept.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The authors managed to provide satisfactory answers.