List of Papers Browse by Subject Areas Author List
Abstract
Reasoning-based medical image segmentation has recently gained prominence by enabling complex target identification through natural language instructions. However, current research in 3D Reasoning Segmentation often prioritizes performance on curated benchmarks, leaving models vulnerable to realistic variations in instructions. To enable a systematic analysis of this fragility, we introduce a novel progressive perturbation benchmark spanning three difficulty levels. On this benchmark, state-of-the-art models show unstable performance and semantic drift, where reasoning becomes misaligned with visual evidence under perturbed instructions. We trace this issue to a critical bottleneck: insufficient visual information flow during the reasoning and mask decoding stages, which leads to cascading errors and significant performance degradation under perturbed inputs. To address these challenges, we propose MRSeg3D, a single-encoder architecture designed to facilitate tighter visual-semantic coupling. The core innovation lies in a suite of cascaded vision injectors that enforce persistent visual grounding by anchoring the model to invariant features at key decision points, thereby mitigating reasoning drift. We further introduce a Seg-Local Projector that collaborates with the mask prompt encoder to refine text-visual cues and improve the precision of [SEG] token execution. Experimental results demonstrate significant and consistent performance improvements in the Dice coefficient and text generation quality. This work provides actionable insights for next-generation intelligent clinical systems. Our code and models are available at https://github.com/lihaoqin168/MRSeg3D.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0767_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{HaoQin_MRSeg3D_MICCAI2026,
author = { Hao, Qin AND Yu, Long AND Tian, Shengwei AND Chen, Bonian AND Zhou, Cheng AND Zhang, Lei AND Ye, Xujiong},
title = { { MRSeg3D: A Perturbation-Resilient Model for 3D Medical Reasoning Segmentation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16883},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper makes two main contributions. First, it addresses the fragility of 3D reasoning segmentation models when facing perturbed instructions. To measure this, the authors introduce a three-tier benchmark evaluating both instruction robustness and reasoning-segmentation alignment. Second, it proposes MRSeg3D. This single-encoder architecture uses cascaded vision injectors and a Seg-Local Projector to keep visual information grounded during reasoning and decoding. This helps reduce the semantic drift that often happens with complex clinical text.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper’s strengths are its clear problem formulation and targeted design. Instead of treating medical image segmentation just as a standard prompt task, the authors break down consistency into instruction robustness and reasoning-segmentation alignment. This is a very useful perspective for clinical translation. Methodologically, MRSeg3D avoids simply scaling up the language model. The authors noticed that baseline models lose visual evidence as reasoning progresses, so they addressed this directly by re-injecting intermediate visual features. This proves the value of targeted architectural tweaks. Additionally, the experimental evaluation is extensive. Testing on 25 datasets is solid work, and the model clearly outperforms a strong baseline, especially in the most challenging free-text scenarios.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The main weaknesses are the clinical realism of the benchmark and the lack of statistical rigor. The authors describe the Level III test as real-world clinical discourse, but these texts were actually generated by DeepSeek. There is no mention of clinician validation or human quality control. Therefore, the demonstrated robustness really only shows generalization to synthetic prompts, which isn’t the same as clinical readiness. Furthermore, for a paper focused on stability, it lacks multiple-run variance, confidence intervals, or statistical significance tests. This makes the performance gains less convincing. Additionally, the data protocols for the 25 datasets are not transparent enough regarding patient-level splits and deduplication. Finally, the mechanistic claim about semantic drift is slightly overstated; it currently reads more like a reasonable hypothesis than a strictly proven mechanism.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
My recommendation of a weak accept reflects the balance between a highly relevant methodological contribution and some noticeable gaps in validation. The paper addresses a genuinely critical and often overlooked bottleneck in 3D reasoning segmentation—linguistic robustness—and provides a structurally sound and empirically effective solution that convincingly outperforms a strong baseline. However, the score is held back from a higher tier because the core narrative heavily relies on the assumption that the synthetic perturbation benchmark equates to real clinical text, a leap made without clinical validation. If the authors can address the dataset transparency issues and provide proper statistical uncertainty bounds during the rebuttal, I would be very open to raising my overall score.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
This paper focuses on improving the robustness of 3D medical reasoning segmentation when input instructions are reformulated or made more ambiguous. Its main contribution is a perturbation-based evaluation protocol together with MRSeg3D, an extension of the single-encoder reasoning-segmentation framework that introduces persistent visual grounding during language reasoning and a dedicated local projection path for segmentation execution. Experiments across multiple datasets and LLM backbones show improved robustness and segmentation accuracy compared with prior single-encoder baselines, especially under more challenging free-text perturbations.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper addresses a timely and relevant problem: robustness of 3D medical reasoning segmentation under perturbed natural-language instructions. Rather than only reporting performance under a fixed prompting setup, the paper explicitly studies instruction-level fragility and reasoning and segmentation consistency, which is an important perspective for clinically relevant multimodal systems.
1.Benchmark contribution The tiered perturbation benchmark with multiple instruction templates enables a finer-grained robustness analysis as linguistic ambiguity increases, compared with conventional label-style prompting. This evaluation design is a concrete strength because it connects prompt variability to measurable degradation modes.
2.Diagnostic decomposition Separately evaluating reasoning quality vs segmentation fidelity is well motivated and can reveal decoupled failures (e.g., stable language understanding with degraded masks), which strengthens interpretability of multimodal errors.
3.Targeted architectural refinement As a targeted refinement of a strong single-encoder reasoning-segmentation framework. The vision injectors and the Seg-Local projection path are both tied to specific hypothesized failure modes, namely loss of visual grounding during reasoning and imperfect transformation from [SEG] token representations to final masks. This makes the method design more convincing than a purely heuristic module addition.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- The degree of novelty is somewhat limited. While the paper is well motivated and the architectural design is coherent, MRSeg3D is best understood as a targeted refinement of strong single-encoder reasoning-segmentation baselines, especially R1Seg-3D, rather than a fundamentally new formulation. In particular, the main changes are the addition of intermediate visual injection during reasoning and a local projection path for segmentation execution. These components are reasonable and potentially effective, but they may still be viewed as incremental extensions on top of prior designs.
2.The central mechanism claim is plausible, but not fully established causally. The authors argue that robustness degradation mainly comes from insufficient visual grounding and semantic drift during reasoning / decoding. While Table 5 and Fig. 6 are consistent with this interpretation, the paper does not include a more explicit mechanism study that separately tests this claim, for example by isolating the effect of visual injection during reasoning versus the effect of the segmentation-side local projection under matched conditions. Therefore, the results support the effectiveness of the proposed design, but they do not fully establish that the claimed mechanism is the primary cause.
3.The perturbation benchmark is useful, but its connection to real clinical language remains limited. The tiered perturbation protocol is a valuable part of the paper and enables more fine-grained robustness analysis. At the same time, it remains a template-based benchmark designed by the authors. Therefore, the results should be interpreted as robustness under the proposed perturbation setting, rather than robustness to the full diversity, ambiguity, and messiness of genuine clinical language in practice.
4.There is a noticeable gap between the training setup and the evaluation scope. The paper states that training uses only instruction templates 1 and 5, while evaluation covers a much broader range of perturbation types. This makes the reported robustness results interesting, but it also raises questions about how much of the gain comes from architectural robustness versus generalization under a particular train/test protocol. More discussion of this point would strengthen the paper.
5.Some implementation and reproducibility details remain relatively high level in the main paper. The paper gives the overall architecture and staged training procedure, but many details that would matter for full reproducibility appear brief in the main text. This is a practical weakness, although it is partially mitigated by the authors’ statement that code and models will be made publicly available.
6.Some claims are framed a bit more strongly than the evidence strictly supports. The experiments support improved robustness and performance under the proposed benchmark, but broader claims about handling realistic clinical discourse should be stated more cautiously, especially since the paper itself acknowledges that pathology-specific and open-ended clinical scenarios remain challenging.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I am slightly positive on this paper because it studies a meaningful and timely problem, proposes a useful perturbation-based evaluation protocol, and presents a coherent robustness-oriented extension of a strong single-encoder baseline. The experiments are fairly broad and generally support improved robustness under the proposed benchmark. My recommendation is only Weak Accept, however, because the methodological novelty is somewhat limited, the semantic-drift explanation is plausible but not fully isolated experimentally, and the benchmark remains template-based rather than fully representative of real clinical language. Overall, I believe the work is above the acceptance threshold, but only marginally.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This work proposes a series of modules based on SAM and LLM to achieve text-guided segmentation. To enhance resilence of the linguistic perturbation, authors utilize multiple projectors and multi-stage training to align text and image. Experimental results (especially Fig. 5) demonstrate the stability under text-guided segmentation. But some details should be clarified for improved comprehension.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The proposed method solves an important question (enhancing linguistic stability).
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Major 1: The author aims to improve the stability under linguistic perturbation. However, the architecture in Fig. 3 seems complicated. The authors should increase the understandability of the architecture. Besides, the green and black arrows in Fig. 3 are not explained. Multi-Modal Projector has two output arrows, which seems wrong. Major 2: In Equation (1) to (3), notations are recommended to draw on the corresponding Figure. Major 3: The authors mention “reasoning process” more than one time. But the example in Fig. 3 only indicates the simply generated text, which does not contain CoT or reasoning process. The authors should specify “reasoning process” or use other phrases. Major 4: In Fig. 5, the standard deviation of the proposed method is always lower than compared method. What’s the reason of this phenomenon? Provide analysis from the perspective of segmentation results and model architecture. Major 5: In Fig. 6, what’s the feature value in distribution plots sampled from? (seg token or mask logits or others?)
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The stability of lingustic perturbation is an important question in VLM. The proposed method partially deal with this task. But the relation between proposed method and linguistic perturbation is not thoroughly explained.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
N/A
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The proposed paper targets at an interesting and critical problem in existing 3D reasoning-based segmentation methods where the instruction perturbation poses great challenges to segmentation effect. The proposed vision injector and seg-local projector appear reasonable to improve the generalization of existing LVM/LLMs under varied instruction levels. All reviewers recognize the contributions of the proposed method and therefore I tend to recommend acceptance. However, the authors are encouraged to polish the paper: 1) improve the writings of implementation details; 2) validate the diversity and authenticity of the {prompt template} with respect to clincal practice; 3) add more modality and lesion into benchmarking.
