List of Papers Browse by Subject Areas Author List
Abstract
Language-guided Medical Image Segmentation (LMIS) has shown great potential to improve the delineation of anatomical struc-tures and lesions by integrating clinical textual information. Existing methods generally rely on either implicit interaction between textual and visual features or auxiliary coarse-grained supervision for cross-modal alignment. However, these methods lack explicit and fine-grained constraints to ensure semantic consistency, causing a mismatch between language and the segmentation outputs. To address this issue, we propose Text-as-Illumination Retinex Network (TIRNet), a novel Retinex-inspired framework that treats text embeddings as semantic illumination for feature modulation, thereby improving semantic consistency in LMIS. TIRNet introduces two key blocks integrated at each decoder stage: (1) the Retinex-inspired Text Modulation Block (RTMB), which employs positive and negative illumination maps to enhance text-relevant foreground features and suppress background interference; and (2) the Consistent Detail Compensation Block (CDCB), which selectively recovers high-frequency details via a consistency-gated mechanism conditioned
on illumination reliability. Furthermore, we propose a Multi-Scale Illumination Supervision Loss (MSIS-Loss), comprising a Region-Grounded Contrastive Loss (RGC-Loss) that enforces cross-modal similarity to be concentrated in text-relevant foreground regions and suppressed in background regions, and a Background Suppression Loss (BS-Loss) that provides pixel-level supervision for negative illumination maps, jointly ensuring a precise cross-modal alignment at each decoder stage. Extensive experiments on the MosMedData+ and QaTa-COV19 datasets demonstrate that TIRNet achieves state-of-the-art performance in LMIS. The code is available at: https://github.com/anaanaa/TIRNet.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1662_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/anaanaa/TIRNet
Link to the Dataset(s)
N/A
BibTex
@InProceedings{ShiJia_Text_MICCAI2026,
author = { Shi, Jian AND Zhen, Cheng AND Zhang, Pingping AND Xu, Rui AND Lv, Yanan AND Ma, Yili AND Bi, Huan AND Li, Haojie AND Lu, Huchuan},
title = { { Text as Illumination: Spatial Contrastive Retinex Learning for Language-Guided Medical Image Segmentation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16883},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper presents TIRNet (Text‑as‑Illumination Retinex Network) for language‑guided medical image segmentation. The core idea is to treat text embeddings as semantic illumination that modulates visual decoder features. y. Four main contributions are claimed: 1.RTMB (Retinex‑based Text Modulation Block) – Derives positive and negative illumination maps from cross‑modal similarity to explicitly enhance text‑relevant foregrounds and suppress backgrounds. 2.CDCB (Consistent Detail Compensation Block) – Recovers high‑frequency details from encoder features via a consistency‑gated mechanism conditioned on illumination reliability. 3.MSIS‑Loss (Multi‑Scale Illumination Supervision Losses) – Combines a Region‑Grounded Contrastive loss that enlarges foreground‑background margins and a Background Suppression loss that provides pixel‑level supervision for negative illumination maps. 4.Experiments on MosMedData+ and QaTa‑COV19 show state‑of‑the‑art performance compared to both unimodal and multimodal methods, including LViT and TeViA.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.Mapping Retinex theory to cross‑modal segmentation is creative and well‑motivated. The idea of using text as “illumination” that positively/negatively modulates features is elegant. 2.Unlike prior implicit or coarse‑grained alignment methods, the proposed RTMB produces per‑pixel similarity maps and two complementary illumination maps, enabling explicit foreground amplification and background suppression at each decoder stage.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The text encoder is a frozen CLIP model pre‑trained on general natural images and captions. Medical reports contain highly specialised terminology and different linguistic structure. No analysis or mitigation (e. g. , fine‑tuning, prompt engineering, or medical‑specific text encoder) is provided. 2.The evaluation lacks empirical efficiency profiling; while architectural complexity is quantified via parameters and FLOPs, clinically critical metrics including inference latency and memory footprint remain unreported. 3.The comparative evaluation in Table 1 exhibits methodological inconsistency in metric reporting. Specifically, some baselines (e. g. , U-Net, nnU-Net) report solely sample-level metrics (m-Dice, m-IoU), whereas certain multi-modal competitors (e. g. , TeViA) provide only global metrics (g-Dice, g-IoU). Notably, TIRNet is the sole method presenting all four metrics, precluding direct and equitable comparison across the complete metric suite. To ensure fair and transparent benchmarking, we strongly recommend that the authors compute and report the missing metrics for all comparative methods through standardized re-implementation.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper offers notable strengths, including a novel Retinex-inspired cross-modal segmentation framework, explicit spatial alignment with complementary illumination maps, and thorough ablation studies. However, several unresolved weaknesses limit its overall impact: (1) lack of discussion on CLIP’s domain adaptation to medical terminology; (2) missing clinically critical metrics such as inference latency and memory footprint; (3) inconsistent metric reporting in Table 1 (mixed sample-level vs. global metrics), preventing fair comparison across baselines. Given the novelty and solid technical core, the paper merits acceptance, but addressing the above issues—especially metric consistency and efficiency profiling—is strongly recommended before final publication.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The paper proposes TIRNet, a language-guided medical image segmentation (LMIS) framework inspired by Retinex theory. It treats text embeddings as “semantic illumination” to modulate visual features via a Retinex-based Text Modulation Block (RTMB) and a Consistent Detail Compensation Block (CDCB). Additionally, a Multi-Scale Illumination Supervision Loss (MSIS-Loss) is introduced to enforce cross-modal alignment through region-grounded contrastive learning and background suppression.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.Innovative Perspective: Mapping Retinex theory from low-level vision enhancement to cross-modal feature modulation provides a novel conceptual framework for LMIS.
2.Architecture Design: The explicit decoupling of foreground enhancement (L+) and background suppression (L−) is a well-structured engineering approach to handle multi-modal interference.
3.Empirical Results: The method demonstrates competitive performance on MosMedData and QaTa-COV19 datasets compared to several recent baselines.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Logical Conflict Between Local Receptive Fields and Global Spatial Semantics: The proposed TIRNet relies on a CNN-based U-Net architecture. While the text embeddings provide global semantic information, the visual features in the decoder stages are inherently local due to the translational invariance of convolutional kernels. The manuscript does not mention the use of explicit Positional Encodings. I am concerned about how the model manages to achieve precise spatial alignment for directional or lateral clinical instructions (e.g., “lower left lung” vs. “lower right lung” ). In the absence of a global coordinate system or positional embeddings, local features in anatomically symmetric regions may yield nearly identical similarity responses to the same global text vector, potentially leading to cross-lateral false positives. The authors should clarify how the architecture maintains global spatial consistency without explicit positional information.
2.Anomalous Performance of MSIS-Loss and Hyperparameter Sensitivity: In Table 3, the sensitivity analysis reveals a concerning trend regarding the proposed losses. The baseline model (λ_cl=0, λ_bg=0) achieves an m-Dice of 84.36.However, adding the Background Suppression Loss (BS-Loss) independently (λ_cl=0, λ_bg=0.5) results in a performance degradation to 84.20.Furthermore, when λ_cl and λ_bg are set to 0.5, the result (84.35) remains inferior to the baseline. It appears the proposed losses only provide a marginal gain at a very specific hyperparameter setting (λ_cl=1, λ_bg=0.5). This suggests that the pixel-level hard constraint of BS-Loss may conflict with the mixed-semantic nature of deep-stage features (large receptive fields), leading to optimization instability. The authors need to provide a deeper mechanistic explanation for why these losses act as negative optimizers in several parameter configurations.
3.Conceptual Overstatement of Retinex Formulation: While the “Text-as-Illumination” metaphor is inspiring, the mathematical implementation in Equation 5 is effectively a sophisticated dual-path spatial attention or feature-gating mechanism. Unlike traditional Retinex theory, which addresses an ill-posed inverse problem (decomposing unknown illumination from a single image), this framework uses feed-forward gating signals derived directly from text. Referring to this as a strict “extension of Retinex theory” feels like a conceptual overstatement. A more rigorous description would frame it as a Retinex-inspired decoupled attention mechanism.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I recommend a Weak Reject. The core idea of using text as a modulation signal via a decoupled positive/negative illumination scheme is interesting and shows promise. However, I have major concerns regarding the architectural logic of spatial alignment in a pure CNN framework without positional embeddings (Weakness 1). More importantly, the ablation and sensitivity studies (Table 3) show that the proposed loss functions often perform worse than the baseline unless tuned to a very specific point, raising questions about the robustness and the underlying synergy of the MSIS-Loss components (Weakness 2). I invite the authors to address these logical and experimental inconsistencies in their rebuttal.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors’ rebuttal addressed the specific questions raised in my review. My final evaluation is based on the following responses: 1.Spatial Alignment: Addressed. The provided CoordConv ablation experiment answered my empirical question. The authors clarified that the model relies on zero-padding for spatial awareness rather than explicit spatial grounding. 2.MSIS-Loss Anomaly: Addressed. The authors explained that BS-Loss causes “over-conservatism” on deep-stage mixed-semantic features. This explanation accounts for the hyperparameter sensitivity observed in the original ablation study. 3.Retinex Metaphor: Addressed. The authors confirmed the framework is a structural inspiration rather than a strict Retinex decomposition, aligning with the critique regarding the terminology.
Review #3
- Please describe the contribution of the paper
The main contribution of this paper is proposing an innovative cross-modal fusion framework for language-guided medical image segmentation. The core of this framework is the Retinex-based Text Modulation Block (RTMB), which dynamically modulates visual features via a dual-path gating mechanism using text embeddings. Secondary contributions include the Consistent Detail Compensation Block (CDCB) for recovering high-frequency structural details guided by illumination reliability, and multi-scale illumination supervision losses (RGC-Loss and BS-Loss) to enforce structured foreground-background contrast and alignment.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The major strength is the Retinex-based Text Modulation Block (RTMB), which utilizes text embeddings as condition vectors to dynamically modulate visual features via a dual-path gating mechanism. This spatial weighting approach is mathematically sound and demonstrates a strong capability to selectively enhance foreground targets while suppressing complex backgrounds, offering an inspiring and practically valuable perspective for multimodal feature alignment in medical imaging.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
In Section 2.1, the Retinex formulation is introduced into a cross-modal setting, interpreting the visual decoder feature as reflectance and the text embedding as illumination. While the dual-path modulation effectively separates foreground and background, the underlying physical analogy appears conceptually forced. Retinex is inherently a low-level physical model for illumination-reflectance decomposition, whereas the proposed method operates entirely in a high-level semantic space as a form of text-guided feature modulation. In the rebuttal, I would appreciate it if the authors could clarify their justification for this strict physical analogy. Moving forward, it would be more appropriate to reframe this design as a “Retinex-inspired modulation mechanism” to improve conceptual rigor. The current ablation studies mainly demonstrate the incremental benefit of the RTMB module, which largely reflects the inherent advantage of simply incorporating cross-modal information. However, this alone does not sufficiently justify the necessity of the proposed architectural complexity. Specifically, why did the authors not explore or discuss comparisons with alternative, standard modules that serve a similar purpose, such as a standard cross-attention mechanism? Without such a de facto baseline, it is difficult to determine whether the proposed dual-path modulation offers meaningful performance or efficiency advantages over conventional cross-modal fusion methods. I look forward to the authors’ discussion and insights on this architectural choice during the rebuttal.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, I lean towards a weak accept for this paper. The major positive factor is the clever design of the RTMB module, which provides an effective and interesting way to disentangle foreground and background in medical images. However, my score is held back by two main issues: the conceptually forced Retinex physical analogy, and the lack of alternative fusion modules in the ablation studies. In the rebuttal, I look forward to the authors’ clarification on the conceptual rationale and their discussion/insights regarding why alternative fusion modules were not considered.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
To All: Q1: Retinex Rationale (MR, R2, R3) A1: In fact, our TIRNet is not a strict Retinex decomposition to solve the illumination-reflectance inverse problem. We instead use Retinex as a structural inspiration: text embeddings provide semantic illumination-like signals to modulate visual decoder features. It enhances text-relevant foregrounds and suppresses irrelevant backgrounds. In the revised version, we will clarify the above facts and provide more motivations about using the dual-path spatial gating mechanism in Eq. 5.Q2: RTMB vs. Other Fusion Methods (MR, R3): A2: To verify the effectiveness of RTMB, we replace it with two standard alternatives, i.e., concatenation fusion and cross-attention fusion. On QaTa-COV19, they achieve 82.40/73.44 and 82.88/73.90 in m-Dice/m-IoU, respectively. They are consistently lower than the ones (83.13/74.45) achieved by RTMB. These results verify that the improvement is not merely from introducing textual information, but from the proposed RTMB. Q3: Spatial Alignment w/o PE (MR, R2): A3: Indeed, explicit PE is not used in TIRNet. However, visual features of our framework are not position-agnostic. This is fully supported by three facts: (1) zero-padding can provide implicit absolute-coordinate cues (Islam et al., ICLR 2020); (2) hierarchical down-sampling and skip connections help preserve and propagate relative spatial information; and (3) deep decoder stages have a large effective receptive field, enabling them to capture holistic organ layouts rather than only local textures. In addition, TIRNet introduces explicit spatial grounding through MSIS-Loss. At each decoder stage, down-sampled masks supervise the illumination maps, thereby penalizing activations outside the text-relevant ground-truth regions. More importantly, adding normalized 2D CoordConv/PE before RTMB results in negligible gains across all metrics [84.50/76.17/91.37/84.11 (QaTa-COV19)] compared to TIRNet. It confirms that explicit PE is unnecessary here. We will add more representative visualizations to show the effect of the localization-aware text.
To R1: Q1: CLIP Domain Gap A1: We further evaluate the text encoder adaptation. On QaTa-COV19, replacing frozen CLIP with BioMedCLIP and MedCLIP-ResNet50 achieves 83.57/75.00/90.58/82.79 and 83.77/75.23/90.79/83.14, respectively. They are both below the frozen CLIP setting. Besides, the prompt engineering with “A medical image showing {Text}.” yields lower results (83.82/75.25/90.80/83.15). Fine-tuning the CLIP produces comparable results (84.48/76.06/91.31/84.01). This suggests that text encoding is not the primary bottleneck in our structured-text setting. Q2: Efficiency A2: Under the same evaluation setting, TIRNet takes 115.46 ms/case with 1426 MiB peak GPU memory. While not the fastest, it remains practical and is more efficient than several LMIS baselines, e.g., LViT (378.59 ms, 17518 MiB) and LGA (117.03 ms, 4338 MiB). We will include the latency/memory and discuss the accuracy-efficiency trade-off in the revision. Q3: Metric Consistency A3: The original table followed the metrics reported in prior works. We agree that standardized evaluation is preferable. We therefore re-evaluated all compared methods under a unified protocol, which enables full comparisons, e.g., [TeViA: 84.12/75.69/90.96/83.41; U-Net: 78.46/68.59/86.66/76.47 (QaTa-COV19)]. We will update Table 1 accordingly in the revision. Note that the main conclusion remains unchanged.
To R2: Q1: MSIS-Loss A1: In fact, BS-Loss is an auxiliary regularizer. With (0, 0.5), MSIS-Loss suppresses backgrounds, making the model over-conservative around small lesions, ambiguous boundaries, or deep-stage mixed-semantic features. When both weights are 0.5, the highlighted foreground remains insufficient to balance this suppression. Thus, the weights (1, 0.5) better match our design: RGC-Loss provides the main semantic alignment, while BS-Loss offers moderate background regularization.
To R3: See General Response A1/A2.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This submission proposes TIRNet, a Retinex-inspired language-guided medical image segmentation framework using text-derived positive/ negative modulation maps, detail compensation, and multi-scale illumination supervision. Reviewers generally found the core idea interesting and technically promising, with competitive results on MosMedData+ and QaTa-COV19 and useful ablations supporting the RTMB/CDCB design. However, several important concerns should be addressed in rebuttal: whether the Retinex framing is conceptually justified versus a standard text-guided attention/gating mechanism; whether the method offers advantages over conventional cross-attention or alternative fusion baselines and how spatially specific text cues are handled in a CNN-based architecture without explicit positional encoding.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The rebuttal addresses reviewers’ concerns reasonably well. Overall, I recommend acceptance. The contribution is technically interesting and the rebuttal substantially reduces the main doubts about architectural necessity, spatial alignment, efficiency, and evaluation fairness.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The proposed Retinex-inspired text modulation framework is recognized as an innovative and effective approach to language-guided medical image segmentation. While initial concerns were raised regarding the forced physical Retinex analogy, spatial alignment logic in CNNs, and the optimization sensitivity of the loss functions, consensus among reviewers post-rebuttal highlights that the authors have successfully resolved these issues through comprehensive empirical clarifications.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
three accepts, i agree.
