List of Papers Browse by Subject Areas Author List
Abstract
Accurate lesion segmentation in PET/CT is critical for oncology, yet remains challenging because physiologic tracer uptake and artifacts can mimic malignant signal. We present RADIANT-PET, a reasoning-augmented framework that couples a high-sensitivity voxel-level segmentation model with lesion-level large language model (LLM) adjudication. Candidate uptake regions are generated with a deliberately permissive segmentation stage, then converted into structured textual descriptions that summarize uptake intensity, morphology, and regional and global anatomical context. An LLM classifies each candidate as true lesion vs. false positive, optionally leveraging the radiology report as additional clinical context. To strengthen lesion-level reasoning, we further optimize a local LLM via reinforcement learning using Group Relative Policy Optimization, with rewards that encourage anatomically plausible decisions. Across AutoPET and an institutional test cohort, RADIANT-PET consistently outperforms strong image-only baselines, with the largest improvements observed when radiology reports are provided. Overall, these results demonstrate that LLM-based lesion-level reasoning adds a novel reasoning layer beyond conventional segmentation, suppressing physiologic false positives and aligning voxel-level predictions with clinical interpretation.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2094_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/jwang-580/RADIANT-PET
Link to the Dataset(s)
https://zenodo.org/records/20785543
BibTex
@InProceedings{WanJia_RADIANTPET_MICCAI2026,
author = { Wang, Jiasheng AND Jitwatcharakomol, Tanun AND Jongpradubgiat, Piyawadee AND Zhu, Simeng},
title = { { RADIANT-PET: Reasoning-Augmented PET/CT Lesion Segmentation with Large Language Models and Reinforcement Learning } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16883},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper presents a framework that use a language model with RL fine-tuning to improve the segmentation performance of a high-sensitivity U-Net model.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This framework incorporate free-text radiology reports as additional clinical context for lesion segmentation. Additionally, the proposed method improves the classification accuracy of the LLM using reinforcement learning.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
My main concerns are about the rl fine tuning. 1.The reward design is discrete and sparse, which leads to weak ranking signals within groups. This makes it harder for the model to learn meaningful distinctions and capture variance within the groups. 2.The anatomical reward design is overly strict, as it only assigns credit for exact matches. This raises the question of how near correct predictions that are close to the ground truth are handled. 3.RL significantly improves performance over the baseline; however, only GRPO was evaluated. Consider comparing it with other LLM fine-tuning methods.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The authors propose a novel methodology, and the Threshold + loss-RL approach significantly improves performance across all metrics. However, the manuscript lacks a clear description of GRPO, comparisons with other fine-tuning methods, and justification for the choice of reward function.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The paper proposes a reasoning-augmented framework for PET/CT lesion segmentation, which combines a high-sensitivity candidate generation stage with LLM-based lesion-level adjudication. The main contribution is the introduction of an explicit reasoning layer that filters false-positive uptake candidates using structured lesion descriptions, anatomical context, and optionally radiology reports. The paper further incorporates GRPO to improve the local LLM’s candidate classification ability, and reports improved performance over strong image-only baselines on both public and institutional datasets.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
(1) Novel framework design. The idea of converting segmentation outputs into structured text and using an LLM to adjudicate each candidate lesion is a reasonable and interesting approach. It draws on how radiologists incorporate anatomical and clinical context into their decisions, which gives the framework a sensible motivation beyond purely image-based processing.
(2) Practical reinforcement learning application. Using GRPO to fine-tune a local LLM is a pragmatic choice given privacy constraints in clinical deployment. The results show that the method provides measurable improvements over the base model, making the contribution practically credible for real-world clinical settings.
(3) Clear presentation and well-organized paper. The paper is logically structured and easy to follow. Figure 1 directly exposes raw JSON code as part of the main pipeline illustration, which make visual clarity for readers familiar with the implementation details. Figure 2 more effectively illustrates the qualitative impact of RL on the model’s reasoning trace. Overall the writing is clear, the motivation is well-stated, and the progression from candidate generation to LLM adjudication to final segmentation is communicated in a coherent and readable way.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
(1) Training details are not sufficiently clear. The description of the training setup is not detailed enough. Important implementation details, such as the GRPO group size, are missing or only briefly mentioned. This makes it difficult to assess the robustness of the method and limits reproducibility.
(2) The experimental design involving report-conditioned input is not explained clearly enough. It is unclear why the radiology report should be used in this setting, especially since the report itself may already contain direct or indirect information about whether a candidate uptake is true or false. If the report already includes clinician interpretation, then part of the reported gain may come from access to the answer rather than from better lesion-level reasoning. The paper should explain the clinical purpose of this setting more carefully and clarify what information from the report is actually used.
(3) Although the paper tries to address the false-positive lesion problem, the clinical value of the task is not well reflected in the method design or experimental evaluation. First, false-positive uptake in PET is not limited to normal physiologic uptake; inflammatory uptake is also an important source of false positives, but this is not clearly discussed or evaluated in the paper. Second, in real clinical practice, clinicians are often more concerned with false-positive lesions in regions such as the abdomen, where interpretation cannot be resolved by simple anatomical plausibility or symmetric patterns alone. The paper does not provide targeted evaluation on these more challenging local regions, which limits its practical clinical value. Third, the method relies heavily on structured information derived from high-sensitivity lesion segmentation, organ segmentation, and hand-crafted features. The paper does not sufficiently discuss whether these inputs are reliable enough, or whether they already encode enough fixed clinical or anatomical knowledge to account for much of the false-positive filtering.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
My score is mainly based on the gap between the methodological interest of the paper and its currently limited demonstrated clinical value. The proposed framework is technically interesting, especially the use of structured lesion descriptions for LLM-based candidate adjudication. However, the paper does not yet convincingly show that the method addresses the most clinically meaningful false-positive scenarios, and some parts of the experimental design, especially the report-conditioned setting, remain insufficiently justified.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
After reading the rebuttal and considering the other reviewers’ concerns, I recommend Reject. While the paper shows some novelty, the rebuttal does not sufficiently address my concern that the overall contribution remains limited. Therefore, I do not think it is strong enough for acceptance.
Review #3
- Please describe the contribution of the paper
In their manuscript “RADIANT-PET: Reasoning-Augmented PET/CT Lesion Segmentation with Large Language Models and Reinforcement Learning” the authors describe an automated framework for automated lesion segmentation in PET/CT imaging. They address a common challenge in PET/CT lesion segmentation: How to find an optimal balance between sensitivity and minimization of false positives.
To this end they propose a two-stage framework: (1) permissive candidate lesion generation (threshold or UNet) and (2) lesion-level classification by an LLM using structured region descriptions and radiology reports, enhanced by GRPO RL.
The authors describe improved performance using this method as compared to direct UNet-based lesion segmentation alone.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main strength and novelty of this work is the introduction of a per-lesion reasoning process for classification of PET lesions. The underlying idea of using structured lesion descriptors allows the introduction of novel training mechanisms like reinforcement learning into this field of application. In addition, the proposed method is inherently interpretable since it is based on natural language descriptors and reasoning chains.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
This work has major limitations:
-
Baselines: The authors only compare their method to internal ablations but do not use SOTA baselines for automated PET lesion segmentations. Suitable baselines exist publicly (e.g. high performing methods of prior MICCAI challenges such as the autoPET challenge). The used baseline, which is the highest performing model from the autoPET 3 challenge was designed for multi-tracer applications and is thus not a specialized SOTA model for FDG-PET segmentation. Comparison to SOTA baselines is an important prerequisite for the assessment of method performance
-
The LLM-based classification method only uses high level text descriptors of a lesion derived e.g. from organ locations segmented with Total Segmentator or PET lesion properties. Such concepts do not capture image information directly and are this limited in their contribution to lesion classification. Using a multimodal VLM would be a much more suitable approach. In addition, the authors should compare this LLM-based classifier to simple baselines such as per lesion patch CNN-based classifier.
-
Related to the point above, an LLM classifier cannot improve per lesion segmentation accuracy. It is thus unclear how a threshold-based baseline can be suitable for such a method. How was the threshold determined? Was it a global SUV, per lesion? Per study? “We chose 2.5 because it is the standard cutoff for defining metabolically active lymphoma” - this not a clinically accurate statement - there is not threshold for metabolically active lymphoma.
-
The clinical application, training and evaluation is exclusively done on Lymphoma cases using FDG-PET images. This limits the generalizability of presented results.
-
Data and results are described superficially. The authors do not provide information on the type of lymphoma, stage, tumor burden, patient cohort characteristics. Also, results are only presented at a high level with global statistics. It is also unclear how model performance was measured. Are the provided metrics per lesion? Per volume? What is the variation (only mean values are presented). Where were most errors observed? etc..
-
It is unclear how accurate the institutional baseline is (“candidate regions were manually labeled by a hematologist and a radiologist”, an expert Nuclear Medicine Physician or specialized Radiologist would be more appropriate for this task). What software was used? Was this also threshold based? Or purely manual? How many lesions were segmented in how many organs?
-
Providing more image examples would be helpful to allow the reader to have a better understanding of model performance
-
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Strengths:
Novel Method in the field of PET lesion segmentation introducing LLM-based reasoning and reinforcement learning.
Weaknesses:
Baselines and generalizability: The study compares mainly internal ablations and a non-applicable autoPET baseline, with no evaluation against state-of-the-art, modality-specific baselines (e.g., top FDG-PET methods from MICCAI challenges). Evaluation is limited to lymphoma FDG-PET/CT, raising concerns about generalizability to other cancers, tracers, and multi-center settings.
Reproducibility gaps: No public release of code. Critical details (cohort characteristics, lesion counts, software/tools used for manual labeling, thresholding choices) are insufficient for replication.
Evaluation details missing: Results are presented at a high level (mean values) without lesion-level breakdowns, uncertainty estimates, or statistical significance tests. Lacking are per-lesion/per-volume metrics, subgroup analyses, and error analyses to understand failure modes.
Methodological concerns: The LLM-based per-lesion classifier relies on high-level text descriptors rather than direct multimodal image features. There is no comparison to multimodal baselines (e.g., VLM or patch-CNN classifiers).
Threshold and clinical claims: Some statements (e.g., “2.5 is the standard cutoff for defining metabolically active lymphoma”) appear clinically inaccurate.
Dataset and deployment specifics: Institutional baseline labeling by “an expert hematologist and a radiologist” lacks detail on expertise, workflow, and number of lesions; no description of imaging protocols, scanner types, or patient-level attributes (stage, tumor burden) provided.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors responded to major concerns and provided helpful context. While major open questions remain about the approach and performance of presented methodology, it can be reasonable to address these in future work.
Author Feedback
We thank the reviewers for the constructive comments. *Reproducibility [All]. We commit to release all code, HS-UNet weights, GRPO scripts, and LoRA adapters upon acceptance. *SOTA baselines [R3]. Our baseline, nnUNet-v2, won autoPET III and ranked first on the FDG sub-track. It outperforms earlier autoPET I/II winners, while autoPET IV reformulated the task as interactive click-prompted segmentation and is therefore not directly comparable. nnUNet-v2 is thus the appropriate SOTA. *Thresholding [R3]. We used fixed-SUV thresholding because it reflects how lymphoma metabolic tumor burden is calculated in multicenter clinical trials. Prespecified thresholds reduce reader- and site-dependent variability, and international TMTV harmonization efforts recommend threshold-based approaches (SUV ≥2.5, SUV ≥4.0, or 41% SUVmax) as standards for delineating active lesions. Thus, per-lesion voxel-level delineation is not the limiting factor in PET lymphoma segmentation; the central challenge, which RADIANT-PET targets, is distinguishing true tumor uptake from false positives at the lesion level. *GRPO details and reward design [R1,R2]. Sec. 2.4 was condensed due to page limits; the final version will include full implementation details. We fine-tuned gpt-oss-20b in BF16 using Unsloth + TRL GRPOTrainer with LoRA on all attention and MLP projections: q/k/v/o and gate/up/down. LoRA rank was 8 with α = 16.GRPO used 4 completions per prompt, temperature 1.0, and 2,048-token limits for prompts and completions. Optimization used AdamW-8bit with LR 5e-5, linear decay, 10% warmup, weight decay 1e-3, per-device batch size 4, and gradient accumulation 4.The reward summed three terms: (i) binary correctness, with +1 for the correct physiological-site versus lesion-site label and −1 if both labels appeared; (ii) anatomical-site accuracy, with +2 for exact matches and +1.5 for curated equivalence groups, such as bone/bone marrow or GI subregions; and (iii) a reasoning trace regularizer, with −0.5 if certain key words were not referenced. Reward weights were selected by ablation, and this configuration achieved the best validation F1.We chose GRPO over SFT because candidate labels and anatomical sites provide verifiable ground truth, a rare opportunity in clinical AI, where labels are often subjective or incompletely specified. *Clinical impact and cohort description [R2,R3]. The project was motivated by our clinical experience treating lymphoma patients, where accurate baseline PET/CT tumor burden is crucial for precision risk stratification, treatment selection, and outcome prediction across modern therapies, including CAR T-cell therapy. All patients in our cohort underwent CAR T-cell therapy, and most had large B-cell lymphoma. We will add cohort and annotation details truncated for space. These are presentation clarifications of existing analyses, not new experiments. *LLM vs VLM/patch-CNN [R3]. During development, we evaluated a medical VLM, MedGemma, and observed weak lesion-level classification, likely due to limited PET/CT data in open-source medical VLM pretraining; effective fine-tuning would require larger datasets, which we note as future work. We agree that a per-lesion CNN classifier would be a valuable baseline; however, a CNN cannot natively incorporate the radiology report as a conditioning signal, which is central to our framework. *Annotation [R1,R3]. Institutional labels were generated by a visiting radiologist with >10 years of experience in general radiology, including nuclear medicine, and finalized by two board-certified clinicians: a hematologist with 8 years of lymphoma and routine PET/CT experience, and a radiation oncologist with 8 years of expertise in tumor-target delineation. Annotation was performed in 3D Slicer. Candidate lesions were manually selected and refined by region expansion or shrinkage to an SUV threshold of 2.5.The framework was also evaluated on the external autoPET dataset to assess generalizability.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper was evaluated by reviewers with expertise in the field who identified limitations resulting in mixed enthusiasm for this work. Questions were raised regarding clinical impact, appropriate comparisons to state-of-the-art, and unclear methodological details.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The reviewers showed some enthusiasm for this work being presented at the conference. Several concerns remain but the overall contributions are strong enough to justify acceptance.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation.
This paper presents an interesting framework that combines lesion-level reasoning with large language models and reinforcement learning for PET/CT lesion segmentation. Reviewers appreciated the originality of incorporating structured lesion descriptions and reasoning-based candidate adjudication, as well as the clinical motivation of reducing false positive findings.
However, concerns remained regarding the strength of experimental validation, choice and breadth of baselines, clinical impact, and overall evidence supporting the proposed design choices. While the rebuttal clarified several methodological and implementation details, it did not fully resolve these concerns. Given the mixed reviews and remaining questions regarding validation and significance, I think the paper remains marginally below the acceptance threshold for MICCAI this year.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Overall, the rebuttal clarified several important methodological and reproducibility details, but significant concerns remain regarding baseline adequacy, generalizability, and the strength of evidence for clinical value. I therefore recommend rejection.