List of Papers Browse by Subject Areas Author List
Abstract
Automated report generation for whole-body PET/CT imaging can substantially reduce radiologists’ workload and improve clinical efficiency. Existing report generation approaches typically maps the entire volumetric scan to a complete report. However, clinically meaningful findings in whole-body PET/CT are sparse and localized. Such global end-to-end modeling tends to omit lesion-specific semantic cues, limiting complete descriptions of lesions. To address this challenge, we propose a whole-body PET/CT report generation framework guided by local lesion-level cues. Our method is built upon three key strategies. First, we propose an aggregation module that compresses variable lesion candidates in PET/CT, enabling many-to-one lesion–description alignment that reflects real clinical reporting practices. Second, we propose local entity alignment and global relation alignment during pretraining to enforce cross-modal semantic consistency between report and image. Third, we use a two-stage report generation training pipeline, namely pretraining on structured reports and fine-tuning on free-text radiology reports, to reduce template bias. We have collected 3,015 whole-body PET/CT cases from three medical centers, including 120,549 annotated local lesions. Experimental results demonstrate that the proposed framework improves performance under clinically reliable evaluations of generated reports.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0238_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{YuMin_Semantically_MICCAI2026,
author = { Yu, Mingyang AND Gao, Yaozong AND Shu, Yiran AND Liu, Jingyu AND Wu, Jiaming AND Sun, Kaicong AND Bai, Shuwei AND Chen, Yanbo AND Zhou, Xiang Sean AND Zhan, Yiqiang AND Zhang, Weifang AND Zhong, Shaonan AND Wang, Xinlu AND Zhao, Meixin AND Shen, Dinggang},
title = { { Semantically Consistent Whole-Body PET/CT Report Generation with Local Lesion-Level Guidance } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16878},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes a local lesion-level guided framework for whole-body PET/CT report generation. The main contributions include three aspects: (1) a SetTransformer-based aggregation module that compresses a variable number of lesion candidates into a fixed-length entity set, enabling many-to-one lesion-description alignment; (2) the introduction of local entity alignment and global relation alignment during pretraining to enhance cross-modal semantic consistency between images and text; and (3) a two-stage report generation training pipeline (pretraining on structured reports followed by fine-tuning on free-text reports) to mitigate template bias. Experiments are conducted on 3,015 whole-body PET/CT cases from three medical centers, and the results are convincing.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.Accurate identification of a key issue. The paper accurately recognizes the central challenge in whole-body PET/CT report generation, namely the many-to-one mapping between lesions and descriptions, and proposes a targeted solution accordingly.
2.Well-designed methodology with coherent logic. The local entity alignment and global relation alignment are complementary, which is a reasonable improvement for PET/CT scenarios where image and text diversity is relatively low. The two-stage training strategy also follows a logically progressive design.
3.Considerable data scale. The dataset of 3,015 multi-center, multi-tracer cases represents a relatively large-scale dataset in the field of whole-body PET/CT report generation, strengthening the credibility of the experimental conclusions.
4.Sound experimental design. Table 1 evaluates performance on both structured reports and free-text reports, providing a comprehensive assessment. Table 2 clearly demonstrates the individual and combined contributions of the alignment strategies and structured report pretraining, which helps in understanding the effectiveness of each component.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Concerns about clinical practicality. The proposed method requires lesion patches along with their corresponding SUVmax and size information as input. In the current setup, these inputs are pre-annotated and manually reviewed. However, such prior information may not be readily available in real clinical practice, and I have reservations about the clinical practicality of this method.
2.Poor reproducibility due to missing implementation details. The paper does not specify the backbone architecture or initialization strategy for the encoder during pretraining, nor does it report the value of K in the SetTransformer, the hyperparameters such as λr and τ in the loss functions, or training details such as the number of training steps and learning rate.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
Minor comments:
1.The clinical evaluation metrics are somewhat unclear, although this is understandable given the page limitations.
2.Test set proportion is too small. The test set contains only 204 cases (approximately 6.8% of the total data), which may result in many disease types being absent or underrepresented in the test set, potentially leading to biased evaluation results.
3.As suggestions for future work, the dataset is rich, encompassing multiple centers and multiple tracers, yet the paper does not report whether performance varies across different centers or tracers; subgroup analyses along these dimensions would strengthen the findings. Additionally, it is regrettable that no manual evaluation by radiologists was included. This is especially so considering that the dataset as a whole had already been reviewed by physicians, which suggests that substantial effort had already been invested by them. In comparison, having radiologists evaluate the results on the test set should have required only a relatively small amount of additional work.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
On the one hand, the paper addresses an important topic, the proposed method aligns well with clinical reporting mechanisms, the dataset is substantial in scale, and the experimental results demonstrate a reasonably stable positive trend. On the other hand, the practicality and reproducibility of the proposed method remain questionable. Overall, the paper offers a degree of inspiration, and the final acceptance decision should be deferred until after the rebuttal.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The rebuttal has addressed my concerns.
Review #2
- Please describe the contribution of the paper
The paper addresses automated report generation for whole-body PET/CT. The authors propose a two-stage pipeline. First, they perform CLIP-style pretraining adapted to PET-specific reporting by modeling many-to-one lesion descriptions through local entity alignment and global relation alignment. Second, for report generation, they use a curriculum strategy that progresses from structured reports to free-text reports.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The method design is clinically well motivated. The paper addresses an important limitation of existing whole-body PET/CT report generation approaches by focusing on local lesion-level guidance, which is well aligned with clinical practice where relevant findings are sparse and localized.
- The proposed local entity alignment and global relation alignment are interesting ideas to better match PET-specific reporting patterns, especially the many-to-one relationship between lesions and textual descriptions.
- Training on structured positive findings before fine-tuning on free-text reports seems like a well-motivated strategy, as it may help the model first learn clinically important content before adapting to more template findings.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- An ablation study for the pretraining design is missing. In particular, the lack of comparisons to standard CLIP-style training, local entity alignment only, and global relation alignment only makes it harder to clarify the contribution of each component.
- Since CLIP-style pretraining is typically evaluated through tasks such as zero-shot classification or retrieval, the absence of such experiments makes it harder to assess the benefit of the proposed pretraining more directly.
- In Figure 2b and in the method description, lesion attributes are also added to the prompt. It is therefore unclear to what extent the model relies on the image content versus the textual lesion attributes in the prompt.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
- Is the dataset publicly available?
- A-all, A-pos, F-all, and F-pos should also be briefly explained in the main text, not only in the table caption.
- In Table 1, LLaVA-Med is a 2D model. Please clarify how it is applied in this setting.
- In Table 1, are all models trained on the same dataset?
- Please provide more details about the CE metrics, as the current description is not fully clear or reproducible.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper presents a clinically well-motivated approach for whole-body PET/CT report generation, and the focus on lesion-level guidance is a meaningful strength. The proposed pretraining and structured-to-free-text curriculum are interesting and seem well aligned with the task. My main concerns are the missing ablations and the limited analysis of how much the gains come from the image content versus the lesion attributes provided in the prompt.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The rebuttal partly addresses my concern about the CLIP baseline, as the authors explain that standard CLIP pretraining provided limited improvement.
However, some concerns remain. In particular, it is still not clear how much the report generation relies on image features versus the textual lesion attributes provided in the prompt.
Nevertheless, the lesion-level formulation and the overall workflow are clinically well motivated.
Review #3
- Please describe the contribution of the paper
In this work, the authors propose a report generation especially designed for whole-body PET/CT scans. As PET/CT scans contain sparse and localized information (in contrast to natural images), the proposed generation framework aims to compress variable lesion candidates in one decscription such as e.g. the report of many lymph nodes. A two-stage report generation training pipeline, is used to avoid template bias often occurring especially in studies trained on single center data. The authors test their approach on a large number of images.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- As PET/CT scans differ highly from natural images, methods especially designed for medical data are of high need, therefore the topic of the paper is interesting.
- The authors also introduce a special training scheme to avoid over-fitting to reduce template bias.
- The authors test their approach on a large number of images demonstrating improvement over other LLMs generating medical reports.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- I am afraid that it can happen that in the many-to-lesions aggregation module important information gets lost. Did the authors analyzed carefully that only similar lesions were merged in one description and that no important information was missed?
- Recently, several paper address also PET/CT report generation such as PETAR and PETRG (https://arxiv.org/html/2510.27680v2,https://arxiv.org/pdf/2511.20145). How does the approach perform compared with these methods?
- While the authors speak about PET/CT, they only use PET images for training. Previous work mentioned that this might lead to a loss in spatial information.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
While the research question is interesting, I would be interested in seeing the results compared with other recently published report generation tools such as PETAR and PETRG. Please clarify the important differences between your work and theirs and also demonstrate that your method is superior. Please also clarify that no important information is lost in your aggregation module and that really only similar lesions are summarized. Moreover, results indicate that your approach is superior to others, it would be important to know if each part of your approach really lead to an improvement. E.g. the lesion aggreagation module might lead to a drop in evaluation metrics while the training scheme might increase the score again as it reduces template-bias. It would be nice if you could clarify the contribution of each of your contributions.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the idea is interesting, I am sceptical that each step proposed by the authors leads to an improvement as described above. Moreover, as similar methods have been published recently, I would like to know the novelties of the proposed approach as well as the performance of the proposed method with the recent papers.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
I thank the authors for their rebuttal. My concerns were clarified in the rebuttal. However, the manual selection of the test-set as mentioned by the authors in the rebuttal is a bit odd to me.
Author Feedback
1.Clinical practicality (R1, R2) The reviewer is concerned that lesion patches, SUVmax, and lesion size may not be readily available in clinical practice. In fact, these inputs are derived from a lesion segmentation model. As described in Section 3.1, lesion masks are extracted using a trained nnU-Net, and quantitative attributes such as SUVmax and lesion size are directly computed from PET/CT images with these masks. Purely end-to-end report generation cannot guarantee the accuracy of quantitative measurements. Therefore, we use mask-based computation to obtain objective lesion-level evidence. With mature lesion segmentation as the upstream step, we believe this workflow is clinically practical for PET/CT report generation.
2.Pretraining design (R2) The reviewer is concerned that the pretraining design is not sufficiently evaluated, due to the lack of comparison with standard CLIP contrastive pretraining and ablations of local entity and global relation alignment. In previous experiments, standard CLIP pretraining was difficult to optimize and brought limited improvement, possibly because SetTransformer aggregates many-to-one lesion candidates, making samples less suitable for standard contrastive learning. We will clarify this previous comparison in the revision. Since report generation is the main task and space is limited, our current ablation focuses on whether the proposed pretraining strategy improves final generation performance. We agree that retrieval experiments and component-level ablations would further clarify its effectiveness and each component’s contribution and thank the reviewer for this helpful suggestion.
3.Comparison with related work (R3) We appreciate the reviewer’s suggestion. Compared with PETAR, our work addresses a different task-specific challenge: PETAR focuses on local finding generation with a near one-to-one lesion-description relationship, whereas complete whole-body PET/CT report generation often requires many-to-one aggregation, such as multiple bone lesions (Fig. 1b) or lymph node lesions (Fig. 1a) summarized in one finding description. Compared with PETRG, our main difference is explicit lesion modeling: PETRG directly models the whole PET/CT volume, while our method introduces lesion-level guidance to address the imbalance between small clinically relevant lesions and the large whole-body background. This is particularly important for small lesions occupying only a tiny fraction of PET/CT images. We will clarify these differences in the revision.
4.SetTransformer aggregation (R3) We thank the reviewer for raising the concern about information loss during lesion aggregation. In our dataset construction, clinically similar lesions were first grouped by anatomical region or lesion category before SetTransformer aggregation. We found that K, the number of SetTransformer output entities, is a key factor affecting performance and reducing information loss. A larger K preserves more representative lesion patterns and improves performance but slows training and increases computational cost. Therefore, we used K=8 as a trade-off between information preservation and efficiency. We will clarify this in the revision.
5.Reproducibility (R1, R2, R3) We thank the reviewers for the comments on reproducibility and evaluation details. Upon acceptance, we will release the code, configs, and evaluation protocols needed to reproduce the reported results. We will add missing implementation details, including the encoder backbone, τ, and other training settings. We will also clarify that λr is a batch-size-related balancing coefficient. The relatively small test set reflects deliberate selection rather than random splitting. We manually selected representative cases covering diverse lesion types, tracers, and centers, making the test set informative under annotation constraints. Our method uses PET/CT data. We will also revise Fig. 2 to avoid the misunderstanding that only PET was used.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper proposes a lesion-guided framework for whole-body PET/CT report generation, using a SetTransformer to aggregate variable lesion candidates and a two-stage pretraining–fine-tuning pipeline (structured reports -> free-text) to mitigate template bias. Reviewers agree the method is clinically well-motivated, addresses the many-to-one lesion-description mapping, and benefits from a large multi-center dataset (N=3,015). The two-stage curriculum and alignment strategies are seen as logical. However, weaknesses are also noted: clinical practicality is severely limited as the method requires pre-annotated lesion patches, SUVmax, and size, which are not awalys available in real practice. Reproducibility is poor due to missing backbone, hyperparameters (K, λr, τ), training details, and no code/data release. Further, the paper lacks crucial ablations (e.g., standard CLIP pretraining, local/global alignment alone), does not compare to recent PET/CT report generation works (PETAR, PETRG), and risks information loss in the aggregation module. The test set is too small (6.8%), and no radiologist evaluation or subgroup analysis by center/tracer is provided.
Issues for rebuttal:
- Clinical practicality: How can the method work without pre-annotated lesion inputs?
- Missing implementation details (backbone, K, λr, τ, learning rate, training steps) – provide specifics.
- Incomplete ablation: Compare to standard CLIP, local-only, global-only; assess contribution of each component.
- Missing comparison to recent PET/CT report generators (PETAR, PETRG) – explain novelty and performance.
- Risk of information loss in SetTransformer aggregation – show that only similar lesions are merged.
- Small test set (6.8%) and lack of subgroup analysis (center, tracer). And absence of radiologist evaluation.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The authors’ rebuttal directly addresses the major concerns raised by the reviewers. Regarding clinical practicality, they clarify that lesion patches, SUVmax, and size are obtained from a pretrained nnU-Net segmentation model, making the workflow feasible with mature upstream lesion detection. For reproducibility, they commit to releasing code and adding missing hyperparameters (K=8, τ, λr, backbone, training details) upon acceptance. The lack of ablations on pretraining design is acknowledged: they note that standard CLIP pretraining was difficult to optimize and will clarify this in the revision, though retrieval‑style ablations are not provided. Comparisons to recent PET/CT report generators (PETAR, PETRG) are explained in terms of task differences (many‑to‑one aggregation vs. one‑to‑one; explicit lesion modeling vs. whole‑volume). Information loss in the SetTransformer is mitigated by grouping similar lesions by region and setting K=8 as a trade‑off. The small test set is defended as a deliberate manual selection for diversity, though this remains a limitation. Missing radiologist evaluation and subgroup analysis by center/tracer are not addressed. Overall, the rebuttal resolves most technical concerns, and all reviewers agree that the remaining issues do not undermine the method’s novelty and clinical motivation.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
This manuscript introduces a local lesion-level guided framework for whole-body PET/CT report generation, mainly based on SetTransformer-based aggregation and a two-stage pretraining–fine-tuning pipeline for report generation. The clinical motivation of the study is clearly explained. The method is evaluated on a large multi-center PET/CT cohort, and the experimental results are promising. The rebuttal has addressed most of the reviewers’ concerns, including clarification of clinical practicality, implementation details, CLIP-style pretraining, and differences from other methods.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
All reviewers have recommended acceptance following the rebuttal. The paper presents a clinically well-motivated approach for whole-body PET/CT report generation, and the focus on lesion-level guidance is a significant strength. The authors have adequately addressed most of the reviewers’ concerns, and the remaining issues do not outweigh the paper’s contributions.