List of Papers Browse by Subject Areas Author List
Abstract
Generating precise diagnostic reports from high-resolution computed tomography is critical for clinical workflows, yet remains challenging due to the high pathological diversity and spatial sparsity within 3D volumes. While video language models (VideoLMs) have demonstrated remarkable spatio-temporal reasoning in general domains, their adaptability to domain-specific, high-volume medical interpretation remains underexplored. In this work, we present AbSteering, an abnormality-centric framework that adapts general-domain VideoLMs to precise report generation. Specifically, AbSteering introduces: (i) an abnormality-centric Chain-of-Thought scheme that encourages finding-first abnormality reasoning, and (ii) a Direct Preference Optimization objective that uses clinically confusable abnormalities as hard negatives to enhance fine-grained discrimination. Our results demonstrate that general-purpose VideoLMs exhibit strong transferability to high-volume medical imaging when guided by AbSteering. Notably, VideoLMs outperform state-of-the-art domain-specific CT foundation models, achieving superior detection sensitivity while mitigating hallucinated findings. Our data and model weights are released at \url{https://github.com/ayanglab/CTRG_VideoLMs}.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2789_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/ayanglab/CTRG_VideoLMs
Link to the Dataset(s)
CT-RATE dataset: https://huggingface.co/datasets/ibrahimhamamci/CT-RATE
CT-RARE-AB dataset: https://huggingface.co/datasets/yw3325/CT-RATE-AB
BibTex
@InProceedings{FanYin_Unleashing_MICCAI2026,
author = { Fang, Yingying AND Zhou, Huichi AND Lee, KinHei AND Wang, Yijia AND Zhang, Zhenxuan AND Huang, Jiahao AND Yang, Guang},
title = { { Unleashing Video Language Models for Fine-grained HRCT Report Generation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16879},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper ystematically validated the cross-modal transfer capabilities of general-purpose VideoLMs on three-dimensional high-resolution computed tomography (HRCT) data. According to the findings presented in the paper, general-purpose VideoLMs demonstrate strong transferability to 3D medical images, offering an efficient alternative to training modality-specific medical foundation models from scratch on massive datasets. Proposed AbSteering, an anomaly-centric steering framework (Method innovation). The paper proposes an efficient language-guided framework that successfully adapts VideoLMs for accurate radiological interpretation and achieves state-of-the-art performance on fine-grained clinical efficacy metrics. The core technologies of this framework are divided into two modules: a Chain-of-Thought (CoT) approach centered on anomalies, which forces the model to perform anomaly reasoning before generating the final report; a Direct Preference Optimization (DPO)-based objective function, utilizing clinically ambiguous anomalies as hard negatives to enhance the model’s ability to distinguish fine-grained lesions. The authors specifically designed and constructed a dataset called CT-RATE-AB to facilitate Chain-of-Thought (CoT) training in medical imaging and to evaluate the model’s ability to identify diverse lesions and fine-grained abnormalities when processing high-volume chest CT scans.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper innovatively treats 3D HRCT volumes as video-like slice sequences, enabling direct use of pre-trained general VideoLMs. This avoids training modality-specific 3D medical foundation models from scratch, which is extremely computationally expensive, and lowers the barrier for researchers with limited resources. It demonstrates strong cross-modal transferability of general VideoLMs to 3D medical imaging, providing an efficient alternative under constraints of high computing cost and limited high-quality medical data. The proposed anomaly-centric language steering framework (AbSteering) includes an anomaly-centered chain-of-thought (CoT) training strategy. By decoupling visual-to-text generation and requiring explicit anomaly reasoning, it reduces hallucinations and improves pathological reasoning, consistent with radiologists’ clinical workflow. In Stage 2, Direct Preference Optimization (DPO) is applied to improve fine-grained discrimination. The method of using LLMs to replace anomalies with clinically confusing ones within the same anatomical region cleverly expands negative samples. The authors construct the CT-RATE-AB dataset by organizing reports into 10 anatomical regions with a unified (region: abnormality) template. This dataset supports CoT training and offers a new benchmark for evaluating fine-grained and diverse abnormality recognition in large-volume CT.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Although the paper open-sources a high-quality, pre-processed dataset, the data structuring process lacks substantial innovation. The construction of Stage 1 and the CT-RATE-AB dataset relies heavily on using GPT-4o to reorganize raw free-text reports into 10 anatomical regions, extracting them into a structured template of the form (region: abnormality). This approach to using natural language processing techniques to structure radiology reports into (region: abnormality) tuples was first proposed in RadGraph (Jain et al. , NeurIPS 2021) and Chest ImaGenome (Wu et al. , NeurIPS 2021). It has long been established as foundational work in the field of medical AI; therefore, it lacks methodological novelty. 2.To accommodate VideoLMs, this paper directly converts HRCT scans into MP4 video format with a fixed frame rate. Forcing 3D data into a time series results in significant loss of physical geometric information; real CT slices have distinct physical slice thickness, pixel spacing, and absolute tissue density values. Treating the physical depth along the Z-axis as equivalent to the temporal dimension of the video completely disregards isotropy and real-world physical scales. 3D U-Net (Çiçek et al. , MICCAI 2016) and subsequent studies on medical volumetric data, such as V-Net (Milletari et al. , 3DV 2016), have long established a consensus in the academic community that preserving 3D spatial geometry and voxel continuity is indispensable for the diagnosis of medical lesions. 3.This paper repeatedly emphasizes in the abstract and introduction that it addresses the challenges of “spatially localized” and “fine-grained” identification of abnormal lesions in HRCT. However, in the experimental section, only text-based metrics are provided, such as BLEU, ROUGE, and the RadBERT text classification metric (F1-score), without any specific fine-grained visual evaluation metrics or other relevant assessment metrics. In authoritative research on medical vision-language pre-training, such as BioViL-T (Boecking et al. , CVPR 2023) or MS-CXR (Boecking et al. , 2022), validating “fine-grained localization” requires providing objective evidence at the pixel or region level. 4.All experiments and model fine-tuning in this paper were conducted using the CT-RATE dataset alone, which consists entirely of data from a single institution. The lack of validation using external datasets means that the clinical feasibility and generalizability of this single-center dataset remain to be verified. 5.Generating negative reports via GPT-4o text substitution in DPO lacks medical rigor and may produce anatomically inconsistent content, without professional clinician verification as required in clinical AI research. 6.The paper contains several typographical errors that disrupt the reading flow (e. g. , ‘tructured’ instead of ‘Structured’, ‘basent’ instead of ‘based’ in Fig. 1). The quality of Figure 1 is below the standard expected for a top-tier conference. The text within the diagrams is illegible even when zoomed in.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper presents an innovative and resource-efficient approach for fine-grained HRCT report generation, which ingeniously treats 3D HRCT volumetric data as video slice sequences, leverages pre-trained general-purpose VideoLMs combined with anomaly-centric CoT and DPO, effectively avoiding the high computational costs of training 3D medical foundation models from scratch and boasting a commendable core idea and problem-solving approach; however, as a submission to MICCAI—a top-tier conference emphasizing medical image computing rigor—it has critical, unavoidable flaws including simplistic 3D-to-video conversion that discards CT’s physical geometric information without sufficient justification, over-reliance on text-based metrics (BLEU, ROUGE, RadBERT) to support claims of spatial localization and fine-grained anomaly detection without any pixel or region-level visual evidence, experiments limited to a single-center dataset with no external validation (raising overfitting concerns) and DPO-generated synthetic reports lacking professional radiologist verification (posing clinical inconsistency risks), along with additional issues such as overreliance on unreviewed arXiv preprints for SOTA claims, low-resolution and unreadable Figure 1, and typographical errors, leading to a Weak Reject decision as its experimental robustness and overall rigor do not meet the conference’s acceptance criteria.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
This study proposes the AbSteering framework, which includes an anomaly center thought chain and a fine-tuning strategy based on direct preference optimization, to address the challenge of generating high-resolution CT reports using a pre-trained video language model.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.This research treats 3D CT sequences as video data and utilizes an advanced general video language model to process medical images. It avoids the high cost of training large-scale medical models from scratch, providing a transfer learning paradigm for high-volume medical data processing. 2.This study used a large amount of data.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The paper relies excessively on external large models (GPT-4o) for data construction. How do you explain the statement in section 2.2, specifically how such a massive dataset was manually refined (“a training set of 46,717 CT scans from 20,000 patients and a validation set of 3,039 CT scans from 1,314 patients.”) “We use GPT-4o [1] to assign each sentence to the predefined regions. or verification purposes, two auxiliary categories—uncategorized and repetitive—are introduced. We employ sentence-level matching followed by manual refinement to reclassify potential problematic content.” 2.The gold standard report didn’t serve as an upper limit for comparative testing models. 3.The values for downstream classification tasks are generally low. Is this meaningful in clinical practice? In clinical practice, it seems that report classification is not commonly used alone; it is recommended to use downstream task tests that are combined with imaging. Furthermore, why not compare it to direct image classification? 4.The “Full Report from Experts” shown in Figure 2 appears to be the gold standard report. Could you provide a corresponding structured report for comparison? How can the quality of the structured report be evaluated? 5.Formatting error: There is some white text in Figure 2; RadFM-?B in Table 1; The article contains inconsistencies in its expression of “CT-RATE” and “CT-RATE-AB”.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Generating reports from high-resolution CT images is a challenging task. The chest CT scans used in this study naturally contain multiple sites, often spanning multiple slices, placing extremely high demands on the model’s 3D spatial reasoning performance. Fine-tuning a general video model is a suitable approach, but such models often lack corpus information from medical images, and their performance, as shown in the ablation experiments, is poor. This may explain why the method used in this paper does not outperform dedicated report generation models on the NLG metric. It is difficult to evaluate this downstream task design of directly using the generated text for classification, as it seems to have little clinical significance. This also makes it difficult to compare the merits of this method with dedicated models, requiring careful design in experimental setup and metric validation. If the author can address these concerns, I will consider raising my score.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
The paper addresses automated radiology report generation for chest CT volumes using video-language models on the CT-RATE dataset. The authors propose an abnormality-centric training strategy with chain-of-thought-style (CoT) supervision to improve the detection and description of abnormal findings, aiming to reduce the tendency to overemphasize normal findings. In addition, they use DPO with hard negatives to encourage fine-grained discrimination. The method outperforms established baselines such as CT-CHAT or M3D on the CT-RATE validation set.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The focus on abnormality-centric supervision is well motivated, especially for datasets with many mostly normal reports.
- The two-stage curriculum learning strategy with abnormality-centric reasoning and hard-negative learning (DPO) is a well-designed approach to better emphasize abnormal findings.
- The method shows improved performance over strong baselines such as M3D and CT-CHAT.
- Releasing the CT-RATE-AB dataset could be valuable for the community.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- The use of video models itself does not appear very novel, as similar ideas have already been explored in medical imaging (e.g. classification and segmentation), including related vision-language settings. (Sambara, S., Kim, S. E., Zhang, X., Luo, L., Johri, S., Baharoon, M., … & Rajpurkar, P. (2025). 3DReasonKnee: Advancing Grounded Reasoning in Medical Vision Language Models. In Biocomputing 2026: Proceedings of the Pacific Symposium (pp. 99-113).)
- The paper positions the use of video models as a main contribution, but compared with the abnormality-centric supervision, which I consider the more novel aspect, the video-based formulation appears less novel.
- The experimental validation would be stronger with evaluation on an external dataset such as RAD-ChestCT.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
- The figures are hard to read due to small font sizes.
- Please clarify the relationship between CT-RATE-AB and the RadGenome dataset, as they seem conceptually similar from the current description.
- The explanation of the autoregressive language modeling loss could be shortened or omitted unless it is directly relevant to the paper’s main contribution.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the use of video-language models itself is not highly novel, I think the abnormality-centric training supervision is a strong and well-motivated idea to address the tendency of report generation models to overpredict normal findings. The two-stage curriculum with abnormality-centric reasoning and hard-negative DPO is supported by improved performance over strong baselines. Despite the missing external validation and some presentation issues, I believe the paper provides a useful and relevant contribution for the MICCAI community.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
N/A
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The proposed framework for report generation via videoLMs is interesting where the abnormality-centric CoT and DPO optimization appear reasonable to me. The paper is generally well-written and easy to follow. However, the reviewers point out the following concerns: 1) 3D medical info loss during transform from imaging format towards video format; 2) the lack of fine-grained visual inspection; 3) the lack of multi-center dataset. The authors are encouraged to further improve their manuscript.
