Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Vision-language models have shown promising results in radiology report generation. However, most existing methods generate reports as flat text and do not explicitly model the semantic dependency between the Findings and Impression sections, which can lead to inconsistencies between clinical observations and diagnostic conclusions. In this paper, we propose RadHiera, a semantic hierarchical reinforcement learning framework for radiology report generation. RadHiera follows the semantic organization of radiology reports by first optimizing overall report quality, then improving the diagnostic accuracy of the Impression section, and finally enforcing consistency between Findings and Impression so that diagnostic conclusions are supported by clinical evidence. Specifically, we begin with a base reward that combines linguistic quality and medical factuality to provide supervision on the whole report. On this basis, we introduce a severity-aware reward for the Impression section that places greater emphasis on errors involving clinically critical conditions, thereby reducing both missed diagnoses and overstatement. We further enforce cross-section consistency using Expert Model-derived label sets, with subset constraints and hallucination penalties to ensure that impressions remain faithful to the findings. Experiments on three public chest X-ray benchmarks show that RadHiera consistently improves diagnostic accuracy and inter-section consistency over state-of-the-art methods, while also demonstrating good adaptability to ultrasound report generation. The source code is available at https://github.com/xmed-lab/RadHiera.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1791_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/xmed-lab/RadHiera

Link to the Dataset(s)

N/A

BibTex

@InProceedings{DuBod_RadHiera_MICCAI2026,
        author = { Du, Bodong AND Yang, Honglong AND Li, Xiaomeng},
        title = { { RadHiera: Semantic Hierarchical Reinforcement Learning for Medical Report Generation } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16878},
        month = {September},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper proposes RadHiera, a semantic hierarchical reinforcement learning framework for radiology report generation. The core idea is a multi-level reward mechanism for GRPO-based RL training: (1) a base reward combining linguistic fluency and clinical correctness at the whole-report level, (2) a severity-aware reward for the Impression section that weights clinically critical conditions more heavily, and (3) a cross-sectional consistency reward enforcing that Impression labels form a subset of Findings labels. Experiments are conducted on three public chest X-ray datasets and one private ultrasound dataset.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The motivation is clinically well-grounded. Radiology reports have an inherent hierarchical structure (Findings → Impression), and explicitly modeling this dependency in the optimization objective is a reasonable and meaningful direction.

    1. The evaluation covers four datasets spanning two imaging modalities (chest X-ray and carotid ultrasound).

    2. The ablation study (Table 3) provides some evidence that each reward component contributes, showing that the full hierarchical reward yields synergistic improvements over partial configurations.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.UniRG (Liu et al. , arXiv 2601.17151, Jan 2026) adopts a very similar paradigm, SFT on a vision-language model followed by GRPO with composite rewards, and has achieved state-of-the-art on the RexRank leaderboard. RadHiera and UniRG share the same RL algorithm (GRPO), similar reward decomposition philosophy (combining linguistic, clinical, and model-based signals), and overlapping evaluation datasets. The paper does not cite, discuss, or compare with UniRG in any form.

    2.The paper claims to follow the RexRank evaluation protocol (citing [24]) and uses RexRank-associated datasets (MIMIC-CXR, IU-Xray, ReXGradient), yet does not submit to the public leaderboard for standardized evaluation. The results on the private test set used by rexrank is worth reporting.

    3.The consistency reward (Eq. 9) is a hard binary signal (1 if I⊆F, else 0). It also only penalizes hallucinated Impression labels but ignores omissions (conditions present in Findings but absent from Impression).

    4.The three rewards in Eq. 4 are summed without weighting coefficients. No justification for direct addition is given, nor is there a discussion of potential conflicts between rewards.

    5.All clinical rewards depend on CheXbert for label extraction (Eqs. 6, 8), yet CheXbert covers only 14 conditions and may have reduced accuracy on generated (vs. human-written) text. This creates a risk of reward hacking. The model could learn to produce text that maximizes CheXbert label agreement without genuinely improving clinical quality. This limitation is not discussed.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper addresses a clinically motivated problem, enforcing hierarchical consistency in radiology report generation, and the high-level idea of a multi-level reward aligning with report structure is appealing. However, I have several concerns as mentioned in the weaknesses section. I would consider increasing my score if those concerns are adequately addressed in the rebuttal.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    The paper proposes to explicitly model the consistency between Findings and Impression section in a radiology report generation and optimize clinical metrics directly and not only word likelihood. To achieve this they propose to train report generation using RL (GRPO) with a multi-level reward formulation targeting both clinical correctness and consistency.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • three level reward design covers important aspect of radiology reporting, such as the general correctness, emphazised caution with critical conditions and internal consistency
    • generalization to US report generation, which is generally a harder and less researched task compared to Xray reporting
    • the ablation study in Table 3 is informative and helps understanding the contributions of the different reward parts
    • comprehensive evaluation on three X-ray datasets and one internal Ultrasound dataset
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    • The set of methods the paper compares to is relatively small. Rexrank includes models which achieve better results in some of the metrics than the models, one especially good excluded model being UniRG-CXR which outperforms on multiple metrics on multiple dataset (e.g. in RadCliQ and SembScore on MIMIC-CXR, in RadCliQ, RadGraph, SembScore and BertScore on IUXray and in RadCliQ, SembScore and RadGraph on ReXGradient). It would strengthen the paper to either explain why this model was excluded or include it in the comparison.

    • The reported IU-Xray and ReXGradient scores for models such as MedGemma and MedVersa differ from the results reported in ReXRank. It would be helpful if the authors could clarify whether these differences come from changed evaluation settings, different checkpoints, prompt/template choices, or their own fine-tuning setup.

    • The consistency reward is rather simple. It is set to 1 if Impression labels are a subset of Findings labels, else 0.That could potentially encourage overly conservative impressions and would ignore deviations in e.g. severity or findings going beyond the 14 chexbert labels.

    • The evaluation metric for consistency follows the same subset logic, so consistency issues outside that paradigm would also be missed during evaluation.

    • Some techinical details are missing:
    • How is the syntax reward calculated, which measurements are used to reward fluency and semantic similarity?
    • How is the clinical reward calculated, what is used to “extract clinical entities” and how is the comparison defined?
    • Which clinical label calssifier is used for both the severity and the consistency reward? (impression reward)
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The general idea and motivation of the paper is well-grounded and the proposed reward design covers important aspects of report generation. Further, the evaluation is extensive, spanning multiple datasets and ablations. However, in its current form their are important points, which I would need to see clarified in the rebuttal before I can justify recommending acceptance. The most important points are clarifying the evaluation results with respect to the official ReXRank leaderboard and adding the missing technical details. I would also like to see the current limitations of the consistency reward discussed.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The authors clarified my main concerns regarding the comparison to UniRG and other leaderboard models, the rational for the subset constraint and missing technical details. I would ask the authors to include all these new information in the final version of the paper.



Review #3

  • Please describe the contribution of the paper

    This paper addresses the inconsistency between the Findings and Impression sections in radiology report generation and proposes a reinforcement learning framework that incorporates explicit cross-sectional constraints into the training objective.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper addresses a well-known issue in radiology report generation—cross-sectional inconsistency—and proposes a simple subset-based constraint that is easy to implement and empirically effective. The consistency reward is clearly formulated and supported by ablation, where most of the gains can be attributed to this component.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The method relies heavily on a frozen classifier to define both supervision and reward, effectively reducing the problem to optimizing agreement with the classifier rather than improving report generation. Since evaluation is also based on classifier-derived metrics (e.g., CheXbert-F1), the reported improvements may largely reflect metric alignment rather than genuine gains in clinical quality. The severity-aware component is not convincingly validated, and its contribution appears marginal. The hard subset constraint is overly restrictive and does not reflect real clinical reporting, where impressions may reasonably extend beyond explicitly stated findings.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    The paper addresses an important problem and proposes a simple and effective constraint. However, the contribution appears to rely heavily on classifier-based supervision, and it remains unclear whether the gains reflect true report quality improvement or metric alignment. Clarifying this point would significantly strengthen the paper.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper addresses an important issue in report generation and proposes a simple consistency constraint that is clearly formulated and empirically effective. However, the method relies heavily on classifier-derived labels for both reward and evaluation, making it unclear whether the gains reflect true improvements in report quality or better alignment with the evaluation metrics. In addition, the severity-aware component is not convincingly validated. Overall, the idea is reasonable but the contribution appears limited, leading to a borderline recommendation.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The rebuttal satisfactorily addressed several of my main concerns. In particular, the authors clarified that the subset constraint is not an arbitrary heuristic but reflects the dominant structure observed in real reports, with approximately 93% of samples satisfying the Findings-Impression subset relation. This substantially strengthens the motivation for the proposed consistency reward. The additional CheXbert-only baseline was also helpful in addressing my concern regarding simple metric optimization. The results suggest that optimizing only classifier-derived agreement does not recover the broader improvements observed with the proposed hierarchical reward formulation, particularly in consistency and report structure. I still believe the overall contribution is more engineering-oriented than fundamentally methodological, and the approach remains heavily dependent on classifier-derived supervision. In addition, the severity-aware component appears weaker than the consistency reward and contributes relatively modest gains overall. However, the rebuttal provides reasonable clarification for these points, including class-wise improvements on clinically important findings. Overall, the paper presents a simple but practically meaningful approach to improving cross-sectional consistency in radiology report generation, and the empirical evidence is sufficient to place the paper slightly above the acceptance threshold.



Author Feedback

We thank the Meta-Reviewer (AC) and reviewers (R1-R3) for their feedback. Key responses below:

1.UniRG comparison (R1, R2, AC-Q1): We will cite/discuss UniRG. Both use SFT+GRPO, but optimize different objects: UniRG is a large-scale RL system using an 8B backbone and 550k+ studies, optimized with report-level metrics. RadHiera instead targets the Findings-Impression dependency: with a 3B backbone and 200k+ studies, we design a novel semantic hierarchy reward that grounds Impression conclusions in Findings evidence. We reproduced a UniRG-style recipe under the same 3B/200k+ setting on MIMIC-CXR:

Method | BLEU2 | BERTS | SembS | RadG | 1/RadC ————-+——-+——-+——-+——-+——- UniRG-3B | 0.188 | 0.402 | 0.421 | 0.255 | 0.901 RadHiera-3B | 0.206 | 0.421 | 0.442 | 0.282 | 0.939 UniRG-8B | 0.205 | 0.419 | 0.491 | 0.266 | 1.006

Under the same scale, RadHiera (ours) outperforms UniRG-3B and remains competitive with UniRG-8B.

2.Consistency reward (R1-R3, AC-Q2): The subset constraint reflects the dominant report structure, not an arbitrary simplification. It enforces a subset relation: Impression diagnoses should be supported by Findings. Under the same label space, 93.0% of MIMIC-CXR samples satisfy this relation: 31.0% equal and 62.0% strict subset; extension cases account for only 7.0%. Similar patterns are observed in other datasets.

Although the consistency reward cannot penalize omissions, omissions are handled by the severity-aware reward, which penalizes missed critical diagnoses. Its overall gain is modest due to sparsity of severity labels, but class-wise results improve clinically important findings: pneumothorax F1 0.5191->0.5560 and pulmonary edema 0.4892->0.5278, supporting its role in mitigating omissions.

We did not use soft overlap metrics because they may penalize valid concise subsets, encourage copying Findings into Impression, and introduce noisy reward. Also, we acknowledge the limitation from finite label space and labeler noise.

3.CheXbert hacking concern (R1, R3, AC-Q3): We directly test a CheXbert-only plus base reward baseline that optimizes classifier-derived label agreement without our hierarchy design:

Method | BLEU2 | BERTS | SembS | RadG | 1/RadC | F1 | Consis ————–+——-+——-+——-+——-+——–+——-+——- CheXbert-only | 0.191 | 0.409 | 0.381 | 0.211 | 0.901 | 0.401 | 0.781 RadHiera | 0.206 | 0.421 | 0.442 | 0.282 | 0.939 | 0.394 | 0.851

CheXbert-only obtains slightly higher CheXbert-F1 but underperforms on NLG quality, RadGraph, RadCliQ, and Findings-Impression consistency. This shows classifier-only optimization can cause narrow metric alignment, whereas RadHiera’s semantic hierarchy reward improves broader report quality and consistency. CheXbert-F1 alone is not definitive clinical validation; the pattern indicates RadHiera is not simply exploiting CheXbert-specific biases. Future: multi-labeler/uncertainty-aware rewards and blinded radiologist evaluation.

4.Implementation, reproducibility, and baselines (R1, R2, AC-Q4): We will add missing details. (1) Rewards are summed equally (1:1:1), selected by hyperparameter search; nearby weights were stable. (2) Syntax/fluency uses BLEU and ROUGE-L; semantic similarity uses BERTScore/SembScore. Clinical entity reward uses RadGraph-style extraction: generated/reference reports are converted into normalized tuples covering observations, anatomy, negation/uncertainty, and relations; micro-F1 is computed. CXR severity/consistency rewards use CheXbert 14-class labels; ultrasound uses a modality-specific labeler trained on privately annotated reports. (3) MedGemma/MedVersa differ from ReXRank because we fine-tuned/evaluated them under the same splits, prompts, decoding, and scripts as RadHiera to ensure fair comparison rather than directly copying leaderboard numbers. We will release code/checkpoints/scripts and submit/report ReXRank private-test results when available.



Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The paper addresses the structural dependency between the “Findings” and “Impression” sections of radiology reports through a hierarchical reinforcement learning framework. The model optimizes a composite reward function that balances linguistic quality, diagnostic severity, and cross-section consistency. The core contribution is a hierarchical supervision strategy that ensures the diagnostic conclusions in the Impression section are logically supported by the clinical evidence described in the Findings.

    While the paper presents good contributions, its recommendation for acceptance is contingent upon the authors clarifying the following points during the rebuttal phase:

    1.Comment on the technical difference between the proposed method and UniRG (Liu et al., 2026).

    2.The “Consistency Reward” (Eq. 9) is criticized for its simplistic. Address the “subset” flaw. Explain how the model avoids being overly conservative or missing findings, and discuss why a soft or more nuanced consistency metric wasn’t used.

    3.The framework relies almost entirely on CheXbert for clinical rewards, severity mapping, and final evaluation. This creates a circularity where the model is essentially trained to “hack” the CheXbert classifier. Improvements in CheXbert-F1 may simply reflect the model’s ability to mimic the classifier’s biases rather than a genuine increase in clinical accuracy or reporting quality. Comment on the evidence that clinical improvements are genuine and not just the result of metric alignment with the CheXbert classifier.

    4.The paper omits several crucial details necessary for reproducibility: (1) the three primary rewards are summed without weights, with no justification for how potential conflicts between fluency and clinical accuracy are resolved; (2) The specific calculations for “syntax reward” and the mechanism for “extracting clinical entities” are not sufficiently explained; (3) Results for MedGemma and MedVersa differ from published RexRank scores, and it is unclear if this is due to re-training, different checkpoints, or altered evaluation settings.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The authors have addressed the reviewers’ comments well. All reviewers have now agreed on acceptance of the paper.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The paper proposes RadHiera, a hierarchical reinforcement learning framework for medical report generation that enforces consistency between Findings and Impression sections via a multi-level reward function. Reviewers raised several concerns: (1) lack of comparison to the concurrent UniRG work; (2) the simplistic binary subset constraint for consistency reward; (3) potential reward hacking due to heavy reliance on CheXbert for both training and evaluation; and (4) missing implementation details affecting reproducibility. The rebuttal successfully clarified the paper’s novelty relative to UniRG, empirically validated the subset constraint, and provided a strong counterexample to the reward-hacking critique. Reviewers acknowledged these points, with R2 and R3 raising their scores from weak reject to accept. Given the thorough rebuttal and the clinical relevance of enforcing Findings-Impression consistency, the paper meets the acceptance threshold. The authors should incorporate the rebuttal clarifications (UniRG comparison, subset statistics, CheXbert-only baseline results) into the final manuscript in the revision.



back to top