Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Structured reporting converts free-text radiology narratives into queryable data keys, facilitating cohort assembly, longitudinal tracking, and training label generation for medical AI. The prevailing paradigm follows a two-stage pipeline: (1) constructing a reporting template, (2) extracting information to populate it. While the extraction stage has benefited from advances in large language models (LLMs), template construction remains a manual bottleneck relying on labor-intensive expert consensus that is static, difficult to scale, and may fail to capture real-world reporting diversity. We address this limitation with ASTAR, an LLM-based framework for Automated induction of STAndardized radiology Reporting templates from large-scale clinical free-text corpora. Extensive experiments on 4,215 fetal brain MRI reports from multiple centers demonstrate that, in this reporting scenario, the ASTAR-induced template surpasses two expert-curated templates across template coverage, information fidelity, diagnostic fidelity, and expert-rated usability, reducing template development from weeks of committee deliberation to hours of automated processing. Code: https://github.com/birthlab/ASTAR

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0020_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/birthlab/ASTAR

Link to the Dataset(s)

N/A

BibTex

@InProceedings{ZhaXin_ASTAR_MICCAI2026,
        author = { Zhang, Xinfeng AND Liu, Mingxuan AND Chen, Yifei AND Zhu, Juncheng AND Anmahapong, Kasidit AND Huang, Yiming AND Zhang, Yuan AND Yang, Hongjia AND Liao, Yi AND Ning, Gang AND Qu, Haibo AND Tian, Qiyuan},
        title = { { ASTAR: Automated Induction of STAndardized Radiology Reporting Templates from Large-Scale Clinical Free-Text Corpora } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16878},
        month = {September},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper proposes ASTAR, an LLM-based framework for automatically inducing standardized radiology reporting templates from large-scale free-text clinical report corpora.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The paper identifies a bottleneck in the structured reporting pipeline, no prior automated method exists for constructing the template itself.
    • The three-level evaluation protocol is a genuine contribution. Template quality, structuring fidelity, and radiologist judgment together provide a multi-faceted assessment.
    • Including 115 out-of-distribution reports from two additional hospitals strengthens the generalizability claims and is a good experimental practice.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • Qwen-Max is used both for template construction and for multiple evaluation steps. This introduces a systematic bias. The same model that shaped the template’s vocabulary and structure is judging whether test report keys match the template and whether reconstructed reports are faithful. The template that Qwen-Max built is naturally more aligned with Qwen-Max’s own semantic representations, potentially inflating coverage and fidelity scores for ASTAR relative to the expert-designed baselines.
    • The ESPR and FetalExtract templates are general-purpose, designed to cover fetal MRI reporting broadly across institutions and languages. The ASTAR template is induced from 4,000 reports from the same institution and population distribution as the 100 ID test reports. It is therefore unsurprising that a data-driven template optimized on the training distribution achieves higher coverage on in-distribution data. A fairer comparison would include constructing ASTAR templates from one institution and testing exclusively on reports from other institutions, or comparing against a template manually curated from the same corpus.
    • All experiments are conducted on fetal brain MRI reports. While the authors frame ASTAR as a general framework, no evidence is provided that it generalizes to other anatomical regions, imaging modalities, or report types. The title and abstract suggest broad applicability (“large-scale clinical free-text corpora”), but the actual demonstration is narrow. At minimum, a second domain would substantially strengthen the paper.
    • In Fig. 4A, the box plots for case-level key coverage rate appear to contain values below 0.Since coverage is defined as a ratio of matched keys to total keys (Eq. 1), it is strictly bounded in [0,1], and negative values are impossible. This appears to be a plotting artifact. The authors should verify and correct this figure, as it undermines confidence in the reported distribution statistics.
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper addresses a critical gap and offers a thoughtful evaluation protocol, but the dual role of Qwen-Max as both builder and evaluator introduces an unaddressed circular bias that undermines the reported gains. Combined with an asymmetric baseline comparison and experiments limited to a single subspecialty, the evidence does not sufficiently support the paper’s broad claims. I would reconsider if the evaluation circularity and generalizability concerns are addressed.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    Thanks for the authors’ response. Most of my concerns have been addressed.



Review #2

  • Please describe the contribution of the paper

    This paper proposes a methodology for constructing standardized radiology report templates from free-text radiology report corpora.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The proposed data-driven approach yields radiology report templates with higher coverage and fidelity compared to conventional expert-defined templates.
    • Performance improvements are demonstrated through both in-distribution and out-of-distribution evaluations.
    • The three-stage pipeline (concept extraction → tree construction → refinement) provides a well-structured framework for template generation.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • As corpus size increases, the resulting template size appears to grow accordingly. While a merging strategy for semantically similar terms exists in the refinement stage, experiments on large-scale report corpora are needed to validate scalability.
    • The study concludes at the level of template evaluation. Additional experiments assessing how actual reports map onto the generated templates, and the accuracy of such mappings, would strengthen the work.
    • No concrete examples of generated templates are presented in the paper, making it difficult to assess the results. Although examples are available via the anonymous code repository, including them in the paper would improve clarity.
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Standardizing large-scale free-text radiology report corpora is an important yet underexplored problem. The attempt to address this through automated template construction is a meaningful contribution.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The authors have adequately addressed my concerns in their rebuttal. Their commitment to extending the open-source code with additional modality templates is a valuable contribution to the community.



Review #3

  • Please describe the contribution of the paper

    This paper proposes ASTAR, a LLM framework designed to automatically induce and construct standardized radiology reporting templates from large-scale clinical free-text corpora. To overcome the bottleneck of manual, expert-consensus-driven template creation, ASTAR automates the process through three stages: concept slot extraction, hierarchical tree construction, and template refinement. The authors evaluate the framework on 4,215 fetal brain MRI reports across three centers and introduce a novel three-level evaluation protocol. The results demonstrate that the ASTAR-induced template outperforms two expert-curated baseline templates in coverage, information fidelity, diagnostic fidelity, and radiologist-rated usability.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper demonstrates high clinical utility by successfully addressing the labor-intensive bottleneck of manual template construction in medical NLP through an efficient, data-driven automated framework. This approach is rigorously assessed using a comprehensive evaluation protocol that commendably combines traditional automated metrics with subjective evaluations from clinical experts. Furthermore, the authors validate the model’s cross-institutional generalization capabilities by testing it on out-of-distribution data from multiple hospitals. To solidify their findings, the work also includes a robust scaling behavior analysis, which convincingly illustrates how the method progressively consolidates rare, long-tail concepts as the training corpus expands.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    Primarily, the baseline comparison introduces an inherent bias by evaluating the bottom-up, data-driven ASTAR against top-down expert templates, predictably yielding higher coverage for the proposed method due to its reliance on historical reports rather than standardized medical taxonomy, which risks overfitting to specific institutional writing habits. Additionally, the paper lacks a critical qualitative error analysis to highlight inevitable language model hallucinations, such as nonsensical anatomical branches or the improper merging of concepts, making it difficult for readers to gauge the system’s real-world reliability. The validity of the clinical metrics is also questionable, as the complete reliance on an “LLM-as-a-Judge” protocol to evaluate outputs from another language model risks introducing preference bias and critically lacks human expert validation. Finally, the authors overclaim the scope of their framework; while framed as a general solution for radiology reporting templates, the experiments are strictly confined to a single, narrow modality, necessitating a clearer definition of the validated clinical scenario and an expanded discussion on the challenges of migrating this approach to more complex, whole-body imaging.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper addresses a highly relevant and urgent bottleneck in medical imaging informatics (the manual creation of structured reporting templates). The workload is solid, the proposed methodology is highly automated, and the three-level evaluation framework is notably rigorous. The decision is a Weak Accept because the authors need to use the rebuttal period to address critical concerns regarding the fairness of comparing a data-driven template to expert guidelines, the lack of human validation for the LLM-as-a-judge clinical metrics, and the absence of qualitative error analysis.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The authors have provided a solid and convincing rebuttal that adequately addresses the core concerns regarding LLM evaluation bias, human validation, and scalability. Specifically, testing GPT as an independent judge effectively mitigates the circularity concern, and the template evaluation by two independent radiologists reinforces the clinical relevance of ASTAR. The explanation of the field-control mechanism also clarifies the scalability doubts. Given these solid clarifications and new analysis, I am satisfied with the response and maintain my recommendation of Weak Accept.



Author Feedback

LLM evaluation bias (R1R3) We thank the reviewers for raising this important concern. To reduce potential circularity from using Qwen-Max in both template induction and evaluation, we tested GPT as an independent evaluator. The ranking remained consistent across templates, e.g., [ROUGE-L (F1)]: ESPR [0.639], FetalExtract [0.406], ASTAR [0.729], supporting that ASTAR’s advantage is not solely due to Qwen-specific preference. We will add this analysis and clarify the role of LLM-assisted metrics in the revised paper.

Human validation of LLM-as-Judge metrics (R3) Since the evaluation involves many reports (ID n=100, OoD n=115), exhaustive human scoring of diagnostic-fidelity metrics is time-consuming, expert-dependent, and impractical. Instead, we designed a structured questionnaire for template-level human evaluation. Two radiologists from different countries rated all templates on structure, usability, and clinical impact, supporting the clinical usability of ASTAR-induced templates.

Fair comparison and OoD robustness (R1R3) (1) Our paper already includes cross-institutional OoD results. ASTAR was induced from one institution and tested on reports from two external hospitals (n=115), where it still outperformed ESPR and FetalExtract. This suggests that ASTAR is not only learning institution-specific writing style. (2) We also invited two radiologists from different countries to evaluate the templates themselves. This evaluation does not depend on any test dataset, so it provides institution-independent evidence for clinical usability. (3) In addition, ASTAR allows each institution to easily induce its own template from local report corpora, while predefined templates lack such institution-specific flexibility.

Generalizability and scope (R1R3) ASTAR is a text-only template-induction framework operating on free-text reports without image annotations or modality-specific imaging features. Thus, it can in principle be applied to other reporting domains. In fact, we have also tested ASTAR on other imaging modalities, but these results were not included due to space limitations. After acceptance, we will add more modality templates (e.g., chest CT, chest X-ray, and whole-body PET/CT) to the repository, and the open-source code allows institutions to test ASTAR on their own report corpora from different imaging modalities.

Scalability and template growth (R2) In practice, we found that the ASTAR template does not grow endlessly as the corpus size increases; instead, the number of keys tends to stabilize. This is because ASTAR controls redundant fields through span deduplication, slot merging, semantic reorganization, and global harmonization. We agree that adding a curve of key number versus corpus size would make this point clearer and avoid possible confusion.

Report-to-template mapping (R2) Due to space limits, the paper did not include mapping examples or quantitative mapping experiments. We agree that adding an evaluation of text-structuring accuracy would further strengthen this study. We will include this evaluation in our future work.

Qualitative errors and template examples (R2R3) (1) In the radiologist evaluation, the two radiologists did not report severe nonsensical branching or clinically unusable structures. ASTAR uses span deduplication, global slot merging, intra-node semantic reorganization, and global concept harmonization to reduce such risks. (2) Due to space limitations, template visualization was not included in the main paper, but the complete induced template JSON file is provided in our open-source repository. (3) We agree that visualizing qualitative errors and adding error analysis would further strengthen the study.

Fig. 4A clarification (R1) We rechecked Fig. 4A and the underlying data: all coverage values are within [0,1]. The apparent out-of-range impression is due to visual presentation. We will improve the axis labeling and visualization to avoid ambiguity.



Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper studies an important and underexplored problem, and the reviewers appreciated the practical motivation and the overall evaluation setup. At the same time, the paper remains borderline because the main concerns are about evaluation fairness and claim scope. The authors should clarify the extent to which the same LLM is used in both template construction and evaluation, explain the fairness of the comparison against expert-designed templates, and better align the scope of the claims with the fetal brain MRI setting actually studied in the paper.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    I recommend acceptance. All three reviewers changed or maintained their final post-rebuttal opinion as Accept, including Reviewer 1 who initially gave a Weak Reject. The rebuttal appears to have adequately addressed the major concerns regarding LLM evaluation bias, fairness of baseline comparison, scalability, and human validation.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The reviewers appreciated the rebuttal and increased their scores. The paper deserves its accept.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The authors answered the reviewer’s concerns appropriately, in particular: 1/ Adding a testing with GPT to mitigate the circularity constraint raised by R1 and R3 ; 2/ Added template evaluation by two independent radiologists for clinical relevance ; 3/ Extended the open-source code with additional modality templates. Therefore, given the relevance of this work to the community, I recommand acceptance.



back to top