Abstract

The automatic generation of medical reports utilizing Multimodal Large Language Models (MLLMs) frequently encounters challenges related to factual instability, which may manifest as the omission of findings or the incorporation of inaccurate information, thereby constraining their applicability in clinical settings. Current methodologies typically produce reports based directly on image features, which inherently lack a definitive factual basis. In response to this limitation, we introduce Fact-Flow, an innovative framework that separates the process of visual fact identification from the generation of reports. This is achieved by initially predicting clinical findings from the image, which subsequently directs the MLLM to produce a report that is factually precise. A pivotal advancement of our approach is a pipeline that leverages a Large Language Model (LLM) to autonomously create a dataset of labeled medical findings, effectively eliminating the need for expensive manual annotation. Extensive experimental evaluations conducted on two disease-focused medical datasets validate the efficacy of our method, demonstrating improved tuberculosis clinical efficacy and most NLG metrics over state-of-the-art models. The code is available at https://github.com/UCPRER/Fact-Flow.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1671_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/UCPRER/Fact-Flow

Link to the Dataset(s)

Montgomery County CXR Set: https://www.kaggle.com/datasets/raddar/tuberculosis-chest-xrays-montgomery Shenzhen Hospital CXR Set: https://www.kaggle.com/datasets/raddar/tuberculosis-chest-xrays-shenzhen

BibTex

@InProceedings{YanCun_LLMBootstrapped_MICCAI2026,
        author = { Yang, Cunyuan AND Song, Dejuan AND Pang, Xiaotao AND Shen, Qianqian AND Nie, Wenjie AND Huang, Yifan AND Han, Wei AND Bu, Jiajun AND Wu, Lei AND Wang, Haishuai},
        title = { { LLM-Bootstrapped Targeted Finding Guidance for Factual MLLM-based Medical Report Generation } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16879},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    This paper proposes Fact-Flow, a framework for reducing factual errors in medical report generation. The key idea is to first extract clinical findings from medical images and then generate reports conditioned on these structured intermediate representations. In addition, the authors leverage large language models (LLMs) to automatically generate labeled clinical finding data, aiming to reduce the cost of manual annotation. Experimental results demonstrate improved factual correctness compared to existing methods while maintaining high linguistic quality.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The paper tackles an important problem in medical report generation, namely factual errors (hallucinations). Introducing clinical findings as an explicit intermediate representation is intuitive and aligns well with real-world clinical workflows, effectively decoupling visual understanding from language generation. 2.The use of LLMs to automatically construct a label taxonomy is a practical and meaningful contribution, addressing the high cost of annotation in medical datasets. 3.The method shows consistent performance improvements across multiple MLLMs and datasets, providing reasonable evidence for its generalizability and effectiveness.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.While the MLLM is prompted with both vision tokens and classification labels, the paper does not provide a comparison with a setup where only prompts (without vision tokens) are used. This makes it difficult to isolate the contribution of visual features to report generation performance. 2.The discussion of related work is somewhat limited. Although the paper covers the general progression from CNN-RNN to Transformer and MLLM-based approaches, it lacks a comprehensive review of closely related work on fact-guided or structured report generation, which is directly relevant to the proposed method. 3.The constructed label set is not sufficiently described in terms of concrete examples and granularity. Without a clear understanding of how concepts are organized (e. g. , handling of similar findings, distinction between general findings and severity levels), it is difficult to assess the clinical validity and appropriateness of the taxonomy. 4.The proposed framework consists of multiple components (label taxonomy construction, multi-label classification, and fact-guided generation), but the current experiments do not sufficiently disentangle their individual contributions. As a result, it is unclear which component is primarily responsible for the observed performance gains. 5.Although comparisons with an oracle (ground-truth labels) are provided, there is limited analysis of the gap between predicted and ground-truth labels. This makes it difficult to identify bottlenecks in the pipeline. Furthermore, the interpretation of evaluation metrics, including clinical efficacy, is not sufficiently explained, making it harder to judge the practical clinical significance of the improvements.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    To strengthen the paper and clarify the effectiveness of the proposed method, the following points should be addressed: 1.Include an ablation study evaluating report generation without vision tokens (i. e. , using only fact-flow prompts) to better isolate the contribution of visual inputs. Expand the related work section to more clearly position the method with respect to fact-guided and structured report generation approaches, improving clarity on novelty and contribution. 2.Provide a more detailed description of the constructed label taxonomy, including representative examples, granularity, and design principles (e. g. , handling of diseases, findings, anatomical locations, and severity levels), to enhance understanding of its clinical validity and reproducibility. 3.Conduct a more thorough analysis of the individual contributions of each component (label generation, classification model, and fact-guided generation) to clarify which parts drive performance improvements. 4.Further analyze the gap between predicted labels and oracle (ground-truth) labels, discussing its impact on final report quality and identifying potential bottlenecks. 5.Provide clearer explanations of evaluation metrics such as clinical efficacy, including their definitions and clinical implications, to better contextualize the reported improvements.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper addresses an important and practical problem with a well-motivated framework and demonstrates consistent empirical improvements. However, several aspects—particularly component-wise analysis, taxonomy transparency, and evaluation clarity—require further clarification to fully establish the contribution and clinical significance of the work.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The rebuttal adequately addressed several of my concerns, particularly regarding the role of visual information, pseudo-label interpretation, and limitations of the sequential pipeline design. While some concerns regarding methodological novelty and component-level analysis remain, I believe the work is technically sound and suitable for acceptance.



Review #2

  • Please describe the contribution of the paper
    • The paper proposes Fact-Flow, a three-stage plug-and-play framework that improves factual accuracy in MLLM-based medical report generation by explicitly decoupling visual recognition from language generation.

    • Stage 1 uses an LLM (GPT-5-mini) to automatically bootstrap a multi-label clinical finding taxonomy from unstructured training reports, thereby eliminating expensive manual annotation via batch extraction, iterative hierarchical merging, and frequency-based filtering.

    • Stage 2 trains a multi-label image classifier (DINOv3-ConvNeXt) with logit adjustment for long-tail class imbalance, predicting finding presence/absence from medical images.

    • Stage 3 fine-tunes an MLLM (Qwen2.5-VL, MedGemma, or LLaVA-Med via LoRA) conditioned on the predicted label set serialised as a structured natural-language prompt, grounding generation in explicitly identified clinical facts.

    • Evaluated on a public TB chest X-ray dataset and a private multimodal Chinese ophthalmology dataset, Fact-Flow demonstrates consistent improvements in RadFact clinical efficacy and NLG metrics over directly fine-tuned MLLM baselines.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • S1: Clinically well-motivated problem framing. Factual hallucination and finding omission are the primary barriers to clinical deployment of MLLM-based reporting. The cognitive decoupling hypothesis (recognition is not narration) is principled and supported by the literature.

    • S2: Novel automated annotation pipeline. The LLM-bootstrapped taxonomy construction (batch extraction, hierarchical synonym merging, frequency filtering) eliminates the need for manual labelling in new clinical domains. As per my understanding, no prior MRG method uses an LLM to generate the multi-label training supervision from existing reports. This is the paper’s most original contribution.

    • S3: Architecture-agnostic, plug-and-play design. The framework is validated across three architecturally distinct MLLMs (Qwen2.5-VL 7B, MedGemma 4B, LLaVA-Med 7B), consistently improving all three. This breadth strengthens the generalisability claim substantially.

    • S4: Strong diagnostic of baseline failure. The mode-collapse pathology of vanilla MLLMs: Qwen2.5-VL with precision=1.0 and recall=0.0145, LLaVA-Med with all-zero clinical efficacy is clearly and quantitatively demonstrated, making a compelling case for factual grounding.

    • S5: Use of RadFact for clinical efficacy evaluation. Adopting LLM-based entailment scoring rather than n-gram metrics for the TB task is methodologically appropriate and ahead of much of the field in MRG evaluation practice.

    • S6: Oracle ablation (Table 4). The predicted vs. ground-truth label oracle experiment cleanly identifies label quality as the pipeline bottleneck, guiding future research priorities.

    I also appreciate the authors for providing code in an anonymised format for reproducibility.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • W1: Logical contradiction: long-tail filtering (Stage 1) vs. long-tail optimisation (Stage 2). Frequency-based filtering at θ=15 preemptively removes rare findings from the taxonomy. Logit adjustment in Stage 2 then attempts to re-balance rare labels, but it cannot recover classes that were deleted in Stage 1.The paper simultaneously claims to protect rare labels (Stage 2) and eliminates them as a design choice (Stage 1). I would request the authors to address this logical inconsistency in the pipeline’s stated goals.

    • W2: No clinical efficacy metric for the larger ophthalmology dataset. The ophthalmology dataset is the larger and more complex of the two. Reporting only NLG metrics (BLEU, CIDEr) for a Chinese-language clinical dataset is insufficient. Could the same GPT-5-mini used in Stage 1 for Chinese report annotation could, in principle, be repurposed for a RadFact-style clinical entailment evaluation? The absence of any factual correctness metric for the primary multimodal dataset is an evaluation gap that could be addressed.

    • W3: TB dataset seems small for reliable claims without statistical testing. 561 training / 160 test images. No confidence intervals, no significance tests, no multi-seed variance. At this scale, single-image shifts move accuracy by ~0.6%. Wilcoxon tests or bootstrapped confidence intervals are the minimum expected standard.

    • W4: No sensitivity analysis on key hyperparameters (θ, b, κ, τ). The entire Stage 1 taxonomy depends on θ=15 (yielding 7 labels for TB, 42 for ophthalmology). No experiment varies θ or examines how the downstream clinical efficacy changes with taxonomy size. This is a missing ablation.

    • W5: Classifier error propagation not analysed. Macro-F1 of 0.52 on TB means substantial per-class errors. These are injected into Stage 3 via a declarative, authoritative prompt (“The image shows the following findings…”), giving the MLLM no mechanism to express uncertainty or reject erroneous inputs. The large gap between predicted-label (F1=0.28) and oracle-label (F1=0.44) performance is reported but not decomposed. Requesting the authors to clarify which finding categories are most commonly misclassified, and how do those errors manifest in generated reports.

    • W6: Stage 1 validation sample of n=50 is statistically inadequate. 50 reports represent 2.7% of the ophthalmology training set. Perfect 100/100 coverage and matching accuracy on 50 samples from a 1,854-case dataset with 42 labels, including many rare ones, cannot be reliably extrapolated. Inter-rater agreement (Cohen’s Kappa) between the LLM and a second clinical reviewer is not reported.

    • W7: Please cite the GPTs models used.

    Once again, I congratulate the authors for their hard work in addressing an important challenge in Medical LLMs.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    NA

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper addresses an important and clinically relevant problem of factual inaccuracies in MLLM-based medical report generation. The work proposes a well-motivated three-stage framework that decouples visual recognition from language generation. The idea of using LLM-bootstrapped multi-label supervision is interesting and potentially impactful, particularly as a scalable alternative to manual annotation. The method is clearly presented and shows consistent improvements across multiple MLLMs and datasets, with appropriate use of clinical evaluation (RadFact) on the TB task. The oracle ablation further provides useful insight into the role of label quality.

    However, several weaknesses limit confidence in the results and must be addressed. There is a conceptual tension between Stage 1 filtering (removing rare labels) and Stage 2 long-tail optimisation that needs clarification. The evaluation is incomplete: the larger ophthalmology dataset lacks any clinical efficacy metric, and the TB dataset is small with no statistical significance testing or variance analysis. Key design choices (e.g., threshold θ and other hyperparameters) are not studied, and error propagation from the classifier to the generator is not analysed. Additionally, Stage 1 validation is based on only 50 samples, which is insufficient to support strong claims about annotation quality. Finally, the LLM used for label bootstrapping is not properly specified or cited, raising reproducibility concerns.

    The paper has promising ideas and solid initial results, but the above issues must be addressed. Hence, my recommendation is to invite the authors to rebuttal to justify stronger acceptance.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    After reading the rebuttal, I maintain a weak accept. The authors provide reasonable and transparent clarifications for several major concerns. In particular, their explanation of frequency filtering versus logit adjustment resolves the apparent contradiction: filtering is used as a minimum-support/noise-control step, while logit adjustment only mitigates imbalance among retained labels. They also adequately address the visual-token concern by pointing to the label-only versus image+label ablation, where visual information provides additional gains. I also appreciate that they agree to rename LLM-generated annotations as pseudo-labels and to treat the oracle rows as upper bounds within the bootstrapped label space rather than true expert ground truth.

    Important limitations remain. The lack of confidence intervals, statistical testing, or multi-seed variance still limits confidence in the reported gains, especially on the small TB test set. The larger ophthalmology dataset still lacks a factual/clinical efficacy metric, so factual correctness claims should be restricted to the TB dataset. Error propagation from the classifier to the generator is acknowledged but not decomposed by category, and the system has no uncertainty/rejection mechanism for incorrect predicted labels. Stage-1 validation remains limited at the report level, and no sensitivity analysis is provided for key hyperparameters.

    Hence, the rebuttal satisfactorily clarifies the main conceptual issues and appropriately softens several claims, but it does not fully resolve the empirical robustness concerns. I therefore recommend acceptance, but only as a borderline accept rather than a strong accept.

    I once again would like to congratulate the authors for their efforts to help grow the MICCAI community with relevant contribution towards improving the factual reliability of MLLM-based medical report generation.



Review #3

  • Please describe the contribution of the paper

    The paper proposes Fact-Flow, a novel framework that decouples visual feature recognition from medical language generation in order to produce diagnostic reports from medical images.

    Given a dataset of (Image, Report) pairs, the approach consists of three main stages:

    • Using a large language model to extract a unified label taxonomy from the reports, which is subsequently refined and filtered;
    • Training a vision-encoder-based multi-label classifier to predict the presence of each label in the taxonomy for a given image;
    • Fine-tuning a multimodal LLM to generate reports conditioned on both the input image and the predicted label set.

    The proposed method is evaluated against existing state-of-the-art approaches and achieves higher performance across several commonly used metrics.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper is well written, clearly structured, and easy to follow. It addresses a well-defined problem and situates itself appropriately within the literature on medical report generation. The proposed decomposition into visual recognition and language generation is well motivated and represents an interesting and potentially impactful contribution.

    The design choices are generally sound, and the experimental evaluation covers the key components of the framework, providing a reasonably comprehensive assessment of its effectiveness.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    My main concern is the absence of any statistical significance analysis supporting the claim that the proposed method outperforms prior work. Reporting higher metric values alone is not sufficient to establish superiority, especially when improvements are relatively small. This is particularly important for metrics such as BLEU-4, ROUGE-L, CIDEr, and METEOR, where absolute differences (e.g., +0.2) are difficult to interpret in terms of practical significance. I strongly recommend performing a paired bootstrap resampling test to compare the proposed method against baselines. This would allow the authors to:

    • Report confidence intervals for each metric
    • Assess whether improvements are statistically significant
    • Strengthen the claims made in Table 1

    Including such analysis would substantially improve the rigor of the evaluation.

    Then, the paper refers to the LLM-generated label vectors (the Y annotations) as “ground truth.” This terminology is somewhat misleading. Typically, “ground truth” implies human-verified, highly reliable annotations, whereas in this case the labels are automatically generated and therefore inherently noisy. A more appropriate term would be “pseudo-labels” or “weak labels.” This is not merely a terminological issue as it also affects how results should be interpreted. For instance, in Table 4, the oracle model using both labels and images outperforms the one using labels alone. While this result is valid, it raises an important question: would performance further improve if the labels were truly accurate (e.g., expert-annotated)? Clarifying this distinction would lead to a more precise discussion of the results.

    Also, as this is a machine learning paper, it is important to clearly specify how data is used at each stage of the pipeline. While it appears that all components are trained using only the training split, this is not stated explicitly. The authors should clarify:

    • Which data is used to derive the taxonomy
    • Which data is used to train the classifier
    • Which data is used to fine-tune the multimodal model Making this explicit would help rule out any potential data leakage or bias and improve reproducibility.

    Finally, the paper would benefit from a brief discussion of some potential limitations of the proposed framework. In particular:

    • The reliance on LLMs both for label generation (Stage 1) and, indirectly, for evaluation (e.g., via RadFact) may introduce a degree of circularity, where the system is partially aligned with the same type of model used for supervision and assessment.
    • The multi-stage nature of the pipeline (label extraction -> classification -> report generation) may lead to error propagation, where inaccuracies in intermediate predictions affect the final output. These aspects do not undermine the validity of the approach, but acknowledging and discussing them would provide a more complete picture of the method and its practical implications.
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This is a solid and well-presented paper with a promising idea and encouraging empirical results. However, the evaluation would benefit from additional rigor, particularly through statistical significance analysis and clearer reporting of experimental details.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Reject

  • [Post rebuttal] Please justify your final decision from above.

    The authors acknowledged the concern regarding the use of the term “significant” and proposed replacing it with “consistently.” I agree that this change is more appropriate in the absence of formal statistical testing. However, my main concern still remains. Such results are a core part of the paper, and therefore I believe this aspect should be supported more rigorously. Importantly, I do not view this as requiring a fundamentally new experiment. Statistical analyses such as paired bootstrap resampling are typically performed on already available model predictions over the test set and therefore generally do not require retraining models or collecting additional data. For this reason, I believe it would be reasonable for the authors to include such analysis in the final version of the paper, even if it cannot be fully incorporated during rebuttal. Regardless of whether this is formally considered a “new experiment” or not, I consider this analysis necessary to properly support the paper’s main empirical claim. Without stronger evidence that the reported improvements are robust and meaningful, I do not believe the current evaluation is sufficient for acceptance.



Author Feedback

We thank the reviewers for recognizing Fact-Flow’s motivation, automated annotation pipeline, and gains across MLLMs/datasets.

[R2W1: filtering vs. long-tail optimization] We agree the wording was unclear. Filtering and logit adjustment address different problems. Filtering is a minimum-support/noise-control step: labels in <θ reports are too weak for reliable pseudo-label supervision or stable MLC training. Logit adjustment is applied only within the retained taxonomy to mitigate imbalance among supported labels. We do not recover discarded labels, and will revise “protect rare findings” to “improve retained low-frequency findings.”

[R3/R2W3: statistical reliability] We agree CIs, paired bootstrap tests, or multi-seed variance would strengthen evaluation. Since rebuttal rules disallow new results, we will avoid statistical claims and change “significant” to “consistent.” Submitted TB results show clinical-F1 gains for all MLLMs: Qwen2.5-VL 0.0286→0.2831, MedGemma 0.2266→0.3055, and LLaVA-Med 0→0.2016.Ophthalmology results are limited to NLG metrics, where Fact-Flow improves most metrics.

[R1Q1: visual tokens] Table 4 includes Label Only (Pred), which uses predicted labels without image tokens. Adding images improves clinical F1 from 0.2115 to 0.2831; with oracle labels, from 0.3750 to 0.4421.Thus, visual evidence and factual prompts are complementary.

[R1Q4/R1Q5/R2W5/R3: components, metrics, and error propagation] Because Fact-Flow is a sequential pipeline, removing an upstream stage makes downstream stages undefined. We therefore combine intrinsic validation with end-to-end ablations. Table 5 evaluates Stage-1 taxonomy quality and Stage-2 MLC performance; Table 4 identifies Stage-2 label quality as the main bottleneck: oracle labels raise Image+Label clinical F1 from 0.2831 to 0.4421.We cannot add per-category decomposition in rebuttal, but will clarify failure modes: false positives may induce unsupported findings, false negatives may cause omissions, and coarse labels may reduce laterality/location/severity specificity. The current system has no uncertainty/rejection mechanism for incorrect Stage-2 labels. To clarify the metric interpretation, RadFact precision/recall reflect unsupported generated and omitted reference entities, respectively, and F1 balances both.

[R3: pseudo-label terminology] We agree “ground truth” is imprecise. We will rename LLM-generated annotations as “pseudo-labels.” The Table 4 oracle rows are upper bounds within the bootstrapped label space, not expert-verified ground truth.

[R2W6/R3: Stage-1 validation and circularity] The n=50 report-level check is a sanity check of report-to-label matching, not proof of noiseless supervision. It is complemented by full-taxonomy ophthalmologist validation: 80% valid/decidable, 15% context-dependent, 5% overly generic, and 0% invalid/pseudo-concepts. We will avoid overclaiming reliability. RadFact does not use our taxonomy and is not a closed loop over our labels, but is still LLM-based, so we also report NLG metrics, and we will discuss this limitation.

[R2W2: ophthalmology factual evaluation] We agree a validated clinical-efficacy metric for Chinese ophthalmology reports would strengthen the work. Building a RadFact-style evaluator would require a new evaluator/experiment; we will restrict factual-efficacy claims to TB.

[R2W4/R1Q2/R1Q3/R2W7/R3: robustness and reproducibility] b and κ are context-budget parameters, θ controls minimum support, and τ=1 follows logit-adjustment practice; all were fixed before test evaluation. Submitted experiments cover different taxonomy sizes (7 TB vs. 42 ophthalmology labels), but not a full sensitivity study, which we will state as a limitation. We will also add taxonomy examples/design principles, position fact-guided/structured MRG, specify/cite LLMs, and clarify data usage: taxonomy/classifier/MLLM training use only training data; validation is for model selection; test only for final evaluation.




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The paper proposes a three-stage framework designed to improve the factual groundedness of Multimodal Large Language Models in radiology. It utilizes a “bootstrapping” approach where an LLM automatically extracts a structured label taxonomy and pseudo-labels from existing free-text reports, bypassing the need for manual annotation. A dedicated classifier is then trained to predict these findings from images, providing a declarative “factual stream” that prompts the MLLM to generate reports aligned with specific clinical findings.

    While the paper presents good contributions, its recommendation for acceptance is contingent upon the authors clarifying the following points during the rebuttal phase:

    1.The pipeline preemptively filters out rare findings in stage 1, effectively deleting them from the taxonomy. However, Stage 2 later attempts “long-tail optimization” via logit adjustment. As reviewers noted, the model cannot optimize for classes it has already discarded, undermining the claim that the framework protects rare clinical findings. Clarify the contradiction between frequency-based filtering and long-tail optimization.

    2.The TB dataset is extremely small (160 test images), and the paper lacks confidence intervals, significance tests (e.g., Wilcoxon or paired bootstrap), or multi-seed variance reporting. Without these, the marginal gains in BLEU or CIDEr scores cannot be distinguished from random noise.

    3.The framework is highly susceptible to cascading errors. Since the MLLM is given an authoritative prompt (“The image shows the following findings…”), it has no mechanism to reject or question incorrect labels from the Stage 2 classifier. The significant performance gap between “predicted-label” and “oracle-label” setups highlights this bottleneck, yet the paper lacks a detailed analysis of which finding categories are failing most frequently. Reclassify “ground truth”, and discuss the impact of label noise on the final MLLM output.

    4.The “ground truth” labels are actually LLM-generated pseudo-labels. Validating this extraction process on only n=50 samples (less than 3% of the dataset) is statistically inadequate to prove the reliability of a 42-label taxonomy. Furthermore, using LLMs for both supervision (Stage 1) and evaluation (RadFact) introduces a risk of circularity that is not addressed.

    5.The paper treats several key hyperparameters as fixed without sensitivity analysis. It is unclear how the size of the taxonomy or the aggressiveness of the logit adjustment impacts clinical efficacy, making it difficult to assess the framework’s robustness across different medical domains.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The authors have addressed the reviewers’ comments well. Two reviewers acknowledged the significance of the work. Therefore acceptance of the paper is concluded. It is strongly recommended to include statistical test results in the final version.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Reject

  • Please justify your recommendation.

    The paper is proposing using automated ground truth labeling for a finding classifier followed by a language model to generate text containing those findings. The evaluation of such generated text is also through mostly lexically-based metrics suitable for evaluation. While one can understand the benefits of automation in annotation, it cannot be taken seriously unless there is a formal comparison with ground truthed datasets when they do exist. Besides, LLM currently mix multiple findings into single concepts, cannot easily respect laterality and location constraints, etc. so the resulting report produced is likely to mismatch ground truth report on such fine-grained aspects of what is a finding and its description. The BLEU, ROUGe and other lexical scores will continue to score high. Since the Radgraph F1 is low (0.3) this indicates that any clinical significance may still be low by this method.



back to top