Abstract

Accurate chest X-ray interpretation is inherently hierarchical. Clinical decisions depend not only on what abnormality is present but where it is situated, requiring reasoning from broad anatomical systems down to specific pathological findings. Yet existing automated systems largely treat this as a flat classification problem, failing to capture inter-level dependencies or enforce coherence between coarse and fine predictions. We propose CHASE (Classification with Hierarchical Analysis and Structured Enforcement), a unified single-stage framework that mirrors radiologists’ coarse-to-fine reasoning through a clinically-driven three-level taxonomy of 9 anatomical regions, 17 sub-regions, and 28 pathological findings. CHASE jointly optimizes multi-level supervision, cross-level probability alignment, and a hierarchy-violation penalty within a shared Vision Transformer backbone. This ensures that fine-grained findings are anatomically supported by their coarser-level context rather than predicted in isolation. Experiments demonstrate that CHASE outperforms flat and hierarchical baselines across all levels while achieving superior probabilistic hierarchy consistency, with level-wise attention maps confirming anatomically grounded predictions. Code is available at: https://github.com/yejix-ai/CHASE.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1023_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/yejix-ai/CHASE

Link to the Dataset(s)

https://www.physionet.org/content/mimic-cxr-jpg/2.1.0/

BibTex

@InProceedings{MooJon_ClinicallyGrounded_MICCAI2026,
        author = { Moon, Jong Hak AND Kim, Minjun},
        title = { { Clinically-Grounded Hierarchical Classification for Consistent Chest X-ray Interpretation } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16886},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    This paper proposes CHASE (Classification with Hierarchical Analysis and Structured Enforcement), a hierarchical multi-label classification algorithm for chest radiographs. At its core, it integrates a hierarchical paradigm into a single-stage framework by constructing a taxonomy covering anatomical systems, subregions, and pathological findings to simulate the coarse-to-fine reading workflow of radiologists. Based on this design, two complementary regularization schemes—global KL alignment and hierarchy-violation penalty—are introduced to ensure that fine-grained predictions are supported by coarse-grained anatomical context. Hierarchy-specific classification heads are attached to different depths (Blocks 6, 9, and 12) of a shared ViT backbone to enable end-to-end joint optimization across multiple levels, avoiding error accumulation in conventional two-stage pipelines. Additionally, a probabilistic consistency metric is designed to quantify the probability monotonicity of model predictions along hierarchical paths. As a result, accurate classification at each level is achieved.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The method is well-designed and innovative. The authors propose the CHASE framework, which effectively models radiologists’ clinical “coarse-to-fine” reading workflow as a hierarchical classification system, aligning closely with real-world diagnostic reasoning. Notably, the Hierarchy-Violation Penalty (HVP) loss function employs an asymmetric design that permits the model to make coarse-grained predictions (e. g. , abnormality at an anatomical region level) before committing to fine-grained pathological labels, reflecting a deep understanding of clinical decision-making. 2.The experimental design is rigorous and well-controlled. All baseline methods share the same EVA-X backbone, effectively eliminating confounding factors due to differences in feature representation capacity. Furthermore, the authors conduct systematic ablation studies—including careful tuning of hyperparameters α and β—to clearly demonstrate the contribution of each component to overall performance. 3.Experiments are conducted on a large-scale dataset and demonstrate competitive results. CHASE achieves or approaches the performance of level-specific single-stage oracles across the three hierarchies. It also shows significant improvements in hierarchical probabilistic consistency, and attention visualizations confirm its strong alignment with anatomical structures, boosting interpretability. 4.The work is highly reproducible. The authors provide an anonymous code repository and include detailed descriptions of training protocols, hyperparameter settings, and implementation specifics, laying a solid foundation for future research and validation.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The flowchart lacks clear visual logic and appears slightly blurry. 2.Validation is performed only on the single MIMIC-CXR dataset. 3.The paper does not discuss the limitations of the proposed method.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    My overall score is based on a comprehensive assessment of the paper’s strengths and weaknesses. The proposed CHASE framework is logically designed, interpretable, and highly innovative. Its hierarchical classification strategy closely follows the clinical coarse-to-fine reading workflow, and the loss design is reasonable and consistent with real clinical decision-making. The experimental setup is rigorous with fair baselines and thorough ablation studies. The method achieves competitive performance on a large-scale dataset, while significantly improving hierarchical probabilistic consistency and interpretability. In addition, the paper provides public code with detailed configurations, ensuring strong reproducibility. Although the paper has some drawbacks, such as an unclear flowchart, validation on only a single dataset, and a lack of discussion on limitations, these issues do not overshadow its considerable merits. In summary, I assign an overall score of 5.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    The paper introduces CHASE (Classification with Hierarchical Analysis and Structured Enforcement), a single-stage framework designed to address the lack of structural consistency in automated chest X-ray (CXR) interpretation. Unlike standard “flat” classification models, CHASE utilizes a clinically-grounded three-level taxonomy (9 regions, 17 sub-regions, and 28 findings) to mirror the coarse-to-fine reasoning of radiologists. Built on a Vision Transformer (ViT) backbone, the model employs level-specific heads at varying depths to capture multi-granularity features.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    (a) Superior Probabilistic Consistency: CHASE significantly improves hierarchy consistency, achieving a probabilistic consistency, outperforming both flat and hierarchical baselines like H-CAST. (b) Anatomically Grounded Interpretability: Level-wise attention maps confirm that the model’s reasoning follows a logical spatial progression: coarse-level heads focus on broad systems, while leaf-level heads isolate specific localized abnormalities.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    (a) Baseline Selection: While the authors compare against H-CAST and Hier-ViT, the comparison could be strengthened by including more recent CXR-specific hierarchical models or anatomy-aware approaches mentioned in the introduction. (b) Insufficient Comparison with Modern Foundation Models and VLMs:Although the authors use EVA-X as a shared backbone to ensure a fair comparison , the evaluation lacks a direct performance benchmark against state-of-the-art (SOTA) Vision Foundation Models (e.g., Ark, EXA-X, RAD-DINO) or Vision-Language Models (VLMs) specifically fine-tuned for chest X-ray tasks (e.g., Med-CLIP, BioViL, or newer multimodal architectures).Without these comparisons, it remains unclear whether the proposed hierarchical constraints offer significant advantages over the implicit multi-task or reasoning capabilities inherent in large-scale pre-trained medical VLMs. (c) Lack of External Validation: Performance is reported exclusively on the MIMIC-CXR/Chest ImaGenome dataset. Validation on a distinct external cohort is necessary to prove the robustness of the hierarchical constraints across different clinical settings and imaging protocols. (d) Selection of Supervision Depths. The authors attach level-specific heads to Blocks 6, 9, and 12, claiming these represent the lower, middle, and final thirds of the backbone to ensure “well-separated supervisory signals”. However, there is no empirical evidence (e.g., an ablation study on different block combinations) provided to justify why these specific layers are optimal for the Top, Mid, and Leaf levels.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    While CHASE demonstrates probabilistic consistency and offers promising interpretability through level-wise attention maps, the lack of architectural justification, missing SOTA benchmarks, and absence of external validation currently outweigh the contributions. The paper would be significantly strengthened by addressing these empirical gaps.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Reject

  • [Post rebuttal] Please justify your final decision from above.

    Thank you for the thorough rebuttal. I appreciate the authors’ efforts in addressing my concerns. I would encourage the authors to include evaluation on an additional dataset and comparison with modern foundation models or VLMs in a future revision, as these remain the main factors behind my current score. I maintain my weak reject.



Review #3

  • Please describe the contribution of the paper

    This paper proposes the CHASE (Classification with Hierarchical Analysis and Structured Enforcement) framework to address the inconsistency between coarse-grained and fine-grained predictions in chest X-ray (CXR) multi-label classification. The authors establish a three-tier classification system (region, subregion, and specific lesions) that mimics the diagnostic logic of real radiologists, and integrate hierarchical constraints into the Vision Transformer via Global KL Alignment and the Hierarchy-Violation Penalty (HVP).

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The problem formulation bears clear clinical motivation. Instead of simply treating chest radiograph analysis as flat multi-label classification, this paper highlights the hierarchical relationship of region–subregion–pathological finding. Such a design aligns well with real clinical reading workflows and effectively addresses the common granularity inconsistency problem existing in automated chest X-ray diagnosis systems. The proposed method is comprehensively designed with clear logic. CHASE integrates multi-layer BCE loss, global KL alignment, and hierarchy-violation penalty, where each component serves a distinct purpose. This joint design guarantees discriminative ability at all hierarchical levels, enforces cross-layer distribution consistency, and explicitly suppresses parent–child hierarchical contradictions. Even readers outside this specialized research domain can easily follow the technical motivation and overall pipeline of the proposed framework. The experiments are conducted at a large scale with standardized settings. All experiments adopt publicly available large-scale chest X-ray datasets. Moreover, all compared models share the identical EVA-X backbone, which eliminates performance gains brought by backbone replacement and ensures fair comparisons. The evaluation considers not only classification accuracy but also hierarchical consistency. The paper reports AUC and ACC across different hierarchical levels and further introduces dedicated consistency metrics. This provides direct quantitative evidence to verify whether the proposed method fundamentally alleviates the core issue of hierarchical prediction conflicts. The manuscript is well-organized and clearly written. Strong coherence is maintained among Figure 1, Table 1 and the methodology section. The correlations among the hierarchical taxonomy, loss functions and overall training objectives are explicitly illustrated and easy to understand.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    From a methodological perspective, the innovation leans more toward structural integration; the sense of a major methodological breakthrough is not particularly strong.

    Some aspects of the design still appear somewhat heuristic, such as the selection of where the heads are attached at different levels.

    The paper does a good job of proving improvements in consistency, but how these improvements translate into greater clinical utility could be further illustrated through more error case studies.

    I am not an expert in this specific sub-field; therefore, I have limited confidence in judging whether the taxonomy itself is optimal or if it covers the most critical clinical hierarchies.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I recommend a Weak Accept. The primary reason is that the problem formulation is sound and has a clear correspondence with clinical interpretation workflows. While the methodological design is not particularly aggressive, it is comprehensive and well-supported by sufficient experiments, specifically validating the core objective of hierarchical consistency.

    My reservations lie in the fact that the methodological novelty is above average rather than exceptional, and some design elements remain somewhat heuristic. Furthermore, as I am not an expert in this specific field, I am providing a cautious Weak Accept overall.

  • Reviewer confidence

    Not confident (1)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Author Feedback

We thank all reviewers and the meta-reviewer for their constructive feedback. R1/R3 highlighted the clinical relevance, rigorous ablations, reproducibility, and hierarchy-consistency improvements of CHASE, while R2 raised concerns regarding baselines, architectural justification, and generalization. Per MICCAI 2026 guidelines, we do not report new experimental results and focus on clarifying the submitted evidence. We address the major points below.

1.Additional baselines (R2) CHASE is a hierarchy-consistency framework, not a SOTA representation model. Our controlled protocol evaluates Flat-ViT, Hier-ViT, H-CAST, and CHASE under the same backbone and unified 54-label space, directly testing whether explicit hierarchical supervision improves anatomically coherent Top-Mid-Leaf predictions.

Prior hierarchical and anatomy-aware CXR methods are related but not directly comparable: CXR hierarchical classifiers typically use limited pathology-centered labels without our Region-Subregion-Finding structure, and anatomy-aware detection models do not define outputs over our 54-label hierarchy. Re-formulating them under our taxonomy and 197 valid anatomical paths would be a prerequisite; otherwise, comparisons conflate different output spaces and evaluation targets.

Table 3 provides direct evidence: removing KL and HVP reduces probabilistic consistency from 74.26 to 64.89, showing that cross-level coherence is not obtained by joint multi-label training alone. In the final version, we will discuss RAD-DINO-v2 and Ark+ as complementary backbone-strength references, distinct from CHASE’s hierarchy-consistency contribution. For VLMs such as BioViL, applying our consistency metrics would require prompt design, text-to-label mapping, and hierarchy-path decoding; we will discuss structured VLM evaluation as future work.

2.Architectural design choices and novelty (R2, R3) We appreciate R3’s comment that the novelty is closer to structural integration than to a single isolated algorithmic component. This reflects our intended contribution: to our knowledge, CHASE is the first single-stage CXR framework to explicitly enforce probabilistic consistency across a dense Top-Mid-Leaf anatomical hierarchy within a shared ViT backbone.

This unified design requires separating supervision across hierarchy levels rather than relying on a single late representation for all predictions. Blocks 6, 9, and 12 were selected to balance discrimination and inter-level separation: earlier settings such as [1,2,12] favored consistency but weakened discrimination, while later settings such as [5,11,12] showed the opposite trend. Blocks [6,9,12] provided the best trade-off. We will clarify this rationale in Sec. 2.2. 3.External validation and limitations (R1, R2, R3) We acknowledge single-source evaluation as a limitation. Chest ImaGenome remains, to our knowledge, the only public CXR dataset providing dense cross-level anatomical linkages required for rigorous Top-Mid-Leaf consistency evaluation. Datasets such as CheXpert lack the Region-Subregion-Finding structure and support only partial leaf-level transferability assessment. We will expand the Limitations section to cover dataset constraints, taxonomy adaptability, and broader validation opportunities.

4.Clinical utility (R3) The asymmetric HVP targets the clinically critical error of activating fine-grained findings without parent-region support, reflecting how radiologists form broad anatomical suspicion before specific diagnoses. A representative flat-baseline failure is a broken hierarchy path, e.g., predicting lung and lung opacity along “lung → left upper → opacity” while assigning insufficient probability to the intermediate “left upper” node. Figure 2 supports CHASE’s intended behavior through attention maps progressing from broad regions to localized findings. We will use such examples to strengthen the clinical discussion.

We believe these clarifications address the reviewers’ concerns.




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper received three good quality reviews with two accept and one reject rating. The final score will be depended on the rebuttal quality and details.

    The paper proposes a clinically motivated hierarchical chest X-ray classification framework that models region–subregion–finding relationships and improves hierarchical probabilistic consistency while maintaining competitive predictive performance and interpretable anatomy-aligned attention. Reviewers generally found the formulation clear, reproducible, and well supported by ablations, but noted that the contribution is more of a thoughtful structural integration than a major methodological breakthrough. The authors should address the lack of external validation beyond MIMIC-CXR/Chest ImaGenome, add stronger comparisons to recent CXR-specific hierarchical/anatomy-aware models and modern medical foundation/VLM baselines, justify the heuristic choice of supervision layers, improve the flowchart/figures, and include a clearer limitations and clinical-utility discussion with error-case analysis.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    From a comprehensive overall review of this paper for its positives and negatives, this paper is clearly a well developed work that should be accepted by MICCAI.

    All reviewers and the primary AC would agree that the proposed CHASE framework is logically designed, interpretable, and sufficiently innovative. Its hierarchical classification strategy follows the clinical coarse-to-fine reading workflow, and the loss design is logical, solid and consistent with real clinical decision-making. The experimental setup is rigorous with fair baselines and thorough ablation studies. Only one dataset was used for evaluation but this may not be a big issue. Some of the negative issues will be fixed in the future version as discussed in the rebuttal.

    “The method achieves competitive performance on a large-scale dataset, while significantly improving hierarchical probabilistic consistency and interpretability.

    In addition, the paper provides public code with detailed configurations, ensuring strong reproducibility.”



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The paper probes the architecture of ViT to extract hierarchical information that would have been otherwise straightforward to capture in CNN architectures. The experiments are showing that extracting this information helps keep the model anatomically consistent in its predictions. The rebuttal addresses most of the concerns.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The paper presents a clinically grounded hierarchical classification framework for chest X-ray interpretation. Reviewers also praised the clear formulation, rigorous ablation studies, strong reproducibility, and the anatomically interpretable attention maps. In rebuttal, the authors clarified that their contribution is a hierarchy-consistency framework rather than a representation learning benchmark, provided empirical reasoning for the block-level head placement, and acknowledged the dataset limitations along with plans to expand the discussion of clinical utility and error cases. Two of the three reviewers ultimately support acceptance, and the rebuttal satisfactorily addressed the main weaknesses.



Meta-review #4

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    CHASE presents a clinically grounded hierarchical classification framework that meaningfully improves probabilistic consistency across anatomical region–subregion–finding levels while maintaining competitive predictive performance. The method’s strength lies not in a single algorithmic breakthrough but in the thoughtful structural integration of clinical reasoning into a single-stage ViT, i.e., a contribution that reviewers acknowledged as well-designed, reproducible, and well-supported by ablations. The rebuttal effectively clarifies scope boundaries: CHASE is a consistency framework, not a representation leader, and comparisons to VLMs or anatomy-aware detectors would require non-trivial output-space reformulation. Remaining concerns, including single-dataset validation (MIMIC-CXR/Chest ImaGenome) and absence of direct benchmarks against modern medical foundation models (e.g., RAD-DINO, BioViL), are legitimate but not fatal. The authors appropriately acknowledge these as limitations and future work. Given the practical constraints of public datasets offering dense hierarchical annotations, the paper meets the bar for acceptance. I recommend acceptance, with encouragement to pursue external validation and VLM comparisons in future extensions.



back to top