Abstract

Weakly-supervised thoracic disease detection in chest X-rays (CXR) is challenging due to subtle appearances and complex anatomical overlap, motivating anatomy-aware modeling for improved localization. However, prior anatomy-aware methods typically rely on coarse region proxies or static spatial priors, which may restrict dynamic instance discovery and limit precise localization of small abnormalities. We propose Anatomy-Structured Hierarchical Multiple Instance Learning (ASH-MIL), a framework that introduces parallel anatomy-structured observation branches (cardiac, pulmonary, and agnostic) combined with hierarchical MIL aggregation. Anatomical priors are injected as soft spatial biases into decoder cross-attention, enabling anatomically grounded evidence maps without disease bounding-box supervision. Instance localization is derived directly from MIL-weighted cross-attention maps. Experiments on CXR8 and cross-domain MIMIC-CXR held-out sets demonstrate consistent improvements over prior weakly-supervised and anatomy-aware approaches, particularly under stricter localization criteria. Our code is available at https://github.com/jn-kim/ash-mil.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/4609_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/jn-kim/ash-mil

Link to the Dataset(s)

N/A

BibTex

@InProceedings{KimJeo_AnatomyStructured_MICCAI2026,
        author = { Kim, Jeongin AND Ahn, Sohyun AND Kang, Seo Young AND Sung, Jaeyi AND Kim, Soomin AND Cho, Sungho AND Lee, Rena AND Kim, Kwanchang AND Noh, Junhyug},
        title = { { Anatomy-Structured Hierarchical MIL for Weakly-Supervised Thoracic Disease Detection in Chest X-Rays } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16880},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    This paper proposes an anatomy-structured hierarchical multiple instance learning framework to incorporate anatomical priors into weakly-supervised thoracic disease detection in chest X-rays. This work details the architecture of different anatomical structures and formulates the detection as hierarchical multiple instance learning with query- and branch-level evidence aggregation. Experimental results verify the effectiveness of the proposed framework, especially its superior performance in cross-dataset experiments.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The major stregths of this paper include: (1) This paper incorporates anatomical structures from medical images as prior knowledge into medical image analysis tasks, with detailed structural design and experimental validation. (2) Furthermore, as the most fundamental and reliable priors, anatomical structures in medical images not only enhance the performance of image analysis tasks but also improve the interpretability of black-box models.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The major weaknesses of this paper include: (1) Although priors in medical imaging are commonly believed to benefit image analysis tasks, this work focuses more on learning features from different regions and fusing them with unexplainable neural networks, rather than clearly demonstrating why such a framework works well and what new problems it solves by introducing anatomical priors. Therefore, from the research motivations and the technologies, the novelty of this work is limited. (2) For chest X-rays, the anatomical structures are relatively simple, and thus the corresponding regions in images are easy to obtain using current technologies. This also limits the generalization of the proposed framework to other medical images. (3) Regarding the experiments, first, the testing datasets have a limited number of annotated detection images, with fewer than 1,000 bounding boxes. This is not very convincing for demonstrating the generalization ability of the method. Additionally, it seems that whether the priors are included or not leads to different performance and inconsistent conclusions when using one or three branches.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper proposes to incorporate anatomical structures into the medical detection task, but the motivation behind the proposed framework is not deeply introduced. Such a framework is also limited when applied to other fields of medical imaging. Some experimental results are also not convincing enough to support this method.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper proposes ASH-MIL, a weakly supervised thoracic disease detection framework for chest X-rays that combines three parallel anatomy-structured branches (cardiac, pulmonary, agnostic) with hierarchical MIL aggregation. The core idea is to inject CXAS-derived anatomy priors as soft additive biases into decoder cross-attention, then localize instances from MIL-weighted attention maps without disease box supervision.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The proposed cardiac/pulmonary/agnostic decomposition is simple but clinically relevant. 2.I like that the anatomy prior is used as a soft bias in cross-attention rather than as a hard mask. 3.On CXR8, ASH-MIL reports 15.77 mAP and improves AP/CorLoc@0.5 to 6.03/0.26; on MIMIC-CXR it reports 22.72 mAP and 7.28/0.25.4.Paper is very well written and code is opensource.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The method does not use disease boxes, but it does rely on an external anatomy segmentation model (CXAS) trained with fine-grained anatomical supervision to produce the priors. That is perfectly reasonable, but it should be stated more carefully.

    2.The paper only reports localization metrics, but not image-level classification performance, even though the model is trained with multi-label BCE and produces image-level scores.

    3.The relative gains are good, but the absolute numbers at strict localization remain low, especially CorLoc@0.5.So while this is a solid methodological improvement, it is still far from clinically deployable localization.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper is well motivated, and the method is solid good empirical results.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    I thank the authors for the rebuttal. The response addresses my main concerns. In particular, I appreciate the clarification that ASH-MIL does not use disease bounding boxes, while its anatomical priors are produced by CXAS. I encourage the authors to state this distinction explicitly in the final manuscript. I also appreciate the added image-level AUROC. Also, I appreciate the authors’ acknowledgement that strict localization remains challenging and that ASH-MIL is not clinically deployable. The relative gains are meaningful, especially at stricter IoU thresholds, and the Nodule result is encouraging. However, the absolute CorLoc@0.5 remains low overall, so the final paper should continue to frame the method as an annotation-efficient weakly supervised localization advance rather than a clinically ready detection system.



Review #3

  • Please describe the contribution of the paper

    This work proposes an MIL framework with anatomy priors for weakly-supervised thoracic disease detection in chest X-rays. Firstly, an attention-based localization framework forms a solid baseline. Secondly, segmentation masks as anatomy prior are injected into attention maps to highlight potential disease regison. Finally, features from different branches are aggregated in a hierachical manner. Experimental results show advantages over WSOD / anatomy-priored WOSD methods.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Involving prior guidance in CXR WSOD task is a practical idea.
    • The attention-based localization method forms solid advantage.
    • The prior branches is proved to be beneficial under the setting of cardiac and pulmonary priors.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • (Minor) Given 157 anatomy categories by CXAS, involving only 2 priors seems arbitrary. It is yet to be proved that this framework is universally beneficial given more anatomies, especially for cross-region diseases.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    General advantages over both WSOD and prior-guided WSOD methods.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    Generally sound method.



Author Feedback

We sincerely thank all reviewers for their constructive feedback and for recognizing ASH-MIL’s anatomy-structured design, soft attention-bias priors, hierarchical MIL formulation, empirical gains, and reproducibility. In line with the rebuttal guideline, we clarify the submitted method/results using analyses from the same trained models, datasets, and evaluation protocol. We will incorporate these clarifications into the revision.

R1W1 (Novelty). ASH-MIL is not region-wise feature fusion. It introduces two distinct components: (i) hierarchical MIL (QLA+BLA), which avoids box-level bipartite matching by routing gradients to informative queries/branches from image-level labels; and (ii) soft additive cross-attention biases from anatomical priors, enabling image-specific grounding and interpretable branch-level evidence. Ablations confirm their roles: removing priors drops CXR8 AP@0.5 from 6.03% to 2.05%; replacing MIL with mean pooling drops it to 0.76%.

R1W2 (Generalization). Although CXR anatomy is relatively standardized, 2D projections contain substantial anatomical overlap, making precise localization non-trivial. ASH-MIL uses soft biases, not hard masks, and includes an agnostic branch for cross-region findings. Thus, while validation is CXR-focused, the formulation is not tied to fixed CXR coordinates and can extend to settings with reliable anatomical masks.

R1W3 (Dataset size & prior inconsistency). Limited box annotations are common in CXR WSOD: ADPD evaluates localization on 882 CXR8 images, and AGXNet on a MIMIC-CXR subset of 390 boxes. Our work evaluates on both datasets, providing cross-domain validation. The apparent prior inconsistency arises because “1 branch w/ prior” uses a merged cardiac+pulmonary union mask in one observer, diluting context-specific cues. To clarify the submitted ablation, we provide the focused-prior breakdown from the same single-branch setting: CXR8 AP@0.5 cardiac 3.43 (+53.8%), pulmonary 2.96 (+32.7%) vs. agnostic 2.23; MIMIC cardiac 7.22 (+68.3%), pulmonary 5.81 (+35.4%) vs. agnostic 4.29.Thus, focused anatomical priors consistently help; the issue is the merged-prior design, not the prior mechanism.

R2W1 (CXAS supervision). CXAS provides disease-agnostic anatomical inductive biases; pathology localization is learned solely from image-level disease labels, without disease boxes. We will clearly distinguish disease supervision from anatomy-prior construction.

R2W2 (Classification). Since ASH-MIL already outputs image-level scores, we summarize AUROCs from the submitted model on NIH/CXR8: Atelectasis 0.865, Cardiomegaly 0.950, Effusion 0.899, Infiltration 0.579, Mass 0.893, Nodule 0.816, Pneumonia 0.734, Pneumothorax 0.827, mean 0.820.The low Infiltration score reflects known label ambiguity/diffuseness. AGXNet reports 0.84 using MIMIC pretraining with RadGraph-derived supervision; ASH-MIL is comparable using NIH image-level labels alone, while outperforming AGXNet in localization across all IoUs on both benchmarks.

R2W3 (Clinical deployability). We agree strict localization remains difficult: absolute CorLoc@0.5 is low across all methods. Still, ASH-MIL achieves CXR8 CorLoc@0.5 of 0.26, surpassing all WSOD baselines (next best 0.19) and comparable to fully supervised DETR (0.25). For Nodule, a known WSOL failure case, ASH-MIL achieves CorLoc@0.5 of 0.46 and AP@0.5 of 9.27%. We will temper deployment claims and frame ASH-MIL as an annotation-efficient step requiring further external validation/calibration.

R3W1 (2 priors). Pulmonary and cardiac priors are clinically motivated coarse compartments aligned with radiologist reading workflow. Using all 157 CXAS categories would over-fragment evidence under weak supervision and introduce data-sparsity/optimization issues. Priors are soft biases rather than hard masks, and the agnostic branch handles cross-region findings, with branch-level attention dynamically weighting contributions per class.




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The paper proposes a soft-weighted anatomy-aware approach for chest X-ray disease detection. The reviewers have some concerns raised. I urge the authors to respond to those in a dedicated rebuttal.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The rebuttal has convincingly answered the reviewer’s concerns. Leaning towards accept.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The authors have clarified the points raised by R#1, and both R#2 and R#3 support acceptance. The authors’ rebuttal regarding generalization on other medical images and relatively small datasets are acknowledged, and the authors are encouraged to also mention it in the revision.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    This paper received Weak Reject (no post-rebuttal submitted), Accept, and Accept.

    The authors’ rebuttal clarified that ASH-MIL uses CXAS-derived anatomical priors (disease-agnostic, no disease boxes), articulated the novelty of hierarchical MIL (QLA+BLA routes gradients from image-level labels), defended the 2-prior design (cardiac and pulmonary as clinically motivated coarse compartments with an agnostic branch for cross-region findings), and tempered the clinical-deployment claim by acknowledging that absolute CorLoc@0.5 remains low across all WSOD methods.

    The authors should make the bounding-box-vs-anatomical-prior distinction explicit in the final manuscript, frame the work as an annotation-efficient WSOD advance rather than a clinically-ready detection system, and clearly state the localization-precision limitations.



back to top