Abstract

Vision-language pretraining for radiology report generation relies on random negative sampling and is blind to clinical polarity, leaving encoders unable to capture the structure that makes a report useful: which region is affected, what the finding is, and whether it is present or absent. We introduce SHARP (Scene-graph Hard-negative Aware Radiology Pretraining), a two-stage framework. Stage 1 pretrains a ViT-B/16 encoder on concept-keyed crop–phrase pairs mined from the 222,180 training studies (of 227,239 processed) of the MIMICExt-CXR-QBA scene-graph corpus: a key of region, entity, and polarity makes same-key crops co-positives, while anatomy-hard and negationhard negatives, injected under a curriculum, force the encoder to discriminate findings it would otherwise conflate. Stage 2 transfers the encoder into single-image CXRMate and fine-tunes end-to-end. Evaluated on the MIMIC-CXR test set, SHARP achieves CheXbert F1 = 0.3032, outperforming ImageNet-pretrained encoders (+15.8%, p<0.001), with gains concentrated in the clinically demanding Impression section. Most of this gain comes from scene-graph concept supervision itself (symmetric InfoNCE on the same pairs reaches 0.2973); the concept-keyed negatives and curriculum add +0.0059 CheXbert F1, an increment we report as a point difference and do not claim to be statistically separated. From a domain-naive start, SHARP recovers 89% of the gain of large-scale domain pretraining (RAD-DINO) from one annotated dataset.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/TIA_034.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to Open Review

Open Review Page: Not Available

BibTex

@InProceedings{AlaAya_SceneGraph_MICCAISAT2026,
        author = { Alaswad, Aya AND Sonbol, Riad AND Bessadok, Alaa},
        title = { { Scene-Graph Contrastive Pretraining with Concept-Based Hard Negatives for Radiology Report Generation } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 17269},
        month = {pending},
        page = {pending}
}


back to top