Abstract
Vision-language pretraining for radiology report generation relies on random negative sampling and is blind to clinical polarity, leaving encoders unable to capture the structure that makes a report useful: which region is affected, what the finding is, and whether it is present or absent. We introduce SHARP (Scene-graph Hard-negative Aware Radiology Pretraining), a two-stage framework. Stage 1 pretrains a ViT-B/16 encoder on concept-keyed crop–phrase pairs mined from the 222,180 training studies (of 227,239 processed) of the MIMICExt-CXR-QBA scene-graph corpus: a key of region, entity, and polarity makes same-key crops co-positives, while anatomy-hard and negationhard negatives, injected under a curriculum, force the encoder to discriminate findings it would otherwise conflate. Stage 2 transfers the encoder into single-image CXRMate and fine-tunes end-to-end. Evaluated on the MIMIC-CXR test set, SHARP achieves CheXbert F1 = 0.3032, outperforming ImageNet-pretrained encoders (+15.8%, p<0.001), with gains concentrated in the clinically demanding Impression section. Most of this gain comes from scene-graph concept supervision itself (symmetric InfoNCE on the same pairs reaches 0.2973); the concept-keyed negatives and curriculum add +0.0059 CheXbert F1, an increment we report as a point difference and do not claim to be statistically separated. From a domain-naive start, SHARP recovers 89% of the gain of large-scale domain pretraining (RAD-DINO) from one annotated dataset.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/TIA_034.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: Not Available
BibTex
@InProceedings{AlaAya_SceneGraph_MICCAISAT2026,
author = { Alaswad, Aya AND Sonbol, Riad AND Bessadok, Alaa},
title = { { Scene-Graph Contrastive Pretraining with Concept-Based Hard Negatives for Radiology Report Generation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17269},
month = {pending},
page = {pending}
}
