Abstract
While histology and spatial transcriptomics (ST) offer complementary
views of tissue biology, integrating them for flexible tissue
exploration remains challenging. Existing approaches typically support
image-RNA search-by-example retrieval or rely on cascaded imageto-
gene prediction, limiting direct natural-language querying of measured
local tissue microenvironments. We introduce QUERY-ST, a lightweight
framework for jointly aligning H&E patches, ST-derived RNA
features, and structured natural-language descriptions in a shared contrastive
space. The textual descriptions are generated from quantitative
ST-derived summaries, including cell-type composition, cellularity,
and radius-based cell-neighbourhood interactions.We evaluate QUERYST
using patient-held-out validation on a public lung Xenium dataset
and spatial block cross-validation on an internal prostate Xenium cohort.
Text queries retrieved candidate microenvironments matching the
described tissue state, while refinement analyses showed that retrieval
changed appropriately when adjacency or cellularity constraints were added.
These results support QUERY-ST as a framework for using natural
language to search spatial pathology datasets for measured tissue states
and their corresponding histology.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/COMPAYL_061.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: https://papers.miccai.org/miccai-2026-sat/supp/COMPAYL_061_supp.pdf
Link to Open Review
BibTex
@InProceedings{KolRaf_QUERYST_MICCAISAT2026,
author = { Kollyfas, Rafael AND Dawood, Muhammad AND Woodcock, Dan J. AND Rittscher, Jens},
title = { { QUERY-ST: Language-Guided Retrieval Across Histology and Spatial Transcriptomics } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17251},
month = {pending},
page = {pending}
}