Abstract
Integrating histopathology with genetic/omics tabular data is central to computational oncology, yet most multimodal models simply concatenate image and gene features and never learn the biological correspondence between them. We introduce JPathGene, a cross-modal joint-embedding predictive framework that formulates image–genetic fusion as latent prediction. A single predictor unifies within-modality masked prediction (I-JEPA) and cross-modal prediction; because one tissue phenotype is compatible with a distribution of genetic programs, a conditional latent-diffusion predictor models that distribution (deterministic JEPA being its single-step limit) and yields a per-patient cross-modal uncertainty. CoRE-Fusionuses that uncertainty to gate an image expert that corrects a genomic anchor, weighting histology by its cross-modal reliability, without decoding raw expression. On a leakage-safe TCGA-BRCA benchmark with UNI2-h features, JPathGene achieves the highest multimodal AUC: it ties a strong but over-confident gene-only baseline on AUC while being significantly better calibrated (p<0.001) and best on proper scoring rules. On TCGA-LUAD, CoRE significantly beats gene-only on both AUC and calibration on a molecular endpoint, while on a non-saturated anatomic stage endpoint fusion adds large discrimination (+0.07 to +0.11 AUC). The benefit is thus contingent on feature quality and on whether genomics already saturates the endpoint. All code and models will be publicly available in our official repository: https://github.com/JianJiaXian/JPathGene.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/MultiTab_018.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=p6dlyOKd5T
BibTex
@InProceedings{JiaJia_JPathGene_MICCAISAT2026,
author = { Jian, Jia-Xian AND Sun, Wei-Chieh AND Lin, Shih-Chih AND Lin, Fang-Yi AND Chu, YunTung AND Hwang, Jenq-Neng AND Chung, Pau-Choo},
title = { { JPathGene: Cross-Modal Latent Diffusion Predictive Learning between Histopathology Images and Genetic Profiles } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17263},
month = {pending},
page = {pending}
}