Abstract
Bladder cancer affects over 573,000 patients annually, and accurate pathological diagnosis from H&E-stained whole-slide images (WSIs) is critical for treatment planning. However, generating structured diagnostic reports from gigapixel WSIs remains labor-intensive and subject to inter-observer variability. Existing pathology report generation models either suffer from significant performance degradation on domain-specific tasks - general-purpose models such as WsiCaption and HistGen achieve ROUGE below 0.36 on bladder cancer benchmarks - or rely on discrete codebook-based connectors that sever gradient flow and discard classification confidence. We propose PathLLaVA, a codebook-free Pathology Vision-Language Model whose fully differentiable connector permits end-to-end optimization, addressing these limitations through three contributions: (i) a LLaVA-inspired continuous alignment MLP replacing the discrete codebook, enabling direct visual-to-linguistic gradient propagation and reducing discretization-induced information loss; (ii) QLoRA (4-bit NF4, r=16) for memory-efficient LLaMA 3.1-8B adaptation on a single GPU with 752 training slides; and (iii) GHA-MIL, a Group-Head Attention MIL motivated by head redundancy and IB perspectives. For the reported results we train only the alignment projector with the vision encoder and LLM kept frozen, which matches full joint fine-tuning at lower cost. PathLLaVA outperforms the LLaMA codebook baseline on all metrics (Score +0.023, ROUGE +0.006, BLEU +0.013, KEY +0.014, EMB +0.050) on the K-Medicon 2024 internal benchmark, while maintaining strong zero-shot generalization on an external cohort: EMB 0.944 on TCGA-BLCA (n=361), suggesting cross-cohort transferability of core diagnostic findings after template normalization.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/ELAMI_012.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: Not Available
BibTex
@InProceedings{HonSun_PathLLaVA_MICCAISAT2026,
author = { Hong, Sung man AND Ahn, Sangjeong},
title = { { PathLLaVA: Continuous Vision-Language Alignment for Bladder Cancer Pathology Report Generation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17262},
month = {pending},
page = {pending}
}