List of Papers Browse by Subject Areas Author List
Abstract
Hepatocellular Carcinoma diagnosis relies heavily on the interpretation of gigapixel Whole Slide Images. However, current computational approaches are constrained by fixed-resolution processing mechanisms and inefficient feature aggregation, which inevitably lead to either severe information loss or high feature redundancy. To address these challenges, we propose Hepato-LLaVA, a specialized Multi-modal Large Language Model designed for fine-grained hepatocellular pathology analysis. We introduce a novel Sparse Topo-Pack Attention mechanism that explicitly models 2D tissue topology. This mechanism effectively aggregates local diagnostic evidence into semantic summary tokens while preserving global context. Furthermore, to overcome the lack of multi-scale data, we present HepatoPathoVQA, a clinically grounded dataset comprising 33K hierarchically structured question-answer pairs validated by expert pathologists.
Our experiments demonstrate that Hepato-LLaVA achieves state-of-the-art performance on HCC diagnosis and captioning tasks, significantly outperforming existing methods. Our code and implementation details are available at Hepato-LLaVA-23DC.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0412_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/PRIS-CV/Hepto-LLaVA
Link to the Dataset(s)
N/A
BibTex
@InProceedings{YanYux_HepatoLLaVA_MICCAI2026,
author = { Yang, Yuxuan AND Yan, Zhonghao AND Zhang, Yi AND Yun, Bo AND Diao, Muxi AND Zhao, Guowei AND Liang, Kongming AND Li, Wenbin AND Ma, Zhanyu},
title = { { Hepato-LLaVA: An Expert MLLM with Sparse Topo-Pack Attention for Hepatocellular Pathology Analysis on Whole Slide Images } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16878},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper presents Hepato-LLaVA, a multimodal large language model tailored for hepatocellular carcinoma (HCC) analysis on whole slide images. The work has three primary contributions. First, the authors introduce HepatoPathoVQA, a multi-scale VQA dataset comprising 33K QA pairs spanning WSI, ROI, and patch levels, constructed via a hierarchical reasoning pipeline using Gemini and validated by expert pathologists. Second, they propose a Sparse Topo-Pack Attention mechanism that organizes patch tokens into local packs with summary tokens, enforcing structured sparse interactions that reflect 2D tissue topology rather than treating the slide as a flat 1D sequence. Third, a three-stage training pipeline (MAE pretraining, MoCo pretraining, and LoRA-based instruction tuning with a Q-Former connector) is designed to progressively align visual and language representations for diagnostic tasks.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The clinical motivation is well-grounded. The paper identifies a real limitation in existing WSI-MLLMs — the mismatch between 1D sequence modeling and the inherently 2D spatial structure of pathological tissue — and proposes a concrete architectural response. The hierarchical diagnostic reasoning embedded in the dataset pipeline (coarse-to-fine, macroscopic to microscopic) aligns with actual pathologist workflows.
The Sparse Topo-Pack Attention is a reasonable design. The local-pack / summary-token structure is intuitive, and the paper makes a plausible case that it reduces redundancy compared with dense slide modeling.
HepatoPathoVQA is a meaningful dataset contribution. Covering three spatial scales with both VQA pairs and caption data for pre-training, and validated by three pathologists under blind evaluation (Pearson r=0.96, <2% rejection rate), it addresses a genuine data gap in HCC-specific computational pathology.
The quantitative results are strong and the ablation studies are reasonably comprehensive. The comparison across connector types, token counts, and training strategies in Tables 2 and 3 provides clear support for the main design choices. The gap between Hepato-LLaVA (Avg 0.83) and the next best method (0.66) is substantial.
Code is made available via an anonymized link, which supports reproducibility.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The main weakness is that the evaluation is largely self-contained. The training data and benchmark are both derived from the same Gemini-assisted pipeline on the same pool of 200 WSIs, so it is hard to judge how much of the gain would transfer to an external cohort with independently curated labels.
I also think the paper could do a better job isolating where the gains actually come from. The method combines a new sparse encoder, a new multi-scale dataset, and a multi-stage training recipe, but the ablations do not fully separate these factors. In particular, the contribution of the MoCo stage is not shown directly.
Another missing piece is efficiency evidence. The paper motivates the method partly in terms of redundancy and computational cost, but does not report runtime, memory, or inference cost.
Finally, the open-ended evaluation relies partly on LLM-based scoring, and I would have liked to see either stronger human evaluation or at least more detail on how reliable that metric is in this setting.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper addresses a legitimate problem in WSI-based pathology MLLMs and proposes technically reasonable solutions backed by solid experimental results. The dataset contribution alone has value for the community. That said, the absence of any external validation is the main factor holding back a stronger recommendation — the benchmark used for evaluation was constructed by the same pipeline used to generate training data, which makes the performance numbers difficult to fully trust at face value. The missing MoCo ablation and lack of runtime analysis are secondary concerns. If the authors can provide even preliminary results on an external HCC cohort and clarify the MoCo contribution in the rebuttal, I would be willing to revise upward.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors have adequately addressed my main concerns. The additional MoCo ablation (w/o MoCo: 0.75 vs full model: 0.83) confirms that each training stage contributes meaningfully. The cross-dataset evaluation on WSI-Bench, while showing a modest gap from SOTA (0.502 vs 0.566), provides at least preliminary evidence of generalizability beyond the self-constructed benchmark. The reported inference time (4.3 mins/slide) also substantiates the efficiency claims. I am raising my score to 5.
Review #2
- Please describe the contribution of the paper
This paper proposes Hepato-LLaVA, a Multi-modal Large Language Model (MLLM) tailored for Hepatocellular Carcinoma (HCC). To effectively and efficiently capture and summarize information from gigapixel whole slide images (WSIs), a Sparse Topo-Pack Attention mechanism is introduced to preserve the 2D tissue topology of WSIs. In addition, the authors construct HepatoPathoVQA, a multi-scale WSI dataset for HCC diagnosis via visual question answering (VQA). Experimental results demonstrate that the proposed method achieves around a 20% improvement in HCC diagnosis, outperforming existing approaches.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper proposes a WSI topology-aware attention mechanism. Most recent WSI models treat patch tokens as a flattened 1D sequence during processing, which overlooks the inherent 2D structure of WSIs and breaks the underlying tissue topology. In addition, tissues or lesions often span multiple neighboring patches while exhibiting weaker correlations with distant patches; thus, tokens from neighboring patches should be more important than distant tokens in the local view at the lowest scale. The proposed Sparse Topo-Pack Attention organizes patch tokens into several 2D hierarchical views and processes them hierarchically, effectively preserving the 2D tissue topology while efficiently aggregating global WSI information across patches. 2.The paper constructs a multi-scale WSI VQA dataset with three different scales of HCC samples. The authors claim that this is the first multi-scale WSI dataset for HCC, which helps bridge the gap between research datasets and real-world clinical practice, and provides a valuable resource for future studies.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The ablation study is incomplete. The paper conducts experiments on different design choices of the connector; however, the connector is only a part of the proposed MLLM. A more comprehensive ablation, with different proposed modules enabled or disabled, is suggested to better evaluate the effectiveness and individual contributions of the proposed components. 2.The proposed method is compared with six state-of-the-art methods on the constructed HCC dataset. However, although the models are trained on an HCC dataset, the proposed method may have the potential to generalize to other diseases. It would strengthen the paper, especially for a journal extension, to include comparisons on additional HCC datasets or datasets from other diseases. 3.A minor issue: the full name of VQA should be defined before introducing the abbreviation.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
In addition to developing the HCC-specialized MLLM, the paper introduces a novel topology-aware attention mechanism that effectively preserves WSI topology while efficiently extracting global WSI information. The paper also constructs a multi-scale WSI VQA dataset for HCC diagnosis, providing a valuable resource for future research.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The reviewer considers the paper to have sufficient contributions, and the authors adequately addressed the reviewer’s concerns in the rebuttal.
Review #3
- Please describe the contribution of the paper
For hepatocellular carcinoma analysis on whole-slide pathology images, the authors propose Hepato-LLaVA, a domain-specific multimodal model with a Sparse Topo-Pack Attention design that explicitly uses tissue topology to compress gigapixel slides more efficiently while retaining diagnostically important local details. They also introduce HepatoPathoVQA, a clinically grounded multi-scale dataset that supports coarse-to-fine reasoning across whole-slide, ROI, and patch views, enabling the model to learn more clinically useful visual-language understanding for diagnosis.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The motivation is well grounded, and the proposed Hepato-LLaVA surpasses previous pathology-specific multimodal large models on hepatocellular carcinoma analysis. 2.The authors also construct HepatoPathoVQA, a large-scale multi-scale dataset that provides a strong foundation for subsequent hepatocellular pathology analysis on whole-slide images. 3.In addition, they introduce HepatoPathoBench, which facilitates a more systematic evaluation of existing multimodal large models in hepatocellular pathology analysis.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The proposed Sparse Topo-Pack Attention (STPA) appears to be applied in the slide encoder, whereas the ablation results in Table 3 focus on the connector. This discrepancy makes it hard to determine whether STPA itself improves token efficiency. 2.The paper also introduces additional MAE pretraining and MoCo pretraining stages, both of which may bring independent performance gains. Since existing models such as SlideChat and WSI-LLaVA do not adopt these training stages, it remains unclear whether the performance improvement truly comes from STPA.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The main reason is that the current manuscript does not provide clear evidence that the proposed Sparse Topo-Pack Attention strategy is truly effective for hepatocellular pathology analysis on whole-slide images.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank all the reviewers and the Area Chair for their insightful comments and recognition of our work. Below, we address the main concerns regarding Ablation, Data, and Inference.
Q1.Ablation (Component Disentanglement & STPA Effectiveness) Dear reviewer #2, #3, #4, Our proposed modules act as an integrated whole to solve redundancy and information deficiency issues in existing slide encoders when processing WSIs. Both the MoCo training and the Q-Former connector depend on the summary tokens produced by the STPA slide encoder. Due to this tight coupling and strict page limits, our paper only included the connector ablation. We now provide additional results on HepatoPathoBench. First, Hepto-LLaVA w/o MoCo scored 0.75, validating the effectiveness of our two-stage pre-training strategy. Although lower than our full Hepto-LLaVA (0.83), it still higher than SlideChat (0.66) and WSI-LLaVA (0.65), which is also consistent with the findings of Prov-GigaPath[1]. Second, fine-tuned WSI-LLaVA scored 0.70, which uses LongNet as the slide encoder and trains on the same dataset. This is higher than WSI-LLaVA (0.65) but lower than Hepto-LLaVA (0.83), directly validating the effectiveness of our designed STPA architecture. Together with the additional ablation results, these findings further demonstrate that the performance gains mainly originate from our designed STPA architecture, and that each proposed component consistently contributes to the overall improvement.
Q2.Data (Quality, Evaluation Fairness, and Generalization) Dear reviewer #2, #3, We strictly separate the training and validation sets at the patient level with no WSI overlap. All samples are generated through the same unified pipeline rather than curated as dataset-specific groups. To guarantee high quality and avoid homogenization, the pipeline integrates 3 different resolutions and 8 sub-categories, and all annotations were subjected to strict quality control. Three professional pathologists conducted blind evaluations of the generated data, achieving high consistency (Pearson r=0.96) and a rejection rate of <2\%. Furthermore, the LLM-based open-ended evaluation results strongly align with our human expert judgments. We also note that the currently available HCC subset in WSI-Bench contains only 27 WSIs and 103 QA pairs, while our HepatoPathoBench further introduces multi-resolution evaluation, making the benchmark more comprehensive and challenging. To further validate cross-dataset generalization, we directly evaluated our model on the HCC subset of WSI-Bench without additional adaptation. Our Hepto-LLaVA scored 0.502, remaining competitive with the current SOTA result of 0.566. Q3.Efficiency and Computational Complexity Dear reviewer #2, #4 By restricting patch-level interactions within local packs and utilizing summary tokens for inter-pack global context, STPA reduces the theoretical attention overhead to roughly 1%. Coupled with a lightweight backbone and blank-patch filtering, our end-to-end feature extraction (from SVS to PT tensors) averages 4.3 mins/slide, a significant acceleration compared to the 19.3 mins/slide required by Prov-GigaPath [1]. Furthermore, our slide encoder and connector condense the final visual input for the LLM to just 32 tokens, whereas Prov-GigaPath requires 576 tokens, yielding substantial efficiency advantages during downstream LLM processing.
Q4.The Minor Issue Dear reviewer #3 We will add the full definition of Visual Question Answering (VQA) before its first occurrence in the revised manuscript.
[1] Xu et al., “A whole-slide foundation model for digital pathology from real-world data”, Nature 2024.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper introduces a multi-modal large language model for fine-grained analysis of hepatocellular pathology which utilizes a “sparse Topo-pack” attention mechanism to integrate local and global information. Additionally, they introduce a multi-scale visual-question-answering dataset, HepatoPathoVQA.
Reviewers agree that the dataset with its multi-scale setup addresses an important task and gap in digital pathology. They further appreciate the overall design of the attention mechanism and the model from coarse to fine.
They criticize, however, that the evaluation is largely self-contained (see comments below) and that the ablation studies do not sufficiently disentangle all components. Additionally, no information about runtime or computational complexity is provided.
In the rebuttal, the authors should address how reliable/independent a benchmark is on a dataset that shares substantial parts of the pipeline with the pipeline for generating training data, and why performance improvements can be expected to translate to independent tasks or datasets. Furthermore, please address the comment on how well the ablation studies demonstrate the utility of the proposed method and to what extent changing (pre-)training steps may bias the results.
Please note there should be no new results added in the rebuttal phase.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Reviewers agree that the authors adequately addressed concerns in their rebuttal and univocally vote for acceptance. The overall assessment however may be seen as borderline as the authors added new ablation studies in their rebuttal.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The paper presents a meaningful methodological contribution, supported by a valuable dataset resource and strong experimental results. Overall, the paper is considered to make a solid contribution and meets the acceptance threshold for MICCAI.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The main concerns raised by reviewers are the lack of external validation, reliance on a self-constructed benchmark, and insufficient evidence supporting the effectiveness of the Sparse Topo-Pack Attention strategy for HCC pathology. The authors have addressed most of the concerns in the rebuttal.