List of Papers Browse by Subject Areas Author List
Abstract
Pathology reports are structured, multi-granular documents encoding diagnostic conclusions, histological grades, and ancillary test results across one or more anatomical sites; yet existing pathology vision-language models (VLMs) reduce this output to a flat label or free-form text. We present HiPath, a lightweight VLM framework built on frozen UNI2 and Qwen3 backbones that treats structured report prediction as its primary training objective. Three trainable modules totalling 15M parameters address complementary aspects of the problem: a Hierarchical Patch Aggregator (HiPA) for multi-image visual encoding, Hierarchical Contrastive Learning (HiCL) for cross-modal alignment via optimal transport, and Slot-based Masked Diagnosis Prediction (Slot-MDP) for structured diagnosis generation. Trained on 749K real-world Chinese pathology cases from three hospitals, HiPath achieves 68.9% strict and 74.7% clinically acceptable accuracy with a 97.3% safety rate, outperforming all baselines under the same frozen backbone. Cross-hospital evaluation confirms generalisation with only a 3.4pp drop in strict accuracy while maintaining 97.1% safety. Code and evaluation protocol: https://github.com/raytions/hipath.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/4029_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{YuaRui_HiPath_MICCAI2026,
author = { Yuan, Ruicheng AND Zhang, Zhenxuan AND Wang, Anbang AND Hu, Liwei AND Hua, Xiangqian AND Peng, Yaya AND Luo, Jiawei AND Yang, Guang},
title = { { HiPath: Hierarchical Vision-Language Alignment for Structured Pathology Report Prediction } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16878},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This study formulates a structured slot-level prediction task by parsing clinical reports into segments and grouping diagnostic terms into pre-defined slots. To support this formulation, the authors propose several modules for aligning visual and textual representations, as well as for structured prediction, including a Hierarchical Patch Aggregator, Hierarchical Contrastive Learning, and Slot-based Masked Diagnosis Prediction. Overall, the proposed framework is reasonably designed, and the experimental results demonstrate its effectiveness to some extent.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The formulation of predicting pre-defined slots, rather than generating free-form text from VLMs, is a practical design choice that reduces task difficulty while preserving clinical applicability. 2.The proposed modules for embedding alignment and structured prediction are well aligned with the task formulation and appear to be reasonably designed. The ablation studies further support the effectiveness of these components.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Some technical details appear to be inaccurate. For example, UNI2 is based on a ViT-Giant (patch14, 224) architecture with an embedding dimension of 1536, as specified in its official configuration (https://huggingface. co/MahmoodLab/UNI2-h/blob/main/config. json), which differs from the description in the paper. Such inconsistencies may affect the clarity and reproducibility of the method. It is also unknown what is Deep MLP refering to in Table 1.
2.The paper mentions two limitations to justify the exclusion of certain comparison methods; however, the provided explanations are not entirely convincing. In particular, regarding the parameter budget constraint, parameter-efficient fine-tuning techniques (e. g. , LoRA) have been widely adopted and can significantly reduce the number of trainable parameters, making such comparisons more feasible.
3.The presentation could be further improved for clarity. In particular, including a more comprehensive discussion of related work, as well as illustrative examples of the training and inference processes, would make the method easier to understand.
4.The choice of baselines appears to be relatively limited. A significant component of the proposed method focuses on aligning visual and textual embeddings; however, Table 1 mainly presents ablations of different components within the proposed framework, rather than comparisons with alternative methods that could address a similar task. For example, it would be informative to include baselines such as directly fine-tuning strong vision-language or multimodal models (e. g. , CONCH or Qwen-VL) on the same dataset. As a result, while the current experiments demonstrate the effectiveness of the proposed components, they do not provide sufficient evidence regarding the competitiveness of the method with respect to state-of-the-art approaches.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The formulation of the task and the corresponding methodological design are reasonable and well aligned, and the experimental results provide evidence that the proposed approach is effective. However, the current evaluation is relatively limited and does not sufficiently reflect the competitiveness of the method with respect to state-of-the-art approaches. In particular, the lack of strong and diverse baselines makes it difficult to fully assess the practical advantage of the proposed method.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
Authors propose HiPath, a lightweight VLM framework designed for structured pathology report prediction. HiPath essentially is a hierarchical framework that treats the generation of structured diagnostic “slots” (diagnosis, grade, IHC) as its primary learning objective, rather than flat classification. Compared to standard pathology VLMs, it adds only 15M trainable parameters to frozen UNI2 and Qwen3 backbones, utilizing hierarchical patch aggregation (HiPA) and Optimal Transport-based alignment (HiCL). Authors validate the method on 749K Chinese pathology cases from three hospitals, outperforming vision-only and non-hierarchical baselines while maintaining a 97% clinical safety rate.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Scale and Rigor: The main strength is the massive scale of the validation (nearly 750K cases) and the inclusion of multi-center, cross-hospital testing, which provides high confidence in the framework’s reliability within its target domain.
- Clinical Alignment: The transition from unconstrained text generation to structured slot prediction directly addresses the clinical reality of pathology reporting, ensuring the model output is reliably parseable and actionable.
- Well-Written Presentation: The paper is exceptionally clear and well-organized. The methodology is explained with technical precision, and the clinical motivation is compellingly argued.
- Safety-Centric Metrics: The introduction of a safety-aware evaluation protocol that penalizes critical errors (e.g., benign-malignant crossings) demonstrates a sophisticated understanding of medical priorities.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- Limited Methodological Novelty: While the framework is effective, the core technical components (Sinkhorn OT for alignment, cross-attention for aggregation, and masked token prediction) are relatively standard in the VLM literature. The contribution is more of a clever integration of existing techniques onto frozen foundation models rather than a fundamental architectural breakthrough.
- Lack of Public Dataset Validation: The entire evaluation is conducted on a private multi-center cohort from China. While the 749K-case dataset is impressive in scale, it is difficult for the community to benchmark HiPath against other SOTA models without results on standard public pathology datasets (e.g., TCGA, Camelyon, or Quilt-1M). This also raises questions about whether the “structured slots” are overly tuned to specific Chinese reporting conventions.
- Inference Structural Priors: The model relies on a report template (specifying segments and slot types) at inference time. The current experiments utilize templates derived from ground-truth reports, which may provide a structural advantage that might not be as consistent in diverse clinical environments where clinical metadata is noisy or incomplete.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
HiPath presents a highly polished and clinically relevant framework for structured pathology diagnosis. The scale of the private multi-center validation is remarkable and the writing is of high quality. However, the technical novelty is somewhat incremental, relying largely on the integration of established components (OT, Cross-Attention) with frozen foundation backbones. Furthermore, the absence of validation on public benchmarks makes it difficult to assess how well this specific “slot-based” formulation generalizes beyond the authors’ private Chinese cohort. A “Weak Accept” reflects the balance between its strong clinical utility/scale and its limited methodological innovation and benchmarking.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This paper proposes HiPath, a lightweight vision-language model (VLM) framework dedicated to generating structured pathology reports. It introduces the HiPA module to aggregate features from multiple pathological images, presents the HiCL module to achieve image-text cross-modal alignment, and develops a slot-based structured diagnosis prediction module for generating structured diagnostic results. HiPath is trained on 749K Chinese pathological cases from three hospitals and outperforms all baselines under the setting of the same frozen backbone network.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The module combination strategy of the lightweight structured report generation framework in this paper is relatively novel. The dataset employed in the experiments is large in scale and of high quality.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The two VLMs cited in this paper are works from 2023 and 2024, which provide no support for the claims of this paper. Why have relevant pathology VLM works from 2025 not been referenced? 2.If pathology reports are inherently structured, why does the VLM trained on such structured pathology reports fail to output structured reports? This can be easily achieved using VLMs in 2025.3.What would be the performance if we directly extract 304 terms from the free text generated by the VLMs cited in the paper and then fill them into the template? 4.First, since the LLM component is frozen and only used to extract text embeddings, why was a MoE model adopted instead of the Qwen3-Embedding model? Second, most LLMs follow a decoder-only architecture, and non-embedding LLMs such as Qwen3-30B-A3B are inherently unsuitable for text feature extraction. 5.No explicit mathematical definition or formula for L_MDP is provided in the paper. 6.The notation of variables in the paper is unclear, which significantly hinders the understanding of the formulas. 7.A lightweight framework should typically be considered only under constrained computational resources. Given the availability of 4×H100 GPUs, why not directly fine-tune a VLM instead? 8.Why does the paper not adopt existing NLP metrics? The proposed four-level clinical evaluation scheme lacks correlation statistical analysis with physician ratings or established metrics, and has no supporting references. As the main evaluation metric of this paper, it fails to provide sufficient convincing evidence to demonstrate the model’s performance. 9.In Table 1, HiPath Cross is evaluated on a different test set and thus should not be included in the same table for comparison. The term “encoder” is used ambiguously, referring alternately to the text encoder and the image encoder, leading to confusion. In addition, there are terminological errors. 10.In Table 1, how does the text-only Qwen3 without image input perform prediction? 11.In the current AI era, translation is no longer a difficult task. Why not translate the Chinese text into English and train the model using bilingual reports in both Chinese and English? This would also facilitate performance comparisons with other models. 12.Are there any public datasets available for comparison? 13.The overall presentation logic of the experiments is confusing, and the experimental design appears to be ablation studies rather than performance comparisons. 14.The number of comparative methods is insufficient. Why did authors not compare it with pathology-specific VLMs from the past year, instead only comparing with two foundation models trained on English corpora? 15.Is it fair to classify Chinese text using English-based pre-trained foundation models? Why not translate the 304 terms?
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(2) Reject — should be rejected, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The motivation of this paper is superficial and unconvincing. This paper gives readers the impression that it is merely chasing the popularity of LLMs without exploring the theoretical knowledge of relevant LLMs, and resembles a contrastive learning work wrapped in a LVLM framework. The overall presentation logic of the experiments is confusing, and the experimental design appears to be ablation studies rather than performance comparisons.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
The quality of the authors’ rebuttal is poor. They failed to properly understand my concerns and did not directly address the key questions I raised. Instead, they repeatedly reiterated the so-called “superiority” claims that had already been stated in the paper.
Author Feedback
We thank the reviewers for noting the technical innovations and clinical relevance of this work: R1 found the slot formulation practical and the modules aligned with the task. R2 highlighted the 749K-case validation and safety-aware evaluation. R3 noted the novel design and high-quality dataset. We address the remaining concerns.
Q1: Novelty beyond known components? (Meta; R2 Q1; R3 Q2) Our novelty is the formulation itself: a safety-auditable structured objective built on frozen backbones with only 15M trainable parameters. Prior pathology VLMs and report generators target contrastive matching, open-ended QA, or free-text generation; prior hierarchical methods structure visual features, not typed diagnostic targets. HiPath instead trains directly on
[DIAG],[GRADE], and[RES]slots over a curated 304-term vocabulary. Compared with free-text or structured-QA generation, it avoids open-vocabulary answers that require post-hoc extraction, making strict accuracy and safety auditable per slot.Q2: What does each module contribute? (Meta; R1 Q4; R2 Q1) Each module removes a different failure mode. Without HiCL, i2t R@1 collapses from 21.2 to 0 and acceptable accuracy drops by 10.4 pp; alignment is not cosmetic. Without HiPA, 4+image Top-1 drops by 3.6 pp, vs 0.8 pp for single-image cases. Without Slot-MDP, acceptable accuracy drops by 10.1 pp; slot prediction is not replaceable by contrastive alignment alone.
Q3: Are baselines sufficient and fair? (Meta; R1 Q2,Q4; R3 Q3,Q14) Table 1 is a controlled comparison ladder, not merely internal ablation. With the same data, split, frozen UNI2 features, and similar trainable budget, performance progresses from UNI2 linear probe 30.64, ABMIL 32.37, Deep MLP 32.49, Flat CrossAttn 55.98, to HiPath 68.89.Thus the gain is not explained by capacity: HiPath gains 36.4 pp over the 15.9M vision MLP and 12.9 pp over the 15.7M non-hierarchical text-supervised alternative. PLIP/CONCH (pathology VLMs, ~1M image-text pairs each) on the same 304 target reach at most 14.63% Strict Top-1, which the paper attributes to domain shift and label-taxonomy mismatch. LoRA Qwen VL/CONCH would be useful but changes encoder and capacity together.
Q4: Generalization beyond private Chinese cohort? (Meta; R2 Q2; R3 Q12) The paper tests cross-hospital transfer. HiPath Cross trains on Hospitals A+C and tests on Hospital B, with no patient, scanner, or staining overlap, and drops only 3.4 pp strict (68.89 to 65.47) with safety 97.06.Existing English pathology resources differ in kind from our data: they typically provide image labels, captions, or report text, not case images paired with segmented reports, typed slots, IHC/grade fields, synonyms, and safety boundaries. That missing structure is exactly what HiPath evaluates.
Q5: Technical corrections (R1 Q1; R3 Q4,Q5,Q10) R1 is right that the UNI2 architecture line has a typo. UNI2 is ViT-Giant/14 with native 1536-d output; our pipeline projects it to 1024-d. Deep MLP is mean-pooled UNI2 features + a 4-block 2048-d LN/GELU/Dropout MLP + a 304-way focal CE head. Slot-MDP uses the focal loss of [11], explicitly L_MDP=avg_k(1-p(m_k|k))^γ * CE_ε(p(·|k),m_k) over the Eq. 6 logits. Qwen3-30B-A3B is a frozen mean-pooled sentence encoder (MoE 3B/30B); the text-only row uses its embeddings without images as a language-prior check.
Q6: Template leakage and metric? (R2 Q3; R3 Q8) The template specifies only slot positions and slot types. It does not reveal slot values. A breast case may specify
[DIAG],[GRADE], and[RES]slots, but not whether diagnosis is benign or malignant or markers are positive. Slot queries attend only to visual features. The four-level metric is used because pathology errors have unequal clinical risk; benign-to-malignant crossings, non-adjacent grade jumps, and IHC polarity reversals should not be treated like synonym differences. The safety boundaries were defined by two senior pathologists.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes HiPath, a lightweight vision-language framework for structured pathology report generation, reformulating report prediction into slot-level structured outputs and combining hierarchical visual aggregation, cross-modal alignment, and masked structured prediction. The paper is supported by a very large-scale dataset (749K cases from three hospitals) and shows consistent improvements over frozen-backbone baselines, along with promising cross-hospital generalization and safety-aware evaluation.
Across reviewers, there is broad agreement on the clinical relevance and practical importance of the problem. Reviewer #1, who appears most familiar with the technical and clinical setting, highlights that the slot-based formulation is a meaningful and practical design choice, and that the architectural components are reasonably well aligned with the task, with ablations providing partial support. Reviewer #2 also emphasizes the strong clinical alignment, scale of validation, and safety-oriented evaluation as key strengths. Reviewer #3 similarly acknowledges the structured multi-level design and reported performance gains.
At the same time, the main concern across all reviews is limited methodological novelty, with the method largely seen as an integration of established components (hierarchical patch aggregation, optimal transport-based alignment, and masked prediction) rather than a fundamentally new architecture. However, importantly, this concern is more about degree of innovation than correctness or validity, and none of the reviewers identify a clear technical flaw that invalidates the approach.
The evaluation strength is viewed positively but not fully conclusive. While the dataset scale and multi-hospital setting are strong advantages, Reviewer #2 raises the key limitation that the evaluation is entirely on a private Chinese cohort, making it difficult to assess generalization beyond the reporting conventions of the dataset. Reviewer #1 also notes that stronger baselines and clearer justification of modeling choices would improve confidence in the claimed gains. Reviewer #3 points to similar concerns about benchmarking depth and methodological justification, but these are framed more as missing analyses than fatal issues.
A secondary but important point is that baseline comparisons and benchmarking breadth could be improved, particularly with respect to stronger or more recent VLM or multimodal baselines. However, again, this is a completeness concern rather than evidence of incorrect results.
Overall, while the paper does not present a clear algorithmic breakthrough, it does offer a well-motivated, clinically grounded, and scalable structured reporting framework, supported by large-scale real-world data and consistent empirical gains. The limitations primarily concern incremental novelty and benchmarking scope, which are addressable through clarification and additional experiments rather than fundamental revision of the method.
Given this balance, the paper is suitable for rebuttal, to allow the authors to clarify:
the exact novelty relative to prior slot-based and hierarchical VLM/reporting methods, baseline selection and fairness of comparison, generalization beyond the private cohort, and the role/necessity of each module in the full pipeline. Rebuttal Guidance for Authors:
To strengthen the submission, the authors are encouraged to address:
Clarify the precise methodological novelty beyond integration of known components (HiPA, OT-based alignment, masked prediction). Provide clearer justification for baseline selection and discuss whether stronger VLM/reporting baselines were considered. Expand discussion of generalization beyond the private multi-center Chinese dataset and its implications. Better articulate the contribution of each module (HiPA, HiCL, Slot-MDP) through more targeted ablations or analysis. Clarify how structured slot design compares to alternative structured report generation approaches in recent literature.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
This paper addresses an important and clinically relevant problem: structured pathology report generation using lightweight vision-language modeling. The work is supported by an unusually large real-world dataset spanning multiple hospitals and demonstrates consistent gains together with safety-aware evaluation, which is particularly valuable for clinical deployment scenarios.
The reviewers generally agreed that the structured slot-based formulation is practical and clinically meaningful. While the individual technical components are largely built upon existing ideas, the integration into a scalable, safety-auditable pathology reporting framework represents a meaningful contribution with demonstrated practical utility. The rebuttal further clarified the role of each module, the rationale behind the structured prediction design, and several technical implementation details.
The primary weaknesses concern limited methodological novelty, incomplete benchmarking against stronger recent pathology VLMs, and limited validation outside the private cohort. These are important limitations, but they do not invalidate the core contribution or experimental findings. Overall, the paper presents a technically sound and clinically grounded system with strong empirical validation at scale, and the strengths outweigh the weaknesses. Therefore, I recommend acceptance.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation.
After reviewing the rebuttal, the reviewers’ concerns remain unresolved. The reviewers found the rebuttal weak, noting that it failed to engage with their core questions and instead largely restated the superiority claims already made in the paper. There is also concern about the authors’ ability to grasp the issues raised. Having read the paper, the rebuttal, and the reviews myself, I agree that the rebuttal is unconvincing and does not adequately address the key concerns. I therefore recommend rejection.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation.
The rebuttal doesn’t fully address the concerns of reviewers.