Abstract

Automatically generating pathology reports from whole slide images (WSIs) is a critical yet highly challenging task, aiming to transform gigapixel-level visual information into structured diagnostic text. WSIs inherently exhibit hierarchical semantic characteristics, spanning from local cellular morphology to global tissue architecture. However, existing methods predominantly rely on Euclidean space modeling and treat multi-scale features as flat representations, which limits their ability to effectively capture the intrinsic hierarchical structure of pathological data, thereby affecting the logical coherence of generated reports. To address this limitation, we propose H$^2$M-Net, a hyperbolic geometry-based hierarchical generation framework that explicitly models the hierarchical relationships between visual and textual modalities in the Lorentz space. Specifically, we embed both pathological images and report texts into a hyperbolic manifold, and introduce a radial position loss together with an entailment loss to enforce geometric and semantic constraints between parent and child nodes. On this basis, we construct a bottom-up cascaded generation architecture that consists of a visual encoder, memory encoder, and text decoder to enable cross-level information propagation. Experimental results demonstrate that our method outperforms current state-of-the-art approaches.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2164_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/YAOSL98/H2M-Net

Link to the Dataset(s)

TCGA Dataset: https://portal.gdc.cancer.gov/

BibTex

@InProceedings{YaoShu_H2MNet_MICCAI2026,
        author = { Yao, Shuilian AND Jia, Qi AND Li, Shirui AND Liu, Yu AND Fan, Xin},
        title = { { H2M-Net: Hierarchical Hyperbolic Memory Network for Pathological Report Generation } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16891},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    This paper proposes H2M-Net, a pathology report generation framework that embeds multi-scale image features and report text into hyperbolic space to better capture the hierarchical structure of whole slide images. The method combines radial position and entailment cone losses with a bottom-up cascaded architecture and memory encoder to model interactions across local, regional, and global scales.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    (1) The motivation for using hyperbolic geometry is clear and well justified. Modeling the local-to-global hierarchy of whole slide images in a non-Euclidean space is a sensible idea, and the paper explains this rationale effectively. (2) The proposed geometric losses are well designed and appear complementary. The ablation results suggest that both contribute to performance, and the visualization provides useful qualitative support for the intended hierarchical structure. (3) The experimental evaluation is thorough and appears fair. The use of consistent data splits and feature extraction across methods is appreciated, and the results show consistent improvements across both datasets and multiple metrics.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    (1) The overall idea is interesting, but the methodological novelty appears somewhat incremental. Many of the main ingredients, hyperbolic embeddings, entailment-style losses, hierarchical WSI modeling, and hyperbolic attention, have been explored previously. The paper would benefit from a clearer explanation of what is technically new in this work and why the particular integration is non-trivial. (2) The experimental evaluation is somewhat limited. The results are reported on two subsets with standard text generation metrics, but there is no clinical or semantic evaluation to support the claim of diagnostically accurate report generation. (3) The hierarchical text decomposition appears central to the method, but it is not described in sufficient detail. Since the approach depends on reliable segmentation of reports into local, regional, and global components, more explanation and some validation of this preprocessing step would be important. (4) The baseline comparison could be expanded. In particular, comparisons with more recent pathology vision-language models and with prior hyperbolic approaches for WSI analysis would help clarify whether the observed gains come specifically from the proposed geometric design. (5) Several implementation details are missing or unclear, including the curvature setting, important loss weights, and the embedding dimensionality. Reporting these details would improve reproducibility. (6) The paper would also benefit from qualitative examples showing generated reports at different hierarchy levels alongside the reference reports. (7) Minor issue: References [4] and [5] appear to cite the same paper.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper presents a well-motivated approach and shows promising empirical results, particularly in its use of hyperbolic geometry to model hierarchical structure in whole slide images. However, my overall score is tempered by the relatively incremental nature of the technical contribution, the limited evaluation scope, and the lack of stronger evidence that the generated reports are clinically reliable beyond improvements on standard text-overlap metrics. In addition, several important methodological details and baseline comparisons are missing, which makes it harder to fully assess the source and significance of the reported gains. Overall, I find the work interesting and potentially useful, but in its current form I place it slightly below the acceptance threshold.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    Thank you for the authors’ detailed responses. Most of my concerns have been addressed satisfactorily. I believe the manuscript is now suitable for acceptance, and I hope the authors will incorporate the suggested revisions in the final version.



Review #2

  • Please describe the contribution of the paper

    This paper proposes H2M-Net, a hierarchical hyperbolic memory network for pathology report generation from whole slide images. Its main contribution is to model the inherent local–region–global hierarchy of pathology images and reports in hyperbolic Lorentz space, rather than using conventional Euclidean feature representations. The method introduces two key ideas: first, hierarchical geometric constraints through a radial position loss and an entailment loss to preserve parent–child semantic structure; second, a bottom-up hierarchical memory mechanism that propagates diagnostic information across levels during report generation. The paper shows that this design improves pathology report generation performance on the PathText BRCA and LUAD benchmarks compared with prior methods.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    (1) The methodology is interesting and reasonably well motivated. The paper introduces a hyperbolic-space formulation for pathology report generation, which is a distinctive and methodologically meaningful idea. Instead of treating multi-scale pathology semantics in a flat Euclidean space, the method is built around the assumption that both WSIs and reports have an inherent local–region–global hierarchy. The use of hyperbolic geometry gives the work a clearer modeling motivation than a simple architectural modification, and the hierarchical memory mechanism together with the geometric constraints makes the framework more coherent.

    (2) The experimental design is reasonably thorough and generally supports the proposed method. The paper evaluates the model on two benchmark subsets, BRCA and LUAD, and compares against several prior pathology report generation methods under a unified setting. The results are reported across multiple metrics, and the paper also includes ablation studies, hyperparameter analysis, and embedding visualization. These additional analyses are useful because they provide evidence for the role of the main design choices rather than relying only on final benchmark numbers.

    (3) The paper is clearly written and well organized. The overall structure is easy to follow, and the motivation, method, and experiments are presented in a logical order. The figures are also helpful for understanding both the conceptual idea and the architecture, especially the overview of the bottom-up hierarchical framework. Overall, the manuscript communicates a technically interesting method in a relatively clear and readable way.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    (1) The current evaluation metrics do not sufficiently demonstrate the clinical efficacy. The paper mainly relies on general NLG metrics such as BLEU, METEOR, and ROUGE. These scores are useful as a basic reference, but they are not sufficient to show whether the generated pathology reports are medically accurate or clinically meaningful. This issue is especially important here because the paper’s main claim is hierarchical diagnostic consistency, yet the evaluation does not separately assess performance at the local, region, and global levels. Overall, the current evaluation does not fully demonstrate the claimed medical or practical value of the method.

    (2) Some key methodological details are still under-specified. In particular, the report hierarchy is constructed using keyword prompts and regex-based matching, but the paper does not provide enough detail about the actual rules, prompt design, or how different text segments are assigned to the local, region, and global levels. Since this hierarchical decomposition is a core part of the method, the lack of implementation detail weakens reproducibility and also makes it harder to judge how robust the reported hierarchy really is.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    My overall score is mainly based on the gap between the methodological interest of the paper and the current level of evidence for its clinical effectiveness. On the positive side, the paper presents an interesting and reasonably novel idea by introducing hyperbolic-space modeling into hierarchical pathology report generation, and the overall framework is coherent. The paper is also clearly written, and the experimental section includes useful ablations and analyses. However, the main factors that lowered my score are related to evaluation. Current evaluation metrics do not sufficiently demonstrate the clinical efficacy of the proposed method, since the paper relies mainly on general NLG metrics without showing whether the generated reports are medically accurate or clinically meaningful across the claimed local, region, and global hierarchy.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    After reading the authors’ rebuttal, I maintain my recommendation to accept the paper.



Review #3

  • Please describe the contribution of the paper

    This paper proposes H2M-Net, a hierarchical hyperbolic memory network for pathology report generation from whole-slide images. The key idea is to model the inherent local-to-region-to-global structure of pathology images and reports in hyperbolic Lorentz space rather than standard Euclidean space. The method combines three main elements: 1) hierarchical image/text decomposition and embedding into a hyperbolic manifold, 2) explicit geometric constraints via radial position loss and entailment loss to preserve hierarchy, and 3) a bottom-up cascaded generation framework with a memory encoder to propagate information across semantic levels. The paper reports improved performance over prior pathology report generation methods on the PathText BRCA and LUAD benchmarks.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The paper identifies an important limitation of prior WSI report generation methods: pathology data and pathology reports are inherently hierarchical, but many existing methods treat representations as relatively flat Euclidean features. This is a reasonable and interesting observation for this task. 2.The use of hyperbolic geometry, specifically the Lorentz model, is an interesting contribution to pathology report generation. The paper goes beyond simply replacing the embedding space by also introducing hierarchy-aware constraints, including radial position loss and entailment loss, which are conceptually aligned with parent-child semantics. 3.The bottom-up, cascaded framework with a visual encoder, a memory encoder, and a text decoder is a sensible architecture for local-to-global diagnostic reasoning. The memory mechanism is a meaningful design choice because it tries to preserve information as the model moves across levels of abstraction.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.Clinical validity of the generated reports remains insufficiently assessed. The evaluation relies entirely on BLEU, METEOR, and ROUGE-L. These are standard text-generation metrics, but for pathology report generation, they are not enough to establish factual correctness, diagnostic safety, or clinical usefulness. A pathology-specific factuality evaluation, section-level correctness assessment, or expert review would substantially strengthen the paper. 2.The “hierarchical report decomposition” seems heuristic and under-specified. The paper states that keyword prompts and regular-expression-based matching are used to split reports into local, regional, and global parts. This step is important because it defines the supervision hierarchy, yet it is only briefly described. It is unclear how robust this segmentation is across reports, how much manual engineering is involved, and whether errors in this decomposition propagate into training. 3.The breadth of validation is limited, and some claims are slightly stronger than the evidence supports. The experiments use two PathText subsets and a unified feature extractor/preprocessing pipeline. It would be helpful to know whether the gains are robust across other pathology datasets, encoders, or report styles, and whether performance holds under domain shift. Then, the conclusion mentions “diagnostically accurate reports,” but no direct clinical accuracy studies or pathologist assessments are presented. That wording should be toned down unless stronger evidence is added.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper addresses an important task and introduces a technically interesting idea by bringing hyperbolic hierarchical modeling into pathology report generation. The method is reasonably novel, the architecture is coherent, and the empirical results are consistently positive on two benchmarks with supporting ablations. These are meaningful strengths. While validation remains mostly text-overlap-based, the hierarchical text decomposition is under-explained, and the practical/clinical significance of the gains has not yet been fully established. Overall, this paper is a solid methodological study with promising results, but it is not yet a fully compelling clinical-language generation study.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The rebuttal successfully clarified the hierarchical report decomposition pipeline and added clinically more meaningful evaluations. These additions substantially strengthen the paper compared with the original submission.



Author Feedback

We thank the reviewers for their insightful comments, and especially the positive recognition of our motivation (noted as ‘clear and well justified’).

[Report Decomposition (R1,R2,R3)] Reports are decomposed using a “low-level priority” regex matching principle designed by our pathologists: individual sentences are categorized via lexicons (Global: diagnosis/carcinoma/stage/grade/subtype/conclusion/overall/status/malignant/benign; Region: size/diameter/infiltration/invasion/margin/boundary/multifocal/location/gross finding/lymphovascular invasion; Local: atypia/differentiation/mitosis/necrosis/calcification/stroma/pleomorphism/papilloma/proliferative/in situ), with unmatched sentences defaulting to Global. Empty levels are backfilled with the full report. During training, the supervision of the Global level is nested, encompassing both the global report and all sub-level reports.This design ensures semantic consistency, preventing error propagation even if minor matching deviations occur. The following comparison (TCGA-B6-A0IP) validates our approach: unlike Bi-Gen, which misidentifies cancer type and size, our method ensures an accurate Global diagnosis while precisely capturing Region (size) and Local (morphology) details. GT (Decomposed): [Global] The pathological diagnosis is infiltrating carcinoma, lobular type… [Region] The gross examination revealed a 1.7 cm tumor with no lymphatic/vascular invasion. [Local] The non-neoplastic breast tissue showed proliferative fibrocystic changes. Ours: [Global] The findings are consistent with invasive lobular carcinoma of the breast… [Region] Gross examination identifies a 1.7 cm solitary lesion; no macroscopic evidence of multifocality. [Local] The presence of intraductal papilloma and fibrocystic changes. Bi-Gen: The pathological slide indicates that the patient underwent a modified radical mastectomy on the right breast. The mass was found to be a poorly differentiated invasive ductal carcinoma with a tumor size of 2.0 cm.

[Evaluation Metrics (R1,R2,R3)] We report BLEU and METEOR to maintain consistency with prior MICCAI works like HistGen. To address medical validation requirements, we provide the following supplementary evaluations: Diagnostic Accuracy: Subtype identification accuracy on the BRCA dataset reached 0.613, significantly outperforming Bi-Gen (0.525). Clinical Semantics: Our model achieved a $Fact_{ent}$ score [1] of 0.483, surpassing Bi-Gen (0.479) and MI-Gen (0.467). Furthermore, balanced scores across the Local (0.535), Region (0.510), and Global (0.483) levels confirm the model’s diagnostic reliability across all granularities.

[About Novelty (R1)] 1.We emphasize that this work is the first to introduce hyperbolic geometry to pathology report generation. WSIs and reports naturally possess hierarchical structures; leveraging this hierarchy effectively mitigates logical inconsistencies during report generation. Accordingly, we utilize Hyperbolic Space to unify both modalities. 2.The radial position constraint explicitly regulates the hierarchical mapping by embedding global features near the hyperbolic origin while pushing local details toward the boundary. 3.Unlike traditional single-modality hierarchical WSI modeling, we propose a cross-modal hierarchical framework. By using a Memory Bank to fuse historical semantic priors with current visual features, our model simulates a pathologist’s global-to-local reasoning, ensuring strict logical consistency across diagnostic granularities.

[Other questions (R1)] Implementation Details: Image embedding dimension is 1024, text is 512, and initial curvature $\kappa = 1.5$. Weights for $L_{ent}$ and $L_{pos}$ are set to 0.1.Code will be open-sourced. References: The duplicate citations resulted from different BibTeX sources(GitHub vs. Google Scholar) and will be corrected in the revised version.

[1] Improving factual completeness and consistency of image-to-text radiology report generation.




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    All three reviewers acknowledged that this paper does have merits. It proposes an interested and well justified idea. The experimental designs are also reasonable and, in principle, support the proposed idea. The paper is well written.

    However, the reviewers also raised some major concerns.

    All three reviewers consider the current evaluation is not sufficient to demonstrate the clinical effectiveness and usefulness of the proposed model. Some reviewers argued that while the idea is interesting, the methodological novelty is somehow limited. The reviewers also suggested comparisons with some recently proposed/released models. Many implementation and experiment details should be further clarified.

    In the rebuttal, please carefully study the comments raised by the reviewers, and provide responses to them particularly to the major concerns mentioned above.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    After rebuttal, all reviewers suggest accepting this paper. Please further revise the manuscript bases on the comments in the final version.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The paper presents a well-motivated and novel approach for pathology report generation by modeling the inherent local-to-global hierarchy of whole slide images within a hyperbolic Lorentz space. In their rebuttal, the authors successfully resolved the reviewers’ primary concerns by detailing the text decomposition pipeline and providing clinically meaningful evaluation metrics, such as diagnostic accuracy and factual completeness. Because all three reviewers found these additions compelling and upgraded their final opinions to “Accept,” the paper is recommended for the conference.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The reviewers appreciate the novelty of incorporating hyperbolic geometry for hierarchical modeling in pathology report generation and acknowledge the coherent framework. However, key concerns include the incremental nature of the technical contribution, limited evaluation scope, insufficient demonstration of clinical reliability, and missing methodological details. Based on the authors’ rebuttal addressing these issues satisfactorily, the general consensus is to recommend acceptance, with suggestions for further strengthening the clinical significance in the final version.



back to top