Abstract

Accurate cross-modal deformable medical image registration is essential for integrating complementary clinical information across modalities such as MRI and CT. However, the significant modality gap makes it difficult to establish reliable correspondences from raw image appearances. Contrastive learning has emerged as a promising approach for extracting modality-agnostic features for correspondence estimation. However, existing methods typically treat voxels as independent instances in contrastive learning, without explicitly modeling hierarchical anatomical context and topology, which are critical for disambiguating correspondences and preserving deformation plausibility. To address this limitation, we propose the Hierarchical Anatomy-Grounded Representation (HAGR) framework. HAGR augments contrastive learning with cross-view feature reconstruction to encourage structurally consistent, anatomy-aware representations beyond isolated local features. It further learns a hierarchical feature space in a coarse-to-fine manner, where a matchability-aware coarse stage provides global anatomical context priors and a coarse-guided fine stage improves local discriminability through implicit hard-sample mining. Extensive experiments on cross-modal registration benchmarks demonstrate that HAGR achieves accurate and robust registration compared with existing registration methods, providing a generalizable and reusable foundation for reliable cross-modal information fusion. (https://github.com/jiaem1/HAGR)



Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0927_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/jiaem1/HAGR

Link to the Dataset(s)

N/A

BibTex

@InProceedings{MiJia_Learning_MICCAI2026,
        author = { Mi, Jia AND Jiang, Caiwen AND Yuan, Haoyue AND Li, Fan AND Li, Xiaoye AND Sun, Kaicong AND Shen, Dinggang},
        title = { { Learning Hierarchical Anatomy-Grounded Representation for Cross-Modal Registration } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16889},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    The paper presents a representation learning method based on contrastive learning, aiming to learn modality-invariant features for cross-modal registration.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The motivation that existing contrastive learning approaches often treat voxels as independent instances and overlook the inherent hierarchical anatomical context is well-founded. Exploring alternative strategies for constructing contrastive learning along this direction is both relevant and promising.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The primary weakness of the paper lies in its lack of clarity, which significantly hinders comprehension of the proposed method.

    Sections 2.1 and 2.2, in particular, are difficult to follow due to ambiguous notation and insufficient explanation. For example, Section 2.1 introduces notation with subscript I without prior definition. Additionally, the description of the hard-sample mining strategy is unclear. The paper states: “for each fine-level anchor, we first select top-K_p coarse correspondence proposals in view B according to coarse feature similarity. We then collect local 3D windows of size \alpha^3 around these proposals and merge them into a fine-level candidate pool P_{f,i}. ” However, the term “correspondence proposals” is not defined, and the fine-level candidate pool P_{f,i} is not specified.

    Furthermore, Equation (2) appears to define a contrastive learning objective but uses a reconstructed anchor as the query. The paper does not provide sufficient motivation or interpretation for this design choice. In Equation (3), the term L_{\text{rec}}^f is introduced without definition. The role of L_{\text{mat}} is also unclear. It is not well explained why this term is necessary or what is the geometry-derived matchability labels are used as ground truth.

    Additional suggestions: Since the experiments are conducted on challenge datasets, it would strengthen the paper to include comparisons with methods from those challenges. The figures are too small, and the color-coded deformation fields are difficult to interpret. It would be more meaningful to visualize the learned representation.

    Overall, while the idea is promising, the paper would benefit from clearer presentation.

  • Please rate the clarity and organization of this paper

    Poor

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The presentation of the core idea lacks clarity, which makes it difficult to fully assess the contribution. Accordingly, I recommend a weak reject at this stage, unless the authors can provide substantial clarification and meaningful revisions to improve the manuscript.

  • Reviewer confidence

    Not confident (1)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    The paper proposes HAGR, a framework for cross-modal deformable registration (e.g., CT-MRI). Key contributions: Hierarchical Anatomy-Grounded Learning: Replaces independent voxel-wise contrast with a coarse-to-fine architecture. The coarse stage captures global context via “matchability-aware” reconstruction, while the fine stage refines local details using implicit hard-sample mining. Cross-View Reconstruction: Augments contrastive learning with a reconstruction task to enforce structural consistency beyond simple feature alignment. Performance: Achieves SOTA accuracy on abdominal CT-MR benchmarks and demonstrates strong zero-shot generalization to unseen domains (Thorax, Whole-Body) without fine-tuning.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The shift from point-wise contrast to structured, hierarchical reconstruction effectively addresses the ambiguity in cross-modal correspondence. The “dustbin” mechanism for unmatchable regions is a clever addition.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The core technical contribution—replacing contrastive loss with a reconstruction loss—is essentially a form of feature interpolation or smoothing over local neighborhoods. Reconstructing a feature vector f_A as a weighted sum of nearby features f_B (via attention) is mathematically similar to existing non-local means or graph convolution operations already prevalent in registration.

    There is no direct ablation comparing Point-wise Contrast vs. Proposed Reconstruction under identical architectural settings. Without isolating the loss function change, it is unclear if the gains come from the “reconstruction” logic or simply from the hierarchical coarse-to-fine architecture (which could arguably work with standard contrast too).

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I recommend a Weak Accept primarily based on the strong empirical results and the demonstrated zero-shot generalization capability, which are of significant practical value to the community. The method consistently outperforms SOTA baselines across multiple diverse datasets. However, my confidence is tempered by concerns regarding the theoretical novelty and experimental rigor: The core “reconstruction” mechanism appears to be an incremental variation of existing feature interpolation/attention techniques rather than a fundamentally new paradigm. The experiments do not fully isolate whether the performance gains come from this specific loss function or the hierarchical architecture itself.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #3

  • Please describe the contribution of the paper

    HAGR proposes a hierarchical representation learning framework for cross-modal deformable registration. Rather than enforcing direct point-to-point feature similarity, HAGR reconstructs an anchor feature in one view via attention-weighted aggregation of candidates from the other view, and pulls the reconstructed feature toward the original via a contrastive objective. A coarse stage captures global anatomical context with a matchability-aware dustbin, while a fine stage performs coarse-guided hard-sample mining for local discriminability. Registration is performed by optimizing deformation fields in the learned feature space using ConvexAdam, requiring no paired multi-modal training data.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. Reconstruction requires the entire similarity distribution to be correct, not just one feature pair — this is a more demanding and informative training signal than standard point-wise contrastive learning.
    2. Soft rejection of unmatchable regions prevents corrupted gradients from augmentation artifacts and FOV mismatches. Geometry-derived labels make this fully self-supervised. 3.Zero-shot transfer to ThoraxCBCT and Whole-Body (CT → 4 MR contrasts) without fine-tuning demonstrates that the representation captures anatomy rather than intensity statistics.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The paper argues point-wise contrastive ignores anatomical context, but 3D ResNet features already encode large receptive fields spanning hundreds of voxels. The real advantage of HAGR is a stronger proxy task, not context-awareness per se. This framing should be corrected. 2.8 paired cases on the primary benchmark, with standard deviation of ±27.50%, limits the statistical reliability of conclusions.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    HAGR delivers consistent state-of-the-art performance across multiple benchmarks with genuine cross-domain generalizability. Beyond adapting existing self-supervised paradigms, the reconstruction-as-proxy-task formulation represents a meaningful conceptual shift: rather than enforcing pairwise point similarity, it compels the encoder to internalize the full correspondence structure across views, yielding more anatomically grounded embeddings. Three concerns, however, prevent an unconditional strong accept. The motivational claim that point-wise contrastive is context-blind is overstated given the large receptive fields of modern volumetric encoders. The hard-sample mining characterization in the fine stage lacks empirical validation of candidate diversity — spatially clustered proposals would degenerate the candidate pool. Finally, conclusions drawn from only 8 test pairs with DSC ±27.50% carry limited statistical weight. These weaknesses do not undermine the core contribution. If addressed, this work would comfortably warrant a strong accept.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    In a registration research paradigm that has largely focused on architectural advances, the proposed pre-registration encoder learning strategy represents an important and timely contribution. In particular, the core idea of learning the correspondence distribution itself, rather than relying only on pairwise similarity, is novel and provides valuable insight. Although the limited number of cases and the large variability in the results remain limitations, the authors’ rebuttal addressed the concerns regarding overstated claims and framing, and I do not believe these issues undermine the core contribution; they can be reasonably addressed in future work.



Author Feedback

R1: Clarity/notation. Subscript i denotes a spatial index. In hard-sample mining, a “correspondence proposal” is a candidate location in view B selected by similarity to the anchor in view A. For a fine-level anchor f^A_{f,i}, we map its location to the corresponding coarse-grid and select the top-K_p proposal locations in view B with highest feature similarity. The fine-level candidate pool P_{f,i} is the set of fine-level feature vectors collected from local windows centered around the proposals. L^f_rec is the fine-level instance of the unified loss L^l_rec in Eq. (2). Table 1 includes ConvexAdam, a strong Learn2Reg challenge method. We will revise Secs. 2.1–2.2 to define symbols on first use and improve the figure visualizations. R2: Design rationale. Using the reconstructed anchor in Eq. (2) requires the candidate similarity distribution to recover the anchor, whereas using the original anchor would only impose pairwise discrimination. L_mat is a binary cross-entropy loss supervising the coarse matchability predictor that controls the dustbin. It enables soft rejection of unmatchable anchors, which have no valid correspondence, thereby reducing noisy gradients from non-overlapping regions. Binary labels come from known spatial augmentations, where anchors mapped inside the valid overlap are matchable and the rest unmatchable. R2: Ablation evidence. Table 3’s “Contrastive-Only” is the requested controlled comparison. It uses the same two-level hierarchy, training data, and registration stage, while replacing reconstruction-consistency learning with point-wise contrastive learning. The DSC drop (74.56→64.31) supports the contribution of the reconstruction-based proxy task beyond the hierarchical design. R1/R2: Novelty beyond interpolation/smoothing. The weighted sum is not a forward output like non-local means or graph convolution, but a training-time proxy task. Eq. (1) turns the cross-view candidate similarity distribution into a reconstructed query, and Eq. (2) forces this distribution to recover the anchor. Thus, HAGR supervises the ranking of plausible correspondences and encourages the candidate distribution to be correspondence-consistent, rather than averaging features for propagation or matching a single pair. R3: Point-wise contrastive framing. We agree and will rephrase. The main distinction is not receptive-field size, since 3D encoders encode large context, but the supervision imposed by the proxy task. Reconstruction constrains the cross-view correspondence distribution, whereas point-wise contrast supervises only pairwise similarity. R3: Evaluation scope. The 8 paired Learn2Reg cases limit statistical power; we agree this benchmark must be interpreted cautiously. However, the comparison remains fair since all methods use the same paired cases and protocol. In Table 1, HAGR improves mean DSC over the strongest baseline (71.23→74.56) while reducing HD95 (7.77→4.97), suggesting the gain is not merely a fluctuation. Whole-Body (20 subjects, 4 MR contrasts) and ThoraxCBCT (11 subjects) further support generalization (Table 2). R3: Candidate diversity. Fine-stage proposals form the reconstruction pool, not explicit contrastive negatives. Their role is to provide plausible but hard-to-distinguish candidates for learning discriminability. Ambiguous coarse responses may yield multiple plausible hypotheses, while concentrated responses define local refinement around confident candidates; the α³ windows still contain locally varying anatomy. Diffuse weighting yields a mixed query that fails to match the positive anchor or suppress ambiguous alternatives in Eq. (2), so supervision comes from resolving local ambiguity rather than spatial diversity alone. We will rephrase to avoid implying diversity-guaranteed mining. R1: Reproducibility. Code will be released; all training/evaluation datasets are public except the in-house Whole-Body set. We will specify all hyperparameters and augmentation ranges.




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper received mixed recommendations from the reviewers, including one Weak Reject, one Weak Accept, and one Accept.

    The paper proposes an unsupervised cross-modal registration approach based on contrastive learning. At the same time, the reviewers identified several substantive concerns, including limited clarity (R1), ambiguous notation (R1), insufficient justification for key design choices (R1), the absence of critical ablation studies (R2), insufficient support for the stronger claims regarding point-wise contrastive learning (R3), and evaluation on a limited dataset (R3).

    Overall, the paper shows methodological merit, and the meta-reviewer therefore recommends that it proceed to rebuttal. The authors are encouraged to focus their response on the major concerns outlined above.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    This paper received mixed initial recommendations. Following the rebuttal, Reviewer 3 upgraded the rating to Accept, explicitly recognizing the value of learning the correspondence distribution itself, while Reviewer 2 maintained an Accept recommendation. Reviewer 1 did not update the original Weak Reject assessment. Having revisited the paper in light of the reviews and rebuttal, I acknowledge both its merit and its limitations: the work offers novelty in multimodal image structural feature learning, while the evaluation could be strengthened, for example through more direct analysis of the learned features and the induced loss landscape. On balance, the strengths of the paper slightly outweigh its weaknesses. In light of the post-rebuttal discussion and the overall reviewer consensus, I recommend Accept. For the camera-ready version, the authors are encouraged to further improve the clarity of the presentation and to incorporate the reviewers’ constructive suggestions where appropriate.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    This paper proposes a cross-modal deformable image registration method leveraging representation learning. The reviewers raised some concerns about limited clarity and notations and limited experimentation. The rebuttal addressed important issues and two reviewers agree that the paper is ready for publication at MICCAI. I follow their recommendation, as the work is interesting and the results are convincing.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    This paper addresses an important and practical problem in cross-modal registration. Two reviewers recommended acceptance after rebuttal and pointed out the strong empirical results and methodological contribution. One reviewer who rated Weak Reject did not give final recommendation, but his/her concerns about clarity were clarified in authors’ rebuttal. Therefore, I recommend acceptance.



back to top