List of Papers Browse by Subject Areas Author List
Abstract
Deformable medical image registration (DIR) benefits from both pixel-level correspondence modelling and high-level semantic understanding of anatomical structures. However, most learning-based methods remain purely data-driven and fail to leverage rich domain knowledge describing anatomical structures and inter-organ relations. In this paper, we present KGT-Reg, a unified multimodal framework that integrates knowledge graph (KG) embedding and textual semantics into deep registration networks. KGT-Reg fuses visual features extracted from paired medical volumes with semantic embeddings derived from organ-level textual descriptions and graph neural encodings of anatomical KGs. Specifically, the KG encodes global relation priors across anatomical entities to represent organ adjacency and spatial topology, while text embeddings capture localized semantic context. A graph-enhanced fusion module further dynamically modulates feature decoding to promote spatially consistent deformations. Experiments on four benchmark datasets — OASIS, IXI, LPBA40 (brain MRI), and Learn2Reg 2020 (abdomen CT) — demonstrate consistent improvements over state-of-the-art baselines, validating the effectiveness of structured knowledge integration and text aggregation for multimodal DIR.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2403_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/Dduuoo621/KGT-Reg
Link to the Dataset(s)
OASIS dataset: https://sites.wustl.edu/oasisbrains/
IXI dataset: https://brain-development.org/ixi-dataset/
LPBA40 dataset: https://www.loni.usc.edu/research/atlas_downloads
Abdomen dataset: https://learn2reg.grand-challenge.org/Learn2Reg2020/
BibTex
@InProceedings{DaiYuh_KGTReg_MICCAI2026,
author = { Dai, Yuhe AND Huang, Zhiyong AND Wang, Xiaohong AND Huang, Weimin},
title = { { KGT-Reg: Knowledge-Guided Multimodal Framework for Medical Image Registration } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16889},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper presents a multimodal registration framework based on Graph-conditioned Text-image Fusion. Compared to the SOTA, using KG with image registration is novel. The author evaluates the proposed method using four datasets, and the result shows the proposed method is either the best or the second best (in terms of Dice).
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The novelty of using KG, which is able to fuse the text and image data. The intensive experiments on different datasets across four diverse datasets confirm its consistency.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Lack of clarity in the methodology: Further clarification is required regarding Figure 1, specifically to match the description of the three components of the proposed framework and the input of the GCFB block and the role of the duplication arrows to ensure reproducibility. It would help to explain the training procedure and the formulation of the loss function to ensure reproducibility.
It would help and make it clearer to provide the results as “images”.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The manuscript presents a novel framework that has undergone rigorous and intensive evaluation. If the authors can enhance the methodological descriptions and provide a repository link for the source code to ensure reproducibility, The findings presented here have the potential to support further development within the medical image registration
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The revisions provide necessary clarity regarding the experimental methodology and training protocols. By detailing these technical aspects more thoroughly, the author has ensured the work is sufficiently documented for reproducibility.
Review #2
- Please describe the contribution of the paper
This paper proposes a multimodal registration framework that integrates textual semantics and knowledge graph representations to enhance anatomical consistency. The approach is well-motivated and technically sound, and is validated across multiple datasets.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors build upon LKU-Net and introduce three components: (1) a frozen CLIP encoder for anatomical text embeddings, (2) a static KG constructed from organ statistics and encoded via GAT, and (3) a Graph-Conditioned Fusion Block (GCFB) that integrates image, text, and KG features through FiLM-based multi-scale modulation. The method is evaluated on OASIS, IXI, LPBA40, and Learn2Reg Abdomen CT, with comparisons to classical, CNN-, Transformer-, and text-guided methods, along with ablation studies.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The visualization of the framework (Fig. 1 and Fig. 2) lacks clarity in highlighting the key methodological contributions. Fig. 1 mainly presents the pipeline without clearly distinguishing the proposed components from standard modules, and does not explicitly convey the different roles of the text and KG branches. Fig. 2, while detailing the GCFB, remains largely procedural and does not sufficiently emphasize the distinction between semantic identity (from text) and relational/topological priors (from KG), nor how these are translated into effective multi-scale, voxel-wise modulation. Improving the visual clarity and emphasizing these roles would enhance interpretability. 2.From Table 1, the performance gains are relatively modest and, in some cases, do not surpass existing methods. In addition, prior work such as TextSCF has already incorporated textual/semantic guidance into registration, suggesting that the novelty in this aspect is limited. 3.The anatomical KG is constructed as a dataset-level static graph from training-set segmentation masks, which mainly captures dataset-specific statistical relationships rather than external or generalizable anatomical knowledge. This raises concerns about robustness under cross-dataset scenarios, and the static design may limit modeling of subject-specific anatomical variations. Consequently, the actual contribution of the KG module is not clearly demonstrated. 4.The manuscript claims a “novel multimodal framework,” which is partially supported by integrating image, text, and graph representations. However, the novelty appears to mainly arise from combining existing components rather than introducing a fundamentally new paradigm, and the strength of this claim may be somewhat overstated. 5.Although the method is evaluated on multiple datasets, the experiments are conducted in a dataset-specific manner, with training and testing performed within the same dataset. Therefore, the results mainly demonstrate robustness across benchmarks rather than true cross-dataset or out-of-distribution generalization. Additional evaluations under domain shift would be necessary to support stronger generalization claims.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the paper presents a reasonable extension of existing registration frameworks and demonstrates consistent, albeit moderate, empirical gains, several claims (e.g., novelty and multimodal contribution) are not fully supported by the current analysis. The work has potential but would benefit from stronger validation and clearer positioning of its contributions.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The responses addressed several of my concerns regarding figure clarity, the role of the KG component, reproducibility details, and the interpretation of the contribution as a knowledge-guided registration framework rather than a purely performance-driven improvement. While limitations remain, particularly the lack of explicit cross-dataset validation in the current manuscript, I believe the work presents a technically sound and interesting direction for incorporating structured anatomical priors into deformable registration. Overall, I find the contribution potentially valuable to the community and therefore support acceptance.
Review #3
- Please describe the contribution of the paper
The paper presents an approach for medical image registration by introducing a multimodal framework that incorporates image features, textual semantics, and knowledge graph priors. The idea of combining structured anatomical knowledge with learning-based registration is interesting and relevant, especially for improving anatomical consistency. The method is evaluated across multiple datasets, and the results show consistent improvements over existing baselines, suggesting the effectiveness of integrating semantic and relational information into the registration pipeline.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper includes extensive ablation studies analyzing different components of the model, such as text embeddings, KG integration, and fusion strategies, which helps in understanding the contribution of each module.
2.The modules are well explained, especially the integration of text and knowledge graph features into the registration network, making the overall idea clear.
3.The method is evaluated on multiple datasets (brain MRI and abdominal CT), which demonstrates the generalization capability of the approach.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The use of segmentation (Dice) loss alongside the registration objective makes it difficult to fully understand the true registration capability of the model. Since segmentation supervision can dominate performance, especially on unseen images, it would be more informative to analyze how the model performs without the Dice loss and how much it contributes to the final gains.
2.The architecture diagram (Fig. 1) is somewhat confusing and hard to follow. The interaction between different components, especially the multimodal fusion and conditioning, is not immediately clear and could be better visualized.
3.While the idea of integrating knowledge graphs is interesting, the paper does not clearly analyze how sensitive the model is to the quality of the constructed KG.
4.The method introduces multiple components (text encoder, KG encoder, fusion blocks), It would be good to see how much computational overhead or training stability is there compared to the baselines.
5.The evaluation mainly focuses on quantitative metrics, and there is limited qualitative analysis of failure cases whether it can be small structures or large deformations.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is solid and presents a meaningful direction by incorporating structured knowledge such as text semantics and knowledge graphs into registration models. The extensive ablation studies and clear modular design are strong aspects of the work. However, some clarifications regarding the role of segmentation supervision, architectural design clarity, and model complexity would further strengthen the paper.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
After reading the rebuttal, I appreciate the authors’ effort to address the reviewer concerns. My understanding of the paper is correct. some concerns still remain. The KG component is still based on a static population-level prior, and while this is reasonable for a first attempt, it may not fully handle strong anatomical variability or pathological changes. Also, some of the proposed additions, such as cross-dataset generalization and dynamic KG updates,
Author Feedback
We thank all reviewers for the constructive feedback. 1.Figure Clarity & Qualitative Results [R1Q1/3, R2Q1, R3Q2/5, M] We acknowledge that Fig. 1&2 require improved clarity and commit to revising them in camera-ready to better match figure and text. We will annotate tensor dimensions, distinguish each branch, and emphasize the translations. Briefly, GCFB takes the feature map F_l as input, along with voxel-wise text and KG embeddings retrieved according to segmentation labels, and the global graph vector g; these inputs are fused via FiLM to modulate F_l. The dashed arrows indicate that T_full, H, and g are computed once and broadcast to all GCFB modules. Due to page limits, we prioritized the complete methodology and quantitative evaluation in the main text. We fully agree with the essential role of qualitative results and will include the visualizations (e.g., warped images, deformation grids, failure cases…) in revision. 2.KG Contribution [R2Q3, R3Q3] As the 1st attempt of KGs in DIR, we began with a static graph to validate paradigm feasibility. The global KG prior is largely anatomy-driven, thus somewhat transferable across acquisition protocols, and it succeeds on standard benchmarks where topological relationships are population-consistent. As mentioned in the discussion, dynamic per-subject KG is the focus of our next work, and we plan to design a Graph Update Module adjusting edge weights via real-time image feature cross-correlation during training. But effectiveness needs to be validated. Regarding KG quality sensitivity, varying the training subset used for KG construction yielded final DSC accuracy variation within ±0.2% (tested during our early experiment design stage), confirming stable population-level statistics. 3.Novelty [R2Q2/4, M] We respectfully clarify the nature of our contribution. KGT-Reg’s primary claim is not a large absolute performance jump, but the introduction of a principled framework unifying 3 complementary information sources (image intensity, localized textual semantics, and global relational topology) in a single hierarchical fusion architecture. To our knowledge, it is the 1st to use KG-encoded relational priors to condition deformation field decoding in DIR. We believe further gains are achievable through future refinements, but our contribution is more about offering a new high-level KG-guided paradigm beyond pixel-level engineering. 4.Others: [R1] Reproducibility: Source code will be released upon acceptance. [R1Q2, R3Q4] Training & Loss: Full training objective is as Eq. (1). Text/KG embeddings are pre-computed once and stored as fixed lookup tables with negligible inference overhead; only the lightweight GCFB MLP adds online cost. Full reproducibility details, including hyperparameters and computational metrics (e.g. Training: ~18h; Inference: ~0.9s/vol; Memory: ~10GB), will be listed in revision. [R2Q5] Generalization: High-level KG priors are anatomically universal and would remain informative under moderate domain shift. Explicit cross-dataset experiments (train on a brain dataset, test on another) are planned for the final version. [R3Q1] About Dice Loss: Table 4(b) could address this concern: KGT-Reg(w/o Dice) outperforms backbone on both datasets, confirming gains stem from the architecture rather than segmentation supervision. The Dice term is introduced to improve smoothness, and it contributes ~1% on brain and ~3% on abdomen (large deformations benefit more from boundary guidance). [M] Jacobian Quality: Due to the DSC–regularity trade-off, we maximized DSC while keeping folding below 0.15%, consistent with accepted practice. The absolute folding rate remains negligible across datasets, confirming overwhelmingly diffeomorphic deformations. “Multimodal” Terminology: “Multimodal” here refers to multiple representation modalities (image/text/graph), not multi-image-modality registration (T1/T2). We will replace it with “multi-representation” in revision to avoid confusion.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The three reviewers agreed in the acceptance of the manuscript with scores of 4, 4, and 5.The confidence of two reviewers in their assessment is weak.
After carefully reading the paper and the reviews, I believe that the review process would highly benefit from the rebuttal phase. The novelty of the proposed method and the differenciation with respect to the state of the art should be better motivated. I advise the authors to study all reviewer comments for the improvement of the manuscript and justify the most relevant ones against the acceptance of the article in the rebuttal document.
In addition, Rv1 raised an important point: the authors did not show any image illustrating the registration results. Showing up only tables with different evaluation metrics is not acceptable for the image registration community. Every paper should show examples of the warped images, the differences after warping, and deformation grids or flows showing the quality of the transformations. Indeed, it is relevant to evaluate the performance of a method using both the DSC and metrics related with the quality of the transformations such as the Jacobian extrema or the percentage of negative Jacobians. There is a trade-off between both metrics, meaning that rising up DSC from lowering regularization inevitably rises down the transformation quality and vice-versa. The proposed method is in the level of NiftyReg for OASIS, LPBA40, and IXI. NiftyReg is not known for transformation quality.
Since multimodality in the framework of image registration means different image modalities (e.g. T1 vs T2 MRI), it would be helpful to clarify what mutimodality actually means.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
After the rebuttal stage, two of the authors accepted the manuscript. The third one did not provided a post-rebuttal answer but his/her original answer was a weak accept. The detailed comment from Reviewer 3 was incomplete, he/she was exposing some remaining concerns on his/her acceptance choice. Reviewer 1 only complained about reproducibility. Since the authors are going to publish the codes after acceptance I believe this is not a problem anymore.
I still have concerns on the novelty of the manuscript with respect to the state of the art, so the camera version should address this issue with accuracy.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
This paper prposes a registration approach that combines a relatively standard registration formulation (using losses based on normalized cross correlation or MSE with a Dice loss and presumably a diffusion regularizer) with text embeddings for what needs to be registered as well as a knowledge graph.
The main contribution of the approach appears to be the introducting of the knowledge graph as TextSFC already introduced textual embeddings but not the knowledge graph. Introducing the knowledge graph appears to provide small improvements (see Table 4). Unfortunately, as comparisons (in Table 1) are largely drawn from literature TextSCF results are not reported for 2 out of the 4 datasets. Performance of the proposed method seems to be slightly better than TextSCF on OASIS but similar or slighly worse on an abdominal CT dataset. LBPA40 and IXI results are not provided for the proposed method.
All reviewers recommended accept after the rebuttal. Hence, as a metareviewer I am not overwriting this consensus decision. Instead, I will provide some context for the community which I think is important to address some current possibilities for improvement for the proposed work.
In my opinion the paper could significantly be improved by: 1) showing actual registration results (in particular, as the folding level for the abdominal registration results seem to be high); 2) providing results for TextSCF on all the datasets to more clearly showcase the performance of the approach with respect to what appears to be the most closely related competitor; 3) providing measures of statistical difference for the approaches (as differences are sometimes rather small); 4) putting most emphasis on the knowledge graph and how it affects results (as this appears to be the core contribution; and Table 4 only shows marginal improvements); and 5) more clearly classifying the different competing approaches with respect to the kind of losses they use (e.g., with and without Dice, what similarity measure – it is for example not even clearly described which similarity measure the proposed method uses in the end (is it MSE or is it NCC)?) and 6) carefully testing the approach (and competing approaches) when the Dice loss is disabled.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Following the rebuttal, all three reviewers provided final recommendations of Accept. The major concerns regarding the clarity of the experimental methodology, the role of the knowledge graph component, and reproducibility details were satisfactorily addressed. In light of the post-rebuttal discussion and the overall reviewer consensus, I recommend Accept.