List of Papers Browse by Subject Areas Author List
Abstract
Many existing learning-based deformable image registration methods impose constraints on deformation fields to ensure they are globally smooth and continuous.
However, this assumption does not hold in cardiac image registration, where different anatomical regions exhibit asymmetric motions during respiration and movements due to sliding organs within the chest.
Consequently, such global constraints fail to accommodate local discontinuities across organ boundaries, potentially resulting in erroneous and unrealistic displacement fields.
In this paper, we address this issue with \textit{MemWarp}, a learning framework that leverages a memory network to store prototypical information tailored to different anatomical regions.
\textit{MemWarp} is different from earlier approaches in two main aspects: firstly, by decoupling feature extraction from similarity matching in moving and fixed images, it facilitates more effective utilization of feature maps; secondly, despite its capability to preserve discontinuities, it eliminates the need for segmentation masks during model inference.
In experiments on a publicly available cardiac dataset, our method achieves considerable improvements in registration accuracy and producing realistic deformations, outperforming state-of-the-art methods with a remarkable 7.1\% Dice score improvement over the runner-up semi-supervised method.
Source code will be available at \url{https://github.com/tinymilky/Mem-Warp}.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1764_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: N/A
Link to the Code Repository
https://github.com/tinymilky/Mem-Warp
Link to the Dataset(s)
https://www.creatis.insa-lyon.fr/Challenge/acdc/databases.html
BibTex
@InProceedings{Zha_MemWarp_MICCAI2024,
author = { Zhang, Hang and Chen, Xiang and Hu, Renjiu and Liu, Dongdong and Li, Gaolei and Wang, Rongguang},
title = { { MemWarp: Discontinuity-Preserving Cardiac Registration with Memorized Anatomical Filters } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15003},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The authors present a novel framework for a learning based registration method with improves on the state-of-the-art. The idea is to use a memory network to output better displacement fields depending on the anatomical structure of the images.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper in principle is well written and well sourced. The idea is well suited to the problem and improves on the state-of-the-art. The additions are ablated and show improvements across various metrics on a public dataset.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The results are only validated on one benchmark. While various hyperparameter are provided it is unclear if the paper can be re-implemented. No significance testing was performed and the visual results are not very convincing. The paper can be hard to follow at times.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
A public dataset was used for validation
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
In the conclusion the term significant improvement is used, even though no significance testing was performed.
The paper can be hard to follow at times, as various definitions are used later on, which depend on other definitions inside the text. For example, we need w_i to construct the residual displacement field \laplace u, which is defined someplace else, while the section about w_i points to Eq. 1 for the construction of the displacement field, which doesn’t even use \lablace u.
Same with J_f. Is that constructed by the algorithm for the fixed image or given as input. Sometimes the notation can be misleading or very hard to decipher.
L_reg is not specified.
The visual example is not very convincing. I am not sure what the ERF is supposed to tell me and if a larger ERF is even preferable. For the camera-ready version it would be preferable to cleanup citations e.g. [15]
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It is a cool adition to registration methods and is tested and ablated on a public dataset. Still, some cleanup of the manuscript would be appreciated by future readers.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The paper introduces an innovative semi-supervised deformable image registration framework optimized for the discontinuities between different cardiac cycles. By incorporating a memory network that stores and applies information specific to anatomical regions, the authors significantly enhance registration accuracy, addressing the limitations of global smoothness assumptions prevalent in traditional methods. Experimental results demonstrate a notable 7.9% improvement in Dice score over existing semi-supervised methods on a public cardiac dataset, marking a significant academic contribution.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This approach is particularly novel because it allows the model to preserve discontinuities at organ boundaries, a challenge that traditional deformable image registration methods, which assume global smoothness, fail to address. This formulation is interesting as it leverages the capacity of memory networks not just for storing but for dynamically applying learned anatomical filters during the registration process, enhancing the specificity and accuracy of the model.
- The paper presents an original application of multi-scale Laplacian pyramids to decouple feature extraction from similarity matching in image registration. This method allows for more precise control over the registration process at different scales, which is crucial for capturing the complex dynamics of cardiac motion. The use of Laplacian pyramids in this way is particularly innovative as it expands the effective receptive field of the network, enabling it to capture finer details across scales, which is critical in medical imaging where precision is paramount.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Although the MemWarp method performs well in experiments, the paper lacks thorough descriptions of some key technical details. For example, the paper mentions the use of dynamic filters and multi-scale Laplacian pyramids but falls short in detailing how these filters are specifically implemented to adapt to particular anatomical structures. To enhance academic rigor and practicality, it is recommended that the authors provide more implementation details or pseudocode to help readers better understand and replicate the findings.
- The paper provides comparisons with other methods but lacks a detailed analysis of why MemWarp outperforms these alternatives. For instance, it would be beneficial to discuss the main technical factors behind the superior performance in the best cases and analyze the contribution of each technological aspect to performance improvements.
- MemWarp introduces a complex network structure and multi-scale processing, potentially leading to high computational demands. The paper should discuss the computational complexity of the model, including training and inference times and hardware requirements. Detailed information on computation times and required hardware resources is crucial for assessing the model’s feasibility in practical clinical environments.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
None.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
-
While the existing dataset experiments validate the method’s effectiveness, the paper lacks robustness tests under different settings, such as images of varying resolutions or under different cardiac conditions. Future studies should include these variables to assess the model’s performance under a broader range of application conditions.
-
The paper provides a relatively weak discussion on the theoretical foundations, such as how dynamic filters and memory networks specifically improve cardiac image registration. It is recommended that the authors strengthen the theoretical discourse in subsequent work, possibly through mathematical modeling to elucidate how these techniques affect registration accuracy and speed.
-
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I value the design motivation of the method and the logical coherence of the article more than the operation and results.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The authors proposed a method called MemWarp to improve cardiac image registration. First, they applied a Laplacian pyramid warping network on feature maps for multi-scale deformation estimation. Second, they introduced a memory matrix for dynamic filtering based on anatomical information learned during training phase. Their experiments showed that the proposed MemWarp outperformed other competing registration methods in terms of Dice score as well as preserving the discontinuity along anatomical borders.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Anatomy-driven memory network: the authors proposed a memory matrix that leverages on the anatomical mask during training phase to enhance the estimation of flow and preserve discontinuities across anatomical structures.
- Thorough comparison with competing methods under various settings: they have compared the proposed method in unsupervised and semi-supervised setting to showcase the superior registration performance of MemWarp.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Indirect evaluation of discontinuities: SDlogJ evaluates the general smoothness in the whole image space, does not necessarily relate to the discontinuities of anatomy boundary. It may be interesting to concentrate only on the boundary of considered anatomical structures to demonstrate the desired discontinuities. In addition, in the right panel of Figure 3, it seems that DDIR demonstrates a more distinctive discontinuities, while its SDlogJ is smaller.
- Insufficient evidence for ERF analysis: the discussion on ERF seems out-of-scope. First, the proposed MemWarp does not demonstrate superior performance than other methods under unsupervised learning setting. It’s not sufficient to support the claim that larger and denser ERF benefits registration performance. This part seems quite unnecessary.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
The dataset is public.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- The authors employed a model to obtain segmentation masks for DDIR. Why did they use model predicted segmentation masks instead of the ground truth masks provided with ACDC dataset? As the authors stated in Section 3.2 that the reason why DDIR under-performs compared to MemWarp potentially comes from segmentation accuracy, it may be interesting to see the performance of DDIR using ACDC ground truth masks to verify this claim.
- Regarding the train-val-test split, do the authors consider patient distribution? Do the train and test splits have image pairs from the same patient?
- Does the HD95 metric represented in Table 1 represent the averaged HD95 of the three anatomical structures considered? It’s not very clear.
- Error in text just above Equation 2: the dimension of Ifi is not correct. It’s recommended that the authors carefully check all the mathematical annotations in Section 2.3.
- In the section of ablation study, it’s not clear the Dice loss refers to which element appeared in Equation 4. Does it refer to L(dsc) or L(rgn)?
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It’s a very interesting work with efficient use of features maps for cardiac registration. The design of memory network has implicitly integrated the segmentation mask into the whole pipeline, which seems to work very well and gets rid of segmentation mask at inference time. However, some claims and conclusions need to be carefully considered and further explained.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
N/A
Meta-Review
Meta-review not available, early accepted paper.