Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

The goal of image registration is to establish spatial correspondence between two or more images, traditionally through dense displacement fields (DDFs) or parametric transformations (e.g., rigid, affine, and splines). Rethinking the existing paradigms of achieving alignment via spatial transformations, we uncover an alternative but more intuitive correspondence representation: a set of corresponding regions-of-interest (ROI) pairs, which we demonstrate to have sufficient representational capability as other correspondence representation methods. Further, it is neither necessary nor sufficient for these ROIs to hold specific anatomical or semantic significance. In turn, we formulate image registration as searching for the same set of corresponding ROIs from both moving and fixed images - in other words, two multi-class segmentation tasks on a pair of images. For a general-purpose and practical implementation, we integrate the segment anything model (SAM) into our proposed algorithms, resulting in a SAM-enabled registration (SAMReg) that does not require any training data, gradient-based fine-tuning or engineered prompts. We experimentally show that the proposed SAMReg is capable of segmenting and matching multiple ROI pairs, which establish sufficiently accurate correspondences, in three clinical applications of registering prostate MR, cardiac MR and abdominal CT images. Based on metrics including Dice and target registration errors on anatomical structures, the proposed registration outperforms both intensity-based iterative algorithms and DDF-predicting learning-based networks, even yielding competitive performance with weakly-supervised registration which requires fully-segmented training data.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1540_paper.pdf

SharedIt Link: https://rdcu.be/dY6k0

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72390-2_62

Supplementary Material: N/A

Link to the Code Repository

https://github.com/sqhuang0103/SAMReg.git

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Hua_One_MICCAI2024,
        author = { Huang, Shiqi and Xu, Tingfa and Shen, Ziyi and Saeed, Shaheer Ullah and Yan, Wen and Barratt, Dean C. and Hu, Yipeng},
        title = { { One registration is worth two segmentations } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15012},
        month = {October},
        page = {665 -- 675}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper embedded the SAM into the registration framework and construct a training-free method to achieve the correspondence of paired ROIs. The idea of “one registration versus two segmentation” is the key idea in the segmentation-based registration, and this work makes a SAM-version in the paradigm.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    -I think this is an interesting attempt to apply SAM to medical image registration. The idea is novel. -The training-free ROI extraction setting caters to the registration requirements of the foundation model era. It utilized the decoupling ability in SAM and construct a registration providing a good example in the foundation models for registration.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    -The experiment is not enough to prove the contribution of this work. The comparison is really limited. As a segmentation-based registration work, it should compare the methods, including the traditional registration method (e.g., Elaxtix, SYN), ROI free DL-based registration (e.g., XMorpher, VoxelMorph), ROI-based registration (e.g., DeepRS, PC-Reg). So that the comparison will be more complete and the upper-bound and lower-bound of this work will be clearly. -Especially, this work’s performance is still rely on the performance of SAM. Therefore, it is confusing why only use the SAM but not the variant of SAM in medical images, like the MedSAM, SAM-Med3D? How to design the prompt? Is this work able to be used in 2D? -A lot of details are lost in the paper. What’s the details of the new network parts in the framework? How to cope with the mis-segmentation of SAM?

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Do you have any additional comments regarding the paper’s reproducibility?

    No

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    See the weekness.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The innovation is good but the experiment is limited.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper present a region-based regisitration method to map the correspondence between two anatomical regions. The core idea of this paper is to first use SAM to obtain a set of unpaired masks, then select paired masks based on prototype similarity, and finally optimize the loss function through segmentation and regularization losses.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. This paper is well-written with a clear motivation.
    2. The experiment is sufficient and the results look good.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. The applicability of this method might be limited. This method is strongly rely on the SAM, and the method may fail if there are no clear boundaries in the image. In many cases, there are no obvious boundaries in the image e.g., brain MRI image.
    2. The loss function (eq. 2) may be further improved. In addition to the seg and reg loss, whether exploiting image similarity loss e.g., NCC can improve registration performance?
    3. The performance of the tradition registration method in Tab. 1 is too poor, which makes the results less convincing. Please give more implementation details during the rebuttal phase.
    4. Missing evaluation of deformation field smoothness, e.g. the percentage of voxels with a non-positive Jacobian determinant. Predicting a diffeomorphic deformation field is important in the task of medical image registration.
    5. The statistical analysis is necessary to demonstrate the effectiveness of the method.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Please address the issues mentioned in “Weaknesses of the paper”.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This is a well-written paper with a clear motivation. The idea seems interesting and the results is promising. However, there are some major issues for the authors to address during the rebuttal phase.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    This paper presents an interesting idea on finding sparse spatial correspondence between two images by matching multiple segmented ROIs. The ROIs are generated by the SAM model. The dense deformation field is derived by minimizing a functional approach with a smoothness penalty.

    The approach is validated on different organs and datasets.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    as above.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Weakness:

    Do you convert ROI correspondence to a dense pixel-level field to warp the images before computing the Dice and TRE? Please clarify this.

    It seems the matching steps can be sensitive to hyperparameters like the number of pairs and minimum similarity threshold (\epsilon). It would be interesting to show qualitatively the impact of mismatched pairs on the derived dense deformation field.

    Minor:

    A missing closing parenthesis in Eq. 2.

    “DDF” is not defined in the abstract before its usage.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    see “weaknesses” section.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Rationale for “Accept”: Using the SAM model for ROI proposal is a novel application in the context of image registration.

    Rationale for “Weak”: The idea of using ROI or patch-based spatial correspondence for image registration is not a surprising concept.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

N/A



Meta-Review

Meta-review not available, early accepted paper.



back to top