List of Papers Browse by Subject Areas Author List
Abstract
We introduce a novel unsupervised deep learning framework for constructing statistical shape models (SSMs). Although unsupervised learning-based 3D shape matching methods have made a major leap forward in recent years, the correspondence quality of existing methods does not meet the demanding requirements necessary for the construction of SSMs of complex anatomical structures. We address this shortcoming by proposing a novel deformation coherency loss to effectively enforce smooth and high-quality correspondences during neural network training. We demonstrate that our framework outperforms existing methods in creating high-quality SSMs by conducting extensive experiments on five challenging datasets with varying anatomical complexities. Our proposed method sets the new state of the art in unsupervised SSM learning, offering a universal solution that is both flexible and reliable. Our source code is publicly available at https://github.com/NafieAmrani/FUSS.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1005_paper.pdf
SharedIt Link: https://rdcu.be/dV57Y
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72120-5_3
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/1005_supp.pdf
Link to the Code Repository
https://github.com/NafieAmrani/FUSS
Link to the Dataset(s)
http://medicaldecathlon.com/
https://luna16.grand-challenge.org/Data/
BibTex
@InProceedings{El_AUniversal_MICCAI2024,
author = { El Amrani, Nafie and Cao, Dongliang and Bernard, Florian},
title = { { A Universal and Flexible Framework for Unsupervised Statistical Shape Model Learning } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15011},
month = {October},
page = {26 -- 36}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper adapts/extends the spectral meets spatial (SmS) approach for shape matching, and applies it to the task of creating anatomical SSMs. The paper introduces a novel regularization strategy called deformation coherency regularization. This removes the need for test-time post-processing in the SmS approach. The approach is evaluated on 5 publicly available anatomical datasets. The proposed method is compared against SmS, S3M and FlowSSM in terms of generalization, specificity, and compactness. The results indicate that the proposed approach produces improved SSMs compared to the other methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
SmS is a state-of-the-art approach for estimating surface-based shape correspondence using deep functional mapping methods. SmS was evaluated for the task of constructing anatomical SSMs (using Luna16), but this was not a focus of the original publication. Further evaluation of this approach for the task of anatomical SSM creation using additional datasets is valuable. Furthermore, the evaluation in this paper is more detailed compared to the evaluation provided in the SmS paper.
The SmS approach uses both spectral and spatial regularization. The spatial loss is described as “encouraging pose-dominant deformations” and “shape-based deformations are not well modeled”. For that reason, the SmS approach can also include a test-time adaption to better capture shape-dominant deformations. This test-time adaptatation optimizes for a shape-dominant deformation and considers the Chamfer distance as well as the Dirichlet energy. In comparison, this paper incorporates these terms (and an edge term) into the spatial regularization directly. This obviates the need to perform a test-time adaptation. This may improve computational efficiency and make the approach more suitable for anatomical datasets that typically do not include pose-like deformations (often associated with bodily articulation as seen in datasets like FAUST). Therefore, the approach is more focused on resolving shape deformations which is the important aspect of anatomical SSM creation. The results indicate that the SSMs produced using the proposed approach are superior to the other comparison approaches. The evaluation metrics are suitable, and detailed results are provided for each dataset, using different #s of PCs.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Methodological novelty is present but relatively limited. The modifications from the SmS approach amount to a different spatial regularization that incorporates two terms (Chamfer and Dirichlet), that were used in the SmS test-time adaptation, plus one additional term (Edge).
It is difficult to evaluate if the reformulated regularization loss is superior, or if differences in regularization weight are driving the improved results. Some of the results in the SmS comparison are counter-intuitive.
I have the following questions after reading the paper. Presumably they could be answered by reviewing their codebase when it is released, but I believe the information should be present in the paper.
Does the deformation coherency loss completely replace the spatial regularization loss used in SmS? Or is it an additional term that is added to the spatial regularization used in SmS?
Is the spectral regularization exactly the same as that used in SmS? Are there any other differences (hyper-parameters, regularization weights etc…) that are different between SmS and their approach? Or is it exactly the same aside from the spatial regularization.
Does the version of SMS that was used for the evaluation comparison include the test time adaption stage? (The evaluation would ideally include SmS both with/without this stage).
What is the benefit of not needing to run the test time adaptation? Does it significantly reduce computational cost? By how much?
How were the loss/regularization weights (lambda variables) chosen?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
Several of the questions that are mentioned in the earlier response on the paper weaknesses are related to reproducibility.
It should be clear within the paper exactly how their spatial regularization differs from that of SmS (and if there are any other differences from SmS).
It should be clear within the paper whether or not the SmS test-time adaptation was used in their evaluation.
It should be clear within the paper how the loss/regularization weights (lambda variables) were chosen for SmS and the proposed approach.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
I find it interesting/strange that the SmS approach produces high frequency artifacts in Fig 3 (particularly visible in the spleen). Given the “as rigid as possible” regularization, I would expect the results to be overly smooth if anything. The results in the SmS publication seem to retain the shape of the source/template (motivating the need for a test-time adaptation to capture shape differences). What exactly about the SmS regularization compared to the proposed regularization could lead to this type of result? I believe this is important to discuss because the regularization approach is the core methodological novelty.
I would presume that the magnitude/weight of spatial regularization will have a large impact on the results. This was not discussed or included in experiments. It is difficult to evaluate if the reformulated regularization loss is superior, or if differences in regularization weight are driving the improved results. I would recommend including this aspect in future work/experiments.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The application of deep functional mapping approaches to the problem of SSM creation is interesting and evaluations of these approaches that are more focused on anatomical data are valuable. The characteristics of data typically used to evaluate these approaches can be quite different (e.g. full body scans with pose variation). The proposed method specializes/adapts the SmS approach for anatomical data in a way that is sensible, focuses on shape differences rather than pose differences, and appears to produce better results.
Methodological novelty is somewhat limited. Further details should be provided in the paper about their method and comparisons.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This work introduces a new deformation coherency loss for an existing framework to build statistical shape models. The loss encourages smooth deformations between shapes by enforcing vertices proximity, small Dirichlet energy, and small shape differences during optimisation. Results on a variety of medical shapes show excellent balance between generalisability and specificity.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- evaluation is comprehensive (using hundreds of shapes from different organs). Results are excellent over all organs with low generalisation and specificity errors outperforming existing methods. It demonstrates that the proposed deformation loss if effective in adapting (7) to work on medical data.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
I think the work should have been compared against 2023 published Mesh2SSM (3) as the code is available and Mesh2SSM showed some benefits over FlowSSM which is used as one of the benchmarks in this work.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
Some implementation details are provided in the supplementary material which I think should be added to the main paper if it is accepted.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
I think the paper is well written and provides sufficient detail of the methodology. An existing SSM framework has been adapted for medical imaging use cases with a novel deformation loss which resulted in excellent performance on challenging datasets and on average outperforming existing state of the art methods. As little details are provided about the shape matching module and deformation trajectory computation, it would be good to reference (7) for clarity. I think there should also be some discussion and comparison to the benchmark methods about computational complexity. Overall, I think this is a relevant approach to improve statistical shape models which should have an impact on further work in the field as it is performing well across benchmarks with no obvious outliers.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method is well evaluated and compared against the state of the art and benchmarks show a very good balance of metrics outperforming existing methods.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper proposes a new deformation coherency loss, which combines the following three loss functions: 1) difference between consecutive shapes based on Chamfer distance (Eq.(1)),
2) Dirichlet energy to promote smooth deformations [18] (Eq.(2)), et
3) proximity of neighboring verticies to promote locally area-preserving deformations (Eq.(3)), for unsupervised Statistical Shape Model (SSM) learning. The method builds upon the state-of-the-art Spectral meets Spatial (SmS) shape matching framework [7], which is designed to predict point-wise correspondences and to interpolate between 3D shapes, in particular for handling articulated objects (e.g. humans or animals), by using an As-Rigid-As-Possible (ARAP) deformation energy [27]. However, this energy is not well-suited to model anatomical structures, so that the authors replace it with the above deformation coherency loss for constructing SSMs of various anatomical structures. - Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The proposed method was evaluated through comparative analysis with existing methods, such as S3M [3], FlowSSM [16] and SmS [7]. The authors demonstrate that their method outperforms the others with five anatomical datasets [25,26], with the pre-processing of converting segmented CT/MRI images to triangular meshes by using ShapeWorks [8].
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
This pre-processing part is not clearly explained. So it is not clear how to obtain triangular meshes from 3D images with a fixed number of verticies (“around 2000” in this article). How about the quality of generated meshes? Is the number of verticies reasonable for anatomical complex structure? If we change the number of verticies, can we obtain similar experimental results? Besides, the first Spectral Shape Matching module computes a point-wise map between two shapes, namely between the two sets of vertices. As this point-wise map serves as an input for the second (and main) Shape Deformation module, the quality of this point-wise map must be very important. However, it is not clearly written in the article how the point-wise map is generated. Moreover, how about the robustness of the Shape Deformation module with respect to noisy point-wise map input?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
It seems necessary to provide input data (triangular meshes).
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The quality of the paper will be greatly improved if authors address the following comments on the experiment. What is a role of each loss function? The third loss function is necessary? If we use only the first two loss functions, the results will be different? What is the computation time compared with the other methods?
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposal and experimental validation of the new deformation coherency loss was judged to be a sufficient contribution.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
N/A
Meta-Review
Meta-review not available, early accepted paper.