List of Papers Browse by Subject Areas Author List
Abstract
Knowing the precise location of the bones inside the human body is key in several medical tasks, such as patient placement inside an imaging device or surgical navigation inside a patient. Our goal is to predict the bone locations using only an external 3D body surface observation. Existing approaches either validate their predictions on 2D data (X-rays) or with pseudo-ground truth computed from motion capture using biomechanical models. Thus, methods either suffer from a 3D-2D projection ambiguity or directly lack validation on clinical imaging data. In this work, we start with a dataset of segmented skin and long bones obtained from 3D full body MRI images that we refine into individual bone segmentations. To learn the skin to bones correlations, one needs to register the paired data. Few anatomical models allow to register a skeleton and the skin simultaneously. One such method, SKEL, has a skin and skeleton that is jointly rigged with the same pose parameters. However, it lacks the flexibility to adjust the bone locations inside its skin. To address this, we extend SKEL into SKEL-J to allow its bones to fit the segmented bones while its skin fits the segmented skin. These precise fits allow us to train SKEL-J to more accurately infer the anatomical joint locations from the skin surface. Our qualitative and quantitative results show how our bone location predictions are more accurate than all existing approaches. To foster future research, we make available for research purposes the individual bone segmentations, the fitted SKEL-J models as well as the new inference methods at https://3dbones.is.tue.mpg.de.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0588_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0588_supp.pdf
Link to the Code Repository
Link to the Dataset(s)
BibTex
@InProceedings{Dak_On_MICCAI2024,
author = { Dakri, Abdelmouttaleb and Arora, Vaibhav and Challier, Léo and Keller, Marilyn and Black, Michael J. and Pujades, Sergi},
title = { { On predicting 3D bone locations inside the human body } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15003},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper tackles the problem of predicting bone locations from skin observations. It builds on the ANON dataset and augments the bone segmentation data with more fine-grained bone segments, which refines the original dataset to be more suitable for this task. It also includes additional regressors on top of the SKEL method to improve bone registration.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper is clearly written. Existing issues with bone localization are identified and the authors propose straightforward and effective means to tackle these gaps from the dataset and methodology perspectives.
- The improvements in bone registration accuracy with respect to the maximum distance discrepancy is quite substantial over existing methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The motivation for the problem of accurately localizing bones within the human body from surface observations is not entirely clear to me. It would seem that the bone localization accuracy offered by existing methods would be sufficient for most intent and purposes. If the medical operation requires an utmost level of accuracy for body placement, it would most likely require scanning devices. Furthermore, a common use case requiring accurate body placement concerns broken / fractured / deformed bones, and I would think that the proposed method cannot handle such cases well.
- The methods in this paper are incremental and novelty is lacking. In particular, the main technical difference with SKEL is the inclusion of additional regressors.
- The mean bone location prediction errors are quite marginal improvements over existing methods and the differences with SKEL are hard to discern in the visualization (Figure 5).
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Please refer to the weaknesses above. I would suggest the authors to provide a discussion of the motivations vis-a-vis my first point.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the paper is clearly written, I do find that the improvements are too marginal and novelty is lacking.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
The authors have adequately addressed my concern over the motivation of this work. While I am not entirely convinced regarding the novelty of the approach, I would lean towards acceptance post-rebuttal.
Review #2
- Please describe the contribution of the paper
The paper introduces a new approach for locating bones within the body, featuring several key contributions. These include an enriched dataset comprised of segmented individual bones paired with skin observations, the development of SKEL+ which adds additional degrees of freedom to the regressed joint locations, and enhancements in accuracy compared to existing SOTA methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The primary strength of this paper lies in its new approach for accurately locating bones within the body. The authors present an enriched dataset comprised of segmented individual bones paired with skin observations. Furthermore, they effectively address the limitations of their method and offer valuable insights into potential future research directions.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
In the results section, the comparison with the state-of-the-art methods is appreciated; however, the absence of statistical analysis compromises the reliability of these comparisons. Additionally, the boxplot in Figure 2 is not properly illustrated. Furthermore, it is unclear how the average distance of SKEL+ in Table 1 is higher than the maximum value reported.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
While the authors provide explanations for their data, the paper would benefit from more detailed information on the diversity and age range of the patient data, as well as the disease manifestations, particularly with respect to health status and imaging device characteristics. Although performance metrics are reported with averages and standard deviations, a more robust statistical analysis would further strengthen the findings. Additionally, I am concerned that there could be inaccuracies in the reported results and the boxplot shown in Figure 2 and Table 1.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper lacks sufficient statistical detail in the results section, notably the absence of p-values, which are essential for assessing the statistical significance of the findings. Additionally, the paper would benefit greatly from a more thorough elaboration on the datasets used. Details such as the diversity and age range of the patient data, disease manifestations, and characteristics of the imaging devices are essential for understanding the generalizability of the results. Moreover, my concerns about potential inaccuracies in Figure 2 and Table 1 indicate that there might be errors affecting the clarity and accuracy of the data presentation and results.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Accept — should be accepted, independent of rebuttal (5)
- [Post rebuttal] Please justify your decision
The authors provided additional details about the datasets and included p-values from the Wilcoxon statistical analysis. They also addressed the correction of typo errors.
Review #3
- Please describe the contribution of the paper
The study introduces a novel approach for predicting the intra-body positions of bones based on observations of the skin. Building upon the existing SKEL model, the authors enhance its capabilities by introducing an additional degree of freedom. This enhancement enables the model to incorporate offsets in the regressed joint locations, facilitating adjustments to bone shapes and joint positions without altering skin morphology. Furthermore, the authors extend an already existing dataset by incorporating multi-labeled bones, enabling individualized analysis of each bone. This advancement allows for more precise and comprehensive investigations into bone localization and alignment within the body.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main strength of the paper lie in its innovative approach to a contemporary research problem, particularly in the context of the increasing diffusion of 3D cameras and virtual reality technologies. One notable aspect is the novel formulation of a new degree of freedom and the corresponding regressor, which enables the representation of various possible skeletons for a single external skin. This approach enhances the realism of the models, making them more closely resemble actual human anatomy.
Moreover, the paper’s validation of this method using real 3D medical data underscores its relevance in clinical settings.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The introduction provides a comprehensive overview of existing methods, but it overlooks recent contributions, such as the work by [Wenninger, S., et al. “TAILORME: Self-Supervised Learning of an Anatomically Constrained Volumetric Human Shape Model.” COMPUTER GRAPHICS forum. Vol. 43. No. 2. 2024.]. Including this reference would enhance the contextualization of the study within the current literature.
While the authors claim to provide a dataset of segmented individual bones, their contribution in this part of the work lies in the differentiation of each bone within an already existing skeleton segmentation dataset. Moreover, this differentiation seems to build upon existing methods, cited by the author, and lack of validation in the result. Thus, even if I understand and value the importance of having such improved labelling, I wouldn’t list it as a proper contribution of this specific paper.
The presentation of results suffers from clarity issues, particularly in tables and figures. Ambiguities in these visual aids restrict the understanding of the method’s accuracy and efficacy. Revising these elements for clarity and coherence would significantly enhance the paper’s readability and impact.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
From what I can understand in the introduction the author will release their method for research purposes. However, other information is not provided within the manuscript.
The paper introduces and references a dataset labelled as ANON [21], which is anonymized. This complicates understanding, particularly when it later appears to be a method for predicting bone location and not only a dataset. The lack of clarity and reference to this dataset/method hinders reproducibility and further analysis.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The paper is overall well written but there is room form improvement and clarifications.
The boxplot in Figure 2 lacks clarity, as it appears that values such as 5mm and 6mm are repeated multiple times. Clarifying the labeling and presentation of data points would enhance the readability and interpretation of the figure.
In Figure 3, the ANON prediction appears to have missing data, particularly in the chest region. Addressing this discrepancy and ensuring the completeness of the visualization would improve the integrity of the results presentation.
Table 1 requires clarification, particularly regarding the discrepancy between the mean distance value and the maximum value in the SKEL+ results (the mean distance value appears to be higher than the maximum from what I understand). Providing explanations or additional context for these discrepancies would enhance the understanding of the comparative results.
Section 2.2 appears brief and somewhat connected to the preceding paragraph. Expanding this section or revising its structure for clarity and coherence would aid readers in following the flow of the paper.
Equation 3 introduces the lambda weight without explanation of how it is selected. Providing details on the methodology or rationale behind the selection of lambda would improve the transparency and reproducibility of the study.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The recommendation for acceptance is primarily influenced by the commendable effort the authors have devoted to tackling a complex and intriguing research area, as well as the clear presentation of their contribution in relation to the existing SKEL method. However, a significant drawback lies in the lack of clarity in presenting the results, which impedes a clear verification and understanding.
It’s evident that the authors have invested substantial effort and expertise in advancing knowledge within their field, especially in refining and extending the SKEL method. Their contribution holds promise for addressing important challenges in the domain, indicating the potential significance of their work for further research and practical applications.
However, the presentation of results falls short in terms of clarity. Ambiguities and inconsistencies in figures, tables, and equations hinder the ability to discern the authors’ findings accurately.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Accept — should be accepted, independent of rebuttal (5)
- [Post rebuttal] Please justify your decision
The authors addressed the issues raised in the review.
Author Feedback
We thank all reviewers for their constructive feedback. We are happy they appreciated the novelty and results of our approach (R4,R7) and the paper clarity (R3,R4,R7). We next address their comments.
Motivation While R4 is enthusiastic and confident about our “innovative approach to a contemporary research problem”, R3 requires further clarification. We note that even though “R3: medical operation requires an utmost level of accuracy for body placement, (so) it would most likely require scanning devices” our approach would increase the “R4: diffusion of 3D cameras and virtual reality technologies” in the medical field, and, as such, advance research in these technologies. We will improve the introduction by adding this motivation.
Novelty While R3 claims a lack of novelty “the main technical difference with SKEL is the inclusion of additional regressors”, this statement is refuted by R4 observation “the novel formulation of a new degree of freedom and the corresponding regressor, which enables the representation of various possible skeletons for a single external skin” and R7 “featuring several key contributions: … segmented individual bones (…) development of SKEL+ with adds additional degrees of freedom (…) and enhancements in accuracy (…)”.
Are multi-label bones a contribution? Weather the multi-label bone segmentation masks are a considered a contribution (R7) or not (R3,R4), these segmentations are key in three aspects:
- their fine grained semantics allow SKEL+ to go beyond the binary bone prediction of ANON and predict anatomically precise structures to create an articulated digital twin;
- they provide the means to have a deeper insight on the results (Fig 4 ablation) allowing to quantify which methods perform better in which body part;
- although not done in the current paper, future work could evaluate the precision of the joint locations (interfaces between the bones groups), which are relevant, for instance, in biomechanics; We will discuss these items in the introduction.
Results We thank R4&R7 for raising the inconsistency in the max values of Tab1 and the wrong scale of Fig2.
- Indeed Tab1 had two typos: SKEL+ max values are 21.69 (females) and 17.73 (males). All other values are correct.
- We will fix Fig 2 scale: the distances between the registrations and the GT point clouds range from 3.5mm to 7mm, median of 4.3mm, and mean of 4.5mm.
- R7 requested statistical tests. We computed the Wilcoxon test (suitable for small sample size <30) and report p-values (male, female) for the comparison of SKEL+ to SOTA:
- Tab1: ANON (0.02, 5e-08), OSSO (0.009, 0.47), SKEL (3e-5, 5e-8). Only SKEL+ and OSSO are not statistically different (>0.05) for females.
- Fig4 (ablation) the only paired differences with SKEL+ which are not statistically different are: (OSSO-Pelvis-males&females) and (SKEL-Tibia-fibula-females). All other p-values are <=0.01. We will add these p-values to clarify the paper claims stating that SKEL+ is an overall better model for 3D bone prediction.
Dataset characteristics (R7) This aggregated information will be added to the revised paper:
- age range (23-65) with balanced 4 groups: A: <35 y; B: 35–44 y; C: 45–54 y, D: >54 y;
- three BMI groups: normal weight (<=25), overweight (25–29.9), and obese (>=30);
- Diversity: white adults from one region of one country;
- Disease: all subjects were considered healthy (physical examination and routine laboratory tests);
- Device: 1.5 T Magnetom263 Sonata, Siemens, following [25].
Reproducibility (R4) As mentioned in abstract and intro, we will release: the multi-part segmentation dataset, the SKEL+ fits and the inference code (with regressors). In addition, to clarify the ambiguity around the lambda weight in Eq3, we will release the fitting code. The lambda value (=0.2) was empirically set to balance the skin and bone losses using the training set.
R4
- thanks, we will cite TAILORME.
- Fig 3&5 ok: ANON only predicts long bones; no chest or skull.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The paper introduces a novel approach to predicting 3D bone locations using additional regressors and enriched datasets, significantly improving bone registration accuracy. The authors address key reviewer concerns regarding motivation, novelty, and result clarity, enhancing the manuscript with statistical tests and corrections. The method’s validation with real 3D medical data and the commitment to releasing code and datasets ensure reproducibility. Given the substantial contributions and effective rebuttal, I recommend accepting this paper for its potential impact and advancements in the field.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The paper introduces a novel approach to predicting 3D bone locations using additional regressors and enriched datasets, significantly improving bone registration accuracy. The authors address key reviewer concerns regarding motivation, novelty, and result clarity, enhancing the manuscript with statistical tests and corrections. The method’s validation with real 3D medical data and the commitment to releasing code and datasets ensure reproducibility. Given the substantial contributions and effective rebuttal, I recommend accepting this paper for its potential impact and advancements in the field.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The reviews have converged to accept accept weak accept after rebuttal.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The reviews have converged to accept accept weak accept after rebuttal.