Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Intervertebral disc (IVD) degeneration poses demanding challenges for improved diagnosis and treatment personalization. Biomechanical simulations bridge the gap between phenotypes and functional mechanobiology. However, personalized IVD modelling is hindered by complex manual workflows to obtain meshes suitable for biomechanical analysis using clinical MR data. This study proposes Pixel2Mechanics: a novel pipeline for biomechanical finite element (FE) simulation of high-resolution IVD meshes out of low resolution clinical MRI. We use our geometrical deep learning framework incorporating cross-level feature fusion to generate meshes of the lumbar Annuli Fibrosis (AF) and Nuclei Pulposi (NP), from the L1-L2 to L4-L5 IVD. Further, we improve our framework by proposing a novel optimization method based on differentiable rendering. Next, a custom morphing algorithm based on the Bayesian Coherent Point Drift++ approach generates volumetric FE meshes from the surface meshes, preserving tissue topology through the whole cohort while capturing shape specificities. Daily load simulations on these FE model simulations were evaluated in three volumes within the IVD: the center of the NP and the two transition zones (posterior and anterior) on mechanical responses. These were compared with the results obtained with a manual segmentation procedure. This study delivers a fully automated pipeline performing patient-personalized simulations of L1-L2 to L4-L5 IVD spine levels from clinical MRIs. It facilitates functional modeling and further exploration of normal and pathological discs while minimizing manual intervention. These features position the pipeline as a promising candidate for future clinical integration. Our data & code will be made available at: http://www.pixel2mechanics.github.io

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/4115_paper.pdf

SharedIt Link: https://rdcu.be/dV5Ew

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72104-5_55

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/4115_supp.pdf

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Nat_Pixel2Mechanics_MICCAI2024,
        author = { Natarajan, Sai and Muñoz-Moya, Estefano and Ruiz Wills, Carlos and Piella, Gemma and Noailly, Jérôme and Humbert, Ludovic and González Ballester, Miguel A.},
        title = { { Pixel2Mechanics: Automated biomechanical simulations of high-resolution intervertebral discs from anisotropic MRIs } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15007},
        month = {October},
        page = {572 -- 582}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper proposes a combination of methods to generate simulation meshes of spinal discs from T2-w sagittal MRI. The claimed technical novelty is differentiable rendering and graph neural networks for shape reconstruction. Simulations have been performed and validated against manual meshes.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The method has been validated on the claimed downstream task of performing simulations, with comparisons with manually reconstructed meshes. The overall process of mesh generation is clearly described.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Technical innovation is relatively weak, considering the core components have all been previously developed - differentiable rendering, Laplacian and normal losses, GNN for template deformation, BCPD++, Abaqus, etc. I find it hard to believe this is the first application of differentiable rendering in a medical context, but even if that were the case, it seems like more of a new application than a methodological novelty. Result presentation is a bit unclear, especially starting Section 4.2 It’s unclear how the 3D ground-truth meshes were obtained. The only comparison with an automated method is with Voxel2Mesh.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    Enough details for reproducibility

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    What exactly is similarity % and morphology mm in Tables 2 and 3? What is Table 2 compared against? Were 3D ground-truth meshes obtained using sagittal slices? Was it interpolation between slices? Was additional data used to obtain the ground-truth meshes? Depending on how the ground-truth meshes were obtained, another reasonable baseline could have been segmentation of every slice and interpolation.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper is generally clear and combines multiple components to address a hard problem of generating good simulation-ready meshes. Simulations have been performed to demonstrate viability for downstream tasks. However, there are still question marks for the core aspects of the paper - how were the ground-truth meshes generated? What’s wrong with a simple segmentation and interpolation approach? Limited technical novelty as well.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • [Post rebuttal] Please justify your decision

    I appreciate the authors’ clarifications, especially with respect to ground-truth meshes and table labels. This info should be included in the manuscript.

    In response to – “our work makes a contribution by applying DR to perform 3D shape reconstruction from scans, which has not been done before” – unless it’s specifically talking about medical scans, I know this is a false statement. 3D shape reconstruction from 2D scans has been one of the main use cases for differentiable rendering from its inception. The use of silhouette and depth maps also seems to be common for various available implementations of differentiable rendering. I still believe the novelty here is more on the clinical application than method development.

    After reading the paper one more time following the rebuttal, I still have similar concerns regarding the lack of experiments (there’s only one comparison), limited novelty, and missing info. I maintain my original rating.



Review #2

  • Please describe the contribution of the paper

    The paper proposes a method to create a FEM biomechanical model from the intervertebral discs of the lumbar spine from a low resolution clinical MRI. First meshes of the discs are created from the MRI images and they are refined using differentiable rendering. Then, using an existing Bayesian Coherent Point Drift++ method, volumetric meshes - ready for FEM simulation which capture individual patient features - are created inside the surface meshes. By running simulations, authors show that they obtain results which are on pair of manually segmented discs. They highlight, that small morphological changes can have an impact on the simulation, and thus the diagnosis.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    • the paper is close to the clinical practice: use of low-resolution clinical MRIs, segmentation of clinically relevant structures (annulus fibrosus and nucleus pulposus), and assesment on how morphology changes affect the simulation

    • the differential rendering approach combining the binary mask and the depth map is sound, as well as the Laplacian and normal regularization losses. Improvement over Voxel2Mesh is clear with sup mat figure.

    • authors use openly available data (anonymized though) [2 and 3]

    • the paper is clearly written, the problem is well motivated and the results are relatively well presented

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    • the contribution with respect to [1,2,3] is somehow unclear - these previous papers from the authors are anonymized, and thus it is difficult to understand the improvement of the present paper with respect to them.

    • the simulation is performed on few subjects. For instance, only one patient for each test set is used for simulation. It would be good to explain why: time constraints? How does this affect the validation?

    • The interpretation of Table 3 is somehow difficult. (see detailed feedback)

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    Reproducibility will be hard without the code. There are many different parts in the pipeline which are not described in detail: image encoder, GCN, cross-level feature fusion…

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    • there is only one line refering to Table 3. Could authors add some explainations on the table interpretations? – In the Morphology part, are these heights compared to the manually obtained ones? – Are PTZ CPN and PTZ differences with the manually obtained? Authors claim in the discussion and conclusion that these values are very good and consistent with [3], but we do not have access to [3] …

    • The anonymization strategy is non-standard. A paper from 2023, such as [2], should be cited and described as work from colleagues. Three anonymized papers hinder the correct assessment of the current contribution with respect to these works.

    -authors mention a 3-fold, but it seems it should be 5. Is this a typo or a bug?

    • Table 1 bold values do not match best method.

    • [A] could be added to the references as it provides a morphological description of the variability in a small cohort of patients. [A] Zhong, Weiye, Sean J. Driscoll, Minfei Wu, Shaobai Wang, Zhan Liu, Thomas D. Cha, Kirkham B. Wood, and Guoan Li. “In vivo morphological features of human lumbar discs.” Medicine 93, no. 28 (2014): e333.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I’m in between weak accept and weak reject.

    On one hand the paper proposes:

    • an automatic pipeline to obtain very relevant information from low resolutions clinical MRIs.
    • The novel approach combinates several technics in a very sound manner.
    • The obtained results compare positively with manually annotated segmentations.

    On the other hand, several elements hinder the impact of the paper: -The fact that there are three anonymized papers creates a risk on the correct assessment of the current contribution with respect to these works. It would be good if the rebuttal provided the clear distinction of this work with respect to [1,2,3].

    • the no release of code will be a break to reproducibility

    Maybe the rebuttal can clarify these points.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Accept — should be accepted, independent of rebuttal (5)

  • [Post rebuttal] Please justify your decision

    The rebuttal clarified the issues relating to

    • the novelty wrt to 1,2,3
    • reproducibility, by promising to release the dataset and scripts, as well as a web-based solution I lean in favor of acceptance.



Review #3

  • Please describe the contribution of the paper

    The study introduces Pixel2Mechanics, a novel pipeline that transforms low-resolution clinical MRI into high-resolution finite element (FE) meshes for biomechanical simulation. Specifically for the lumbar spine’s Annuli Fibrosis and Nuclei Pulposi. The method proposes a new optimization method using differentiable rendering and a custom morphing algorithm that maintains tissue topology while adapting to individual shape variations. The authors simulate daily loads on the IVD, comparing its mechanical responses with those from traditional manual segmentation. The goal of the methods is to minimize manual effort.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1. The figures, visuals and rederings are of outstanding quality and support the contributions in this work.

    2. Another strength, in my opinion, is also the differentiation between the annulus fibrosus and nucleus pulposus in network creation. Overall, the FEM model itself appears very solid to me, especially with the (if I understand correctly) material parameters obtained from the MRI. However, the construction of the detailed FEM model seems to have occurred in another study - [3].

    3. Generating meshes directly from MRI and the associated irrelevance of segmentation can have advantages and disadvantages. Advantage: no Marching Cubes algorithm for generating surface meshes, which can introduce inaccuracies and require smoothing algorithms afterwards.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1. Conference fit. I am not sure if MICCAI is the right fit for this work. I think potentially are more orthopedic or bioengineering related venue may be a better fit.

    2. To generate the FEM meshes, a template FEM mesh is morphed into the shape of a surface mesh. Mesh morphing is often associated with poor mesh quality because the template meshes are deformed until they fit the shape of the surface (target) mesh. This often results in elements of low quality. The good thing is that hexahedra are used here, not tetrahedra. With tetrahedra, FEM solvers become unstable more frequently because the elements are more susceptible to distortions when solving the FEM equations. Accordingly, information on the mesh quality – both of the triangulated surface meshes and the volume meshes of the FEM – would have been interesting in addition to the comparison of the Hausdorff distances.

    3. Generating meshes directly from MRI and the associated irrelevance of segmentation can have advantages and disadvantages. Disadvantage: No vertebral body contact surfaces. Moreover, it is difficult to assess to what extent their mesh is better than what could potentially have been generated by the Marching Cubes algorithm, precisely because of the lack of information on mesh quality.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    -

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Please see my strengths and weaknesses above. I am not an expert on the topic.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    see above

  • Reviewer confidence

    Not confident (1)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • [Post rebuttal] Please justify your decision

    When reviewing the paper I was not sure if it is a good conference fit as well as I had a few other concerns which have not been addressed. Having read the other reviews I share a few concerns of R1. Therefore I will keep my rating in the realm of borderline.



Review #4

  • Please describe the contribution of the paper

    The main contribution of this study is the development of Pixel2Mechanics, a novel pipeline that enables biomechanical finite element simulation of high-resolution intervertebral disc (IVD) meshes from low-resolution clinical MRI data.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    – We propose Pixel2Mechanics, a deep learning-based pipeline to reconstruct AF and NP meshes of the L1-L2 to L4-L5 IVDs and perform FE simulations – We propose a novel differentiable rendering-based optimization to reconstruct AF and NP meshes from clinical MRIs using graph neural networks – The reconstructed AF and NP meshes are used to generate PP models for FE simulations to evaluate the variability of internal mechanical responses, including inter-personal variability.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    n/a

  • Please rate the clarity and organization of this paper

    Excellent

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    This is an excellent pipeline that enables biomechanical finite element simulation of high-resolution intervertebral disc (IVD) meshes from low-resolution clinical MRI data. The paper includes excellent quantitative and qualitative results that improve state-of-the-art.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Novel pipeline with excellent quantitative and qualitative results that improve state-of-the-art.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

We are grateful to the reviewers for taking the time to review our paper and providing constructive feedback. We have grouped the comments into four categories and later addressed some of the individual questions:

Ground truth mesh: To obtain a high-resolution volume, the sagittal and coronal sequences were fused using multi-planar fusion as described in [Castro-Mateos et. al, 2014]. This was done only to train and validate the model and is unavailable in clinical routine. The discs in the resulting high-resolution volume were annotated by clinical experts

Novelty: We appreciate the concern about the novelty of differentiable rendering (DR), but our work makes a contribution by applying DR to perform 3D shape reconstruction from scans, which has not been done before. This specific application, along with our novel hybrid rendering approach utilizing depth maps and silhouette images, has not been explored before in the medical domain. Additionally, we have shown that the reconstruction of meshes from rendered images can be beneficial for 3D reconstruction from sparse annotations.

Reproducibility: We acknowledge that reproducibility is an important aspect, and we will address this by sharing the dataset, the DL model, and FEM scripts upon acceptance of the manuscript. Moreover, a web-based cloud platform to perform simulations directly from MRIs will be made available.

Anonymization: We apologize if some statements were unclear due to the anonymity. We hope that the following explanations regarding certain aspects of our previous works will be useful for your evaluation.

R1Q1: Similarity % and morphology mm?: “Similarity %” refers to a similarity score based on Hausdorff distance (defined in our previous publication). We will include the definition in the caption of Tables 2 and 3. “Morphology mm” refers to the height of the disc. We will change it to “Height mm” to make it clearer. Table 2 compares the similarity score of our morphed models with those obtained by DL.

R1Q2: Segmentation and interpolation: We apologize for the lack of segmentation methods in the results section due to limited space, and our focus was on integrating a DL pipeline with FEM results for direct clinical practice. Segmentation approaches such as [Turella et. al, MICCAI 2021; Isensee et al, 2021], processed by marching cubes are available in [1], and we found that segmentation excels at achieving high Dice scores but struggles to capture unique geometric variations in each disc, which is crucial for our simulation requirements. R2Q1: Interpretation of Table 3: Table 3 shows the similarity score, disc heights, and the principal stresses in the three defined volumes (PTZ, CNP, and ATZ). These results correlate with those observed in our previous publication, in which there was no linear relationship between stresses and mid-height, as other publications suggested (Urquhart et al., 2014; Tavana et al., 2024), highlighting the need for a holistic analysis of the problem. This is mentioned in the discussion, where the reference to Table 3 will be included. R2Q2: PTZ, CPN, ATZ manually obtained?: The Transition Zone (TZ) in our FE mesh structure emerged from a need for computational stability. This TZ exists, as revealed by quantitative MRI, synchrotron imaging, cell phenotypes, and composition measurements through the IVD (Marchand and Ahmed, 1990; Bruehlmann et al., 2002). Defining this region resulted in a more realistic description of the IVD, unlike the abrupt change of material properties from the NP to the AF used in many FEMs. Nonetheless, the TZ is interesting, representing a volume of increased radial compression at the periphery of the inner IVD, where the nucleus pulposus material is pressed against the confining annulus material due to the lateral expansion of the nucleus under mechanical loads. This mechanical particularity coincides with the emergence of early signs of tissue disorganization in IVD (Smith et al., 2011), which is relevant.



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    Overall, apart from R1, the reviews are quite positive. Though this paper was listed under both “MICCAI Methodology” and “Clinical Translation,” I would personally view this as the latter, and perhaps the author should consider “Clinical Translation” specifically for similar future works. In short, all the technical details of the paper can be found elsewhere, but this exact proposed method, and of course the validation work showing that it is better than the baseline (Voxel2Mesh), is novel for this application; this was also R1’s main concern. The comment about how segmentation-based approaches “struggle to capture unique geometric variations in each disc” is interesting, and I would prefer a simple ablation clarifying this in the final version of the paper if possible (highlighted by R4-6-3 and to a certain extent R1-10-3 and R1-12-2). The question about the specifics generated by R4-6-2 was largely unanswered and should also be addressed.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    Overall, apart from R1, the reviews are quite positive. Though this paper was listed under both “MICCAI Methodology” and “Clinical Translation,” I would personally view this as the latter, and perhaps the author should consider “Clinical Translation” specifically for similar future works. In short, all the technical details of the paper can be found elsewhere, but this exact proposed method, and of course the validation work showing that it is better than the baseline (Voxel2Mesh), is novel for this application; this was also R1’s main concern. The comment about how segmentation-based approaches “struggle to capture unique geometric variations in each disc” is interesting, and I would prefer a simple ablation clarifying this in the final version of the paper if possible (highlighted by R4-6-3 and to a certain extent R1-10-3 and R1-12-2). The question about the specifics generated by R4-6-2 was largely unanswered and should also be addressed.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    This is a borderline paper for clinical application.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    This is a borderline paper for clinical application.



back to top