Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

The reconstruction of high-quality shape geometry is crucial for developing freehand 3D ultrasound imaging. However, the shape reconstruction of multi-view ultrasound data remains challenging due to the elevation distortion caused by thick transducer probes. In this paper, we present a novel learning-based framework RoCoSDF, which can effectively generate an implicit surface through continuous shape representations derived from row-column scanned datasets. In RoCoSDF, we encode the datasets from different views into the corresponding neural signed distance function (SDF) and then operate all SDFs in a normalized 3D space to restore the actual surface contour. Without requiring pre-training on large-scale ground truth shapes, our approach can synthesize a smooth and continuous signed distance field from multi-view SDFs to implicitly represent the actual geometry. Furthermore, two regularizers are introduced to facilitate shape refinement by constraining the SDF near the surface. The experiments on twelve shape datasets acquired by two ultrasound transducer probes validate that RoCoSDF can effectively reconstruct accurate geometric shapes from multi-view ultrasound data, which outperforms current reconstruction methods. Code is available at https://github.com/chenhbo/RoCoSDF.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2031_paper.pdf

SharedIt Link: https://rdcu.be/dY6jM

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72083-3_67

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/2031_supp.pdf

Link to the Code Repository

https://github.com/chenhbo/RoCoSDF

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Che_RoCoSDF_MICCAI2024,
        author = { Chen, Hongbo and Gao, Yuchong and Zhang, Shuhang and Wu, Jiangjie and Ma, Yuexin and Zheng, Rui},
        title = { { RoCoSDF: Row-Column Scanned Neural Signed Distance Fields for Freehand 3D Ultrasound Imaging Shape Reconstruction } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15004},
        month = {October},
        page = {721 -- 731}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper aims to reconstruct high-quality shape geometry for freehand 3D reconstruction, by generate an implicit surface through continuous shape representations. The proposed method used neural signed distance function with multi-view US scanning ( row-column scans), and introduce two regularizers to facilitate shape refinement by constraining the SDF near the surface.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    This paper used multi-view US scans rather only single direction scans, for freehand US reconstruction, using neural signed distance function. This enables a smooth and continuous signed distance field from multi-view SDFs to implicitly represent the actual geometry.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    This paper is not well-written. The description of method is not clear, for example, the definition of distance in Eq. (1), the motivation behind Eq. (2), and the definition of loss functions. The experiment results is not convincing: 1) the data set comes from phantom data, without using real world data set; 2) the statistic testing is missing.

  • Please rate the clarity and organization of this paper

    Poor

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    1. Why the surface of an object can be represented by fθ(.) = 0? Is this a definition/assumption? If so, why use this assumption/definition
    2. The last sentence in Page 3, “d(x, S) is the positive distance from x to surface S.”, how to compute the distance? Better to formulate this using equations.
    3. The loss function is not explained well. How does L_{scc} come out? What is L_{adl} with adversarial learning strategy? The definition is X’ is missing.
    4. “The distances are computed between the points randomly sampled from reconstructed mesh and the corresponding CAD models.” Why sample points to calculate the evaluation metrics, rather calculating distance over all available points?
    5. “We train frow, fcol and fopt for 1.0w * 10^4, iterations using the Adam optimizer.*”, what does “w” mean?
    6. What is the selection rule for the hyper-parameters, such as α_{nonmfd}, λ_{mfd}, etc.
    7. The data set are all from phantoms. What is the difference between phantoms data set and in vivo data set? Will there be any difference if the proposed method is implemented on in vivo real world data set?
    8. In table 1, is the results significantly different between UNSR and the proposed method? Better to do statistic testing and provide p value for this.
    9. The author claimed that they have solved the challenges of view-dependent and pixel connectivity, i suggest analyse and discuss this function of the proposed method, with results in experiment part.
    10. The data set used in not clear. How many scans are acquired using the 6 models? How many frames in a scan? What is the split ratio between train, validation, and test sets?
    11. In the Supplementary, what is Tracking Resolution in Table 1? Is this system error from NDI tracker? To my knowledge, the system error from NDI is less than 0.25 mm?
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Reject — should be rejected, independent of rebuttal (2)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper is confusing both with the method and experiment part. The pipeline is not clear. Some contents are missing, such as the motivation/derivation of equations/loss functions. The data set is not convincing, and the experimental results is not analysed using statistic testing.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • [Post rebuttal] Please justify your decision

    The authors address most of my questions while the workflow of the paper still has room to improve. Hope the authors could make the methodology part clearer once this paper got accepted.



Review #2

  • Please describe the contribution of the paper

    This paper describes a novel method for 3D shape reconstruction from 2D freehand ultrasound. The method is based on implicit modelling of the target surface using a signed distance field.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The novelty in this paper is that it can combine the data from two near-orthoganal scaning sweeps. Compared to a previous, similar, method that only uses a single scanning direction, the presented methods demonstrates convincing results.

    While the paper is brief, the method and the description are understandable.

    The method has been tested on synthetic 3D verterbra using two different US probes. The described evaluation strategy is sound and convincing.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    There are a few missing words but else the text is fluid and readable. When writing clear text inside an equation, please use latex \mbox{} or \text{} to make it more readable.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    It is an interesting paper with a somewhat novel method, convincing experiments and good performance.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    It is an interesting and relevant method built on previous published research.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Accept — should be accepted, independent of rebuttal (5)

  • [Post rebuttal] Please justify your decision

    I am satisfied with the rebuttal



Review #3

  • Please describe the contribution of the paper

    The authors propose a procedure to reconstruct 3D vertebra models scanned via ultrasound. The procedure is based on neural signed distance fields and combines row and column scans of the shape followed by a smoothing step. The authors demonstrate that their method is superior to focussing only on row or column scans in an in-vitro experiment.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The proposed method is theoretically sound and achieves good reconstruction performance in the experiments.
    • The proposed method could possibly have translational impact.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • The authors do not comment on plans to make their data publicly available.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    It would be valuable if the data was made public, if legally possible.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    • Introduction: Is there a reason why you “learn two SDFs for the row-scan and column-scan” and not one SDF interpolating between both?
    • Fig. 2: The image slices in segment (b) make it look the SDFs represent discrete slices. From my understanding they do not. Perhaps converting them to solid blocks would make it clearer.
    • 2.2 “Row-Column Neural SDFs Prediction”: In equation (2) the “x” for scalar-vector product might be misleading.
    • 2.2 “Row-Column Neural SDFs Prediction”: Does the last paragraph refer to the procedure of fitting the MLP or pre-processing? Please clarify.
    • 2.2 “Signed Distance Fields Fusion”: Please use “\mathrm{Intersection}” or “\text{Intersection}” in equation (4).
    • 2.2 “Signed Distance Fields Fusion”: Are there physiological or US-related reasons why an intersection between the row and column scan represents the most accurate shape? Why not an interpolation between the two?
    • 2.3 “Model Training and Loss Functions”: How is the cosine of two vectors in equation (6) defined? What is “L_sdf”?
    • 2.3 “Supervised Learning”: The index for “manifold” should be synchronised with the one for “non-manifold” in the previous section.
    • 3.1 “Data Acquisition and Preparation”: Since the phantoms are from 3D printing, does that mean ground truth is available for them? Please add a remark (even though you mention it later).
    • 3.1 “Data Acquisition and Preparation”: The authors do not disclose if they plan to make their data publicly available. This would be valuable.
    • 3.2 “Evaluation Metrics”: How does your method compare to the traditional orientation-based reconstruction? Please mention it here.
    • Conclusion: The neural SDF approach always introduces some form of implicit regularisation on the reconstructed shape (see Gropp et al. “Implicit Geometric Regularization for Learning Shapes”). It would be good to mention this here.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The authors solve the problem of shape reconstruction from ultrasound in an elegant way and the proposed method might be useful for many people.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Accept — should be accepted, independent of rebuttal (5)

  • [Post rebuttal] Please justify your decision

    The authors answered all open questions.



Author Feedback

We greatly appreciate positive feedback from reviewers regarding the novelty (R1 R3 R4), “convincing experiments” (R1), “elegant way” (R3), clear descriptions (R1 R3), and all the constructive comments (R1 R3 R4). We summarize comments and our responses below.

R1,R3: We thank R1&R3 for their careful review and encouraging suggestions. The minor concerns will be carefully revised in final paper.

R1,R3,R4: Reproducibility. We will release the code for reproducibility.

R3: 1) Interpolation. In Fig.1, different US scan views introduce varying distortions. We train a separate SDF for each view to learn its unique shape, and fuse them using set operation. For a naive 2D example, the column-scan elongates a rectangle’s height, whereas row-scan elongates the width but maintains the original height. The union of these scans reflects all the distortions: including elongated height and width, while the intersection keeps the original height and width. The union and interpolation can be used for future shape completion[18]. 2) Eq.2. We will use “.” instead of “x” to void misleading. 3) The last paragraph refers to fitting the MLP.

R3,R4: Loss. In addition to newly introduced regularizers, Lscc and Ladl are two constraints in UNSR[6]: sign consistency constraint and on-surface adversarial learning, to improve Lsdf in [17]. Lsdf is an L2-distance loss to optimize the projected x′ in Eq.2 reaching its nearest x^hat in Eq.6. The released code can help follow the details.

R4: Method Clarifications 1) Eq.1. SDF is a concept in computer vision to describe the object’s shape and geometry. In Euclidean space, for a 3D point x and a watertight object surface S, d(x,S) is the orthogonal distance from x to S. The sign is positive (negative) when x reaches the surface from outside (inside) of object.

2) LevelSet. The value of d(x,S) decreases to 0 when x is on the surface. Thus, the surface is defined by zero-level-set of SDF, formulated by all points where SDF(.)=0.

3) Neural SDF. The use of MLP (fθ) to represent SDF has gained great attention [11-18]. The zero-level-set of neural SDF is denoted as fθ(.)=0. Here, we build a novel framework to address the challenges of view-dependent and discrete pixel connectivity in multi-view US reconstruction by directly operating shapes in a continuous neural SDF field.

4) Eq.2. This equation, defined following Eq.1, describes how a point x can reach its nearest surface point x’ along or against gradient based on the signed distance. Here, fθ(x) is the predicted signed distance, and ∇fθ(x) is the gradient derived by back-propagation. Eq.2 is part of Eq.6 for self-supervised learning of SDF.

5) Hyperparameters. We empirically set them based on the findings in [21].

R4: Data 1) Phantom. For one model scanned by one UT, each row-column scan corresponds to 1 shape and 2 scans. Two UTs obtain 12 shapes and 24 scans from 6 models. An average of 540±159 frames from UT1 and 686±244 frames from UT2 are collected. We learn the SDF directly from each individual input without splitting dataset for implicit neural representation (INR)[6,17].

2) In-vivo. In this paper, our primary goal is building a framework to accurately restore the shape geometry from multi-view US scans. To validate our method, we use the phantom data since it has clean images and ground-truth CAD model. As noted in the Introduction, for hard tissue imaging, US signal struggles to penetrate target surface and is susceptible to interference from soft tissue in in-vivo data, which hinders the evaluations. Thus, the phantom or simulated data becomes a practical choice to validate INR-US in recent studies [6,9,10]. In future work, other modules can be added to the framework to consider interference, e.g., denoising module.

R4: T-test. Statistical significance is established with p-value<0.01 against UNSR across all metrics.

R4: Tracking resolution. We report it using accuracy-position-1.4mm-orientation-0.5^o from specifications of NDI-driveBAY.



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    all reviewers agree on acceptance now. The authors did a great job with the rebuttal!

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    all reviewers agree on acceptance now. The authors did a great job with the rebuttal!



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    Rebuttal addressed most of the concerns. Paper can be accepted.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    Rebuttal addressed most of the concerns. Paper can be accepted.



back to top