Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Ultrasound computed tomography (USCT) is a promising technique that achieves superior medical imaging reconstruction resolution by fully leveraging waveform information, outperforming conventional ultrasound methods. Despite its advantages, high-quality USCT reconstruction relies on extensive data acquisition by a large number of transducers, leading to increased costs, computational demands, extended patient scanning times, and manufacturing complexities. To mitigate these issues, we propose a new USCT method called APS-USCT, which facilitates imaging with sparse data, substantially reducing dependence on high-cost dense data acquisition. Our APS-USCT method consists of two primary components: APS-wave and APS-FWI. The APS-wave component, an encoder-decoder system, preprocesses the waveform data, converting sparse data into dense waveforms to augment sample density prior to reconstruction. The APS-FWI component, utilizing the InversionNet, directly reconstructs the speed of sound (SOS) from the ultrasound waveform data. We further improve the model’s performance by incorporating Squeeze-and-Excitation (SE) Blocks and source encoding techniques. Testing our method on a breast cancer dataset yielded promising results. It demonstrated outstanding performance with an average Structural Similarity Index (SSIM) of 0.8431. Notably, over 82% of samples achieved an SSIM above 0.8, with nearly 61% exceeding 0.85, highlighting the significant potential of our approach in improving USCT image reconstruction by efficiently utilizing sparse data.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1595_paper.pdf

SharedIt Link: https://rdcu.be/dV5BE

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72104-5_10

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/1595_supp.pdf

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{She_APSUSCT_MICCAI2024,
        author = { Sheng, Yi and Wang, Hanchen and Liu, Yipei and Yang, Junhuan and Jiang, Weiwen and Lin, Youzuo and Yang, Lei},
        title = { { APS-USCT: Ultrasound Computed Tomography on Sparse Data via AI-Physic Synergy } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15007},
        month = {October},
        page = {99 -- 108}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper proposes a method to perform ultrasound computed tomography with sparse measurements by leveraging deep and physics models.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The proposed method is well-motivated and outperforms other approaches
    • Experimental results highlight the potential of the proposed method to reduce data acquisition cost without significant performance degradation
    • Extensive ablation results
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • Although experiments are extensive, they only correspond to one, private dataset. Dataset release is not discussed.
    • Authors present the paper as AI-Physics synergy, but they are simply utilizing physics to (1) generate supervision for training (“APS-physics”) and (2) inform reconstruction (“APS-FWI”). It is clear how physics is enabling AI, but it is unclear how this creates a synergy.
    • Writing quality should be improved. Some examples: (1) “AI-physic” most likely would be AI-physics; (2) in the contributions the authors mention “the” breast reconstruction dataset, which has not been presented yet, (3) and in the method section A the authors refer to “the underline APS-physics”, which is unclear.
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    Dataset access in unclear. From the description it seems to be a private dataset, which hinders reproducibility of the results presented in the paper.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Organization of the paper is good, but writing should be improved. The method makes sense and it is well-motivated, it is an interesting case of how physics can enable AI. However, evaluating the method in a single, private dataset significantly hinders its strength.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper is interesting, experiments are detailed but they are limited to a private dataset. I’m willing to increase the score if the method is evaluated in a public dataset and the writing quality is improved.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • [Post rebuttal] Please justify your decision

    I decided to maintain my score because the authors’ response did not convincingly address my concerns in terms of the dataset availability and reproducibility of the results. In particular their response is vague, replying that “a part” of the phantoms are publicly available. In a minor note, I also consider that the evidence is insufficient to present this method as a “synergy” between physics and AI (e.g. no results of the impact of this method in downstream physics tasks are presented), and therefore I would suggest to use the most common term “physics-informed AI”.



Review #2

  • Please describe the contribution of the paper

    This work developed a deep learning network named AI-physic synergy (APS) for ultrasound computed tomography (USCT) reconstruction. The network is capable of automatically reconstruct speed of sound images by using sparse signals in terms of both the number of emitters and the number of receivers. The framework was validated in simulation using 2D breast images and outperforms the state-of-the-art.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper is overall well-structured with good clarity. The novelty of the work is in its methodology, where a new deep learning framework was developed to reconstruct sparse USCT signal aiming to reduce the hardware dependency and to improve the imaging speed. The developed method outperforms the state-of-the-art.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    This work was validated on 2D breast images in simulation. There is a lack of discussion on its performance in realistic scenarios and its clinical feasibility. The waveform simulation was not sufficiently described. The reconstruction performance, hardware cost, and the image acquisition time were not compared with conventional physics-based model such as full-waveform inversion, so the applicability and the performance of the deep learning model is hard to tell compared with conventional methods.

  • Please rate the clarity and organization of this paper

    Excellent

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    This paper is overall well structured with good clarity. Several points should be addressed before further consideration.

    Major:

    1. This study was developed using numerical breast phantoms for 2D USCT image reconstruction. There is a lack of description on how the waveform were simulated. Specifically, what parameters were considered in simulation? Was the heterogeneity only in the speed of sound, or also in tissue density, acoustic attenuation, nonlinearity, etc.? Was scattering considered?

    2. Following 1, why do some of the USCT reconstruction methods (e.g., Neareast+InversionNet) perform very differently on dense breast? From a physics perspective, it is understandable for mammograms because of the tissue overlap over a summed projection, but for USCT using FWI, all the information along the projection path is recorded in the acquired time series. In fact, it seems that dense breasts have lower speed of sound variation, so it should be less challenging. Why is this performance especially different for Neareast+InversionNet, where Neareast+InversionNet did a great job on the normal breast, but it reconstructs the dense breast even into a very different size?

    3. There is a lack of discussion on the limitation or foreseen challenges of the adoption of the proposed method in clinical applications. For instance, how different is the simulation data from actual data? What are the challenges in acquiring ground truth label in vivo? How much data is required to guarantee the performance, etc.

    4. How does the sparse reconstruction look like using physics-based model such as full waveform inversion? There is a lack of comparison between physics based model and deep learning models in terms of image quality, hardware cost, and image acquisition time. So even though the proposed method outperforms the state-of-the-art deep learning models, if it outperforms physics-based models is unknown.

    Moderate:

    1. Was any noise added during data generation? For instance, a certain amount of Gaussian noise mimicking electronic noise on ultrasound systems.

    2. In Fig.4, the side lobes in the label waveform mainly span laterally, but why do they look like vertical stripes in the axial direction in the APS-wave image?

    Minor:

    1. Were all figures plotted in the same speed of sound range in Fig. 3? A color bar indicating the speed of sound range should be given. It is better to add a scale to indicate the geometric size. Also is InversionNet the only one that uses a different number of emitters and receivers here? It will be better if this can be indicated in the figure.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The overall scope and the amount of work are suitable for MICCAI. The technique is novel. Though several concerns in data simulation and validation need addressing.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Accept — should be accepted, independent of rebuttal (5)

  • [Post rebuttal] Please justify your decision

    All comments have been addressed.



Review #3

  • Please describe the contribution of the paper

    The paper describes the implementation of the APS-USCT framework, which is a new approach to ultrasound computed tomography that integrates AI with physical modeling principles to improve image reconstruction for sparse data. It combines two components, APS-wave and APS-FWI to enhance the density and integrity of the waveform data. The approach provides high-quality imaging and also offers a scalable and resource-efficient solution to ultrasound reconstruction.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Enhanced Image Quality from Sparse Data: The APS-USCT framework reconstructs high-quality images from sparse waveform data, which is beneficial for medical imaging where obtaining dense data can be computationally expensive and technically challenging. Efficiency and Cost Reduction: The framework demonstrates a potential reduction in hardware costs by up to 8.5 times while minimally affecting the image quality (SSIM degradation less than 0.031). This is particularly beneficial in clinical settings where budget constraints might limit the accessibility of advanced imaging technologies. High Performance on Standard Metrics: The paper reports excellent scores on standard imaging quality metrics such as SSIM and PSNR, indicating that the reconstructed images maintain high fidelity compared to ground truth. This is important for ensuring accurate diagnoses. APS-wave and APS-FWI Components: The integration of AI with physical modeling to upscale sparse data into dense waveform before reconstruction is a novel approach. The APS-wave uses an encoder-decoder system to enhance data density, and APS-FWI employs advanced deep learning structures like InversionNet, optimized with SE-Blocks for superior image reconstruction. Synergy Between AI and Physics: Unlike conventional methods that primarily rely on either computational algorithms or physical models, APS-USCT integrates both disciplines to improve data quality before processing, which is a relatively new concept in medical imaging.

    Addressing a Critical Need: The innovative use of sparse data to achieve high-quality reconstructions addresses a critical need in medical imaging for cost-effective, efficient, and accessible imaging technology. Potential for Broad Application: The methodology could potentially be adapted for various types of medical imaging beyond breast cancer detection, making it a versatile tool in the field. Interdisciplinary Approach: The combination of AI and physical principles may inspire further interdisciplinary approaches in medical technology development, which can lead to breakthroughs in other complex imaging challenges.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The model’s performance is validated only on numerical breast phantoms, which, while anatomically accurate, may not fully represent the variability encountered in clinical settings. It would be beneficial to provide a quantitative comparison between the NPBs and a real human cohort. I would like to see more details about how the in silica trial is implemented rather than just referencing a method, which even after following the reference is not obvious.

    The comparison might lack depth if it does not consider a wide enough range of existing technologies or modalities that are currently used in clinical settings. For example, the paper primarily compares APS-USCT against other AI-based methods without discussing traditional ultrasound technologies that are widely used and accepted in clinical practice.

    There is little discussion about the scalability of the system, including the computational demands and the practicality of implementing such a system in a typical clinical environment. Advanced AI models and large-scale data processing requirements might impose significant demands on computational resources.

    While the paper provides metrics like SSIM and PSNR, there is limited discussion on the clinical validation of the reconstructed images in terms of diagnostic accuracy and outcomes. Statistical significance of the improvements in imaging quality and their impact on clinical decision-making are not thoroughly explored. Clinical trials or retrospective studies using APS-USCT reconstructed images to validate diagnostic accuracy and outcomes would be essential to demonstrate clinical feasibility.

    It is stated that APS-physics is implemented in “Python”. This is the equivalence of my review stating that the paper is written in English. Much more detail is required here.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    Since the analysis is performed on virtual phantom data, it should be possible to share a dataset that could be independently tested.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    One thing I do not understand about your equation 2 is that you seem to treat the speed of sound c as a constant in the equation, but obviously this is not the case as you are also computing SOS maps. Maybe there is some subtle point I am not understanding, but does the variability of c not invalidate the acoustic wave equation for this purpose? Should c be replaced by the SOS map somehow? Or should the equation be solved for each region of c individually and use the SOS map to create boundary conditions? A more explicate description of how APS-physics is implemented and how the differential equation is solved would likely clear up these questions.

    Since the model was trained on numerical phantoms, the authors should provide evidence at the equivalence of the virtual test to a real test. Even if it is just for a small amount of data – does the solution also work in humans?

    Including a broader comparative analysis with established clinical ultrasound technologies could provide a clearer picture of where APS-USCT stands in the current technology landscape.

    A detailed analysis of the computational costs, required infrastructure, and potential implementation hurdles in clinical settings would make the case stronger for APS-USCT’s deployment.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I would like to see the paper accepted, but I do think it will make a large difference if an improved explanation of the implementation of APS-physics is included. How is the equation solved in python, and what is the effect of the variation of the speed of sound on the solution?

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Accept — should be accepted, independent of rebuttal (5)

  • [Post rebuttal] Please justify your decision

    I felt that the author’s rebuttal satisfactorily addressed my questions. In particular the clarifications about how c is handled in the differential equation clarified that point for me.



Author Feedback

We appreciate the reviewers’ valuable time and comments. In the rebuttal, we summarize and clarify questions from reviewers.

C1. How do we generalize to real data? (R1/R4 Q3.1, R1 Q6.5) We agree that it is important to generalize the proposed method to real data for testing clinical feasibility; however, labeling and obtaining enough training data from clinics is challenging. Some commonly used approaches, such as simulating real data by adding noise or increasing data diversity, can be applied. Note that our approach can still be useful to generate an initial model, which can be fine-tuned on the new datasets. Benefiting from the capability of learning, the generalized learning-based imaging approach can be effective for real-world data where noise is present. This is evidenced by Supplementary Section 11 of the paper [1], where the robustness of InversionNet can be improved by adding Gaussian noise into the dataset.

C2. The details of waveform simulation. (R1 Q3.2 & 6.1, R4 Q7.1) The waveform simulation parameters are set according to those listed in Table 1 of [2]. Our simulation considers heterogeneity only in sound speed, assuming constant tissue density without acoustic attenuation or nonlinearity. Using full-waveform observations, we consider wave scattering. We thank R4 for pointing out the confusion on Equation 2. The notation c in the equation should be c(x), representing the speed of sound at location x. We used a finite difference method (2nd order in time, fourth order in space) for the simulation. We also applied an absorbing boundary condition on all boundaries.

C3. Comparison with conventional physics-based methods. (R1/R4 Q3.3) Performance & Hardware: According to Fig. 5 of [2], the SSIM of physics-based FWI is 0.8, while our method achieved 0.8431 with a reduced transmitter number from 256 to 32. Image Acquisition Time: According to the complexity analysis in section 3C(3) of [3], physics-based FWI’s imaging time is at least one order larger than the ML-based approach (used in our work).

C4. Performance on different types of breasts in Figure 3 (R1 Q6.2 & Q6.7) Figure 3 shows visualization results for specific instances, not the entire dataset. We will clarify this in the revision.

C5. The cause of wave reconstruction by APS-wave in Figure 4 (R1 Q6.6) In the raw waveform, there are both strong and weak events. During learning, APS-Wave prioritizes the reconstruction of strong events of very high quality. But, it also results in less accurate reconstruction of weak events, such as the vertical stripes observed by R1. Note that the strong events are important for imaging, making good inversion results from APS-wave.

C6. Dataset and writing (R3 Q3.1) A part of the phantoms used in the experiments have open access [4]. We will follow your suggestions and check the whole paper to correct typos.

C7. AI-Physics synergy (R3 Q3.2) ML-based APS-Wave is proposed to reconstruct dense waveforms from sparse data, enhancing the physical information. Then, we ML-based APS-FWI are developed for imaging. Both components exemplify how AI assists physics. Combined with the reviewer’s “physics enabling AI,” we call our method an “AI-Physics synergy.”

C8. Diagnostic accuracy (R4 Q3.4) We agree that our current manuscript lacks a discussion on clinical validation regarding diagnostic accuracy. Our paper focuses on image reconstruction from sparse data; diagnostic evaluation, system efficiency, and fairness will be addressed in future work. [1] “OpenFWI: Large-scale multi-structural benchmark datasets for full waveform inversion.” NIPS 2022 [2] “Learned full waveform inversion incorporating task information for ultrasound computed tomography.” IEEE TCI 2024 [3] “InversionNet: An efficient and accurate data-driven full waveform inversion.” IEEE TCI 2019 [4] “2D Acoustic Numerical Breast Phantoms and USCT Measurement Data”, Harvard Dataverse



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    This is an interesting and high-impact work that is of high interest to the MICCAI community. There are some minor issues such as not being clear about what data will be released (even in their rebuttal), but still this is an interesting work and I recommend accepting it.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    This is an interesting and high-impact work that is of high interest to the MICCAI community. There are some minor issues such as not being clear about what data will be released (even in their rebuttal), but still this is an interesting work and I recommend accepting it.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    The reviewers highlighted the framework’s ability to improve imaging speed while reducing hardware dependency, which they noted as a solid contribution to the field. Your method’s ability to outperform the state-of-the-art in simulations using 2D breast images. While the review process has highlighted areas for improvement, particularly in terms of clinical feasibility and the depth of simulation details, the overall strength of your experimental design and results supports the acceptance of your paper. Therefore, majority of reviewers and AC leans to accept.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    The reviewers highlighted the framework’s ability to improve imaging speed while reducing hardware dependency, which they noted as a solid contribution to the field. Your method’s ability to outperform the state-of-the-art in simulations using 2D breast images. While the review process has highlighted areas for improvement, particularly in terms of clinical feasibility and the depth of simulation details, the overall strength of your experimental design and results supports the acceptance of your paper. Therefore, majority of reviewers and AC leans to accept.



back to top