Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Deep learning (DL) methods typically require large datasets to effectively learn data distributions. However, in the medical field, data is often limited in quantity, and acquiring labeled data can be costly. To mitigate this data scarcity, data augmentation techniques are commonly employed. Among these techniques, generative models play a pivotal role in expanding datasets. However, when it comes to ultrasound (US) imaging, the authenticity of generated data often diminishes due to the oversight of ultrasound physics.

We propose a novel approach to improve the quality of generated US images by introducing a physics-based diffusion model that is specifically designed for this image modality. The proposed model incorporates an US-specific scheduler scheme that mimics the natural behavior of sound wave propagation in ultrasound imaging. Our analysis demonstrates how the proposed method aids in modeling the attenuation dynamics in US imaging. We present both qualitative and quantitative results based on standard generative model metrics, showing that our proposed method results in overall more plausible images. Our code is available at https://github.com/marinadominguez/diffusion-for-us-images

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2125_paper.pdf

SharedIt Link: pending

SpringerLink (DOI): pending

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/2125_supp.pdf

Link to the Code Repository

https://github.com/marinadominguez/diffusion-for-us-images

Link to the Dataset(s)

https://www.cs.cit.tum.de/camp/publications/segthy-dataset/ https://humanheart-project.creatis.insa-lyon.fr/database/#collection/6373703d73e9f0047faa1bc8

BibTex

@InProceedings{Dom_Diffusion_MICCAI2024,
        author = { Domínguez, Marina and Velikova, Yordanka and Navab, Nassir and Azampour, Mohammad Farid},
        title = { { Diffusion as Sound Propagation: Physics-inspired Model for Ultrasound Image Generation } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15004},
        month = {October},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper introduces a novel diffusion model tailored for ultrasound image generation, integrating a physics-based noise scheduler that simulates sound wave propagation characteristics. This approach aims to improve the realism of synthetic ultrasound images by addressing the physical properties of sound wave interactions, such as attenuation and echo, which are often overlooked in standard diffusion models. The model demonstrates enhanced image quality and fidelity, as evidenced by qualitative and quantitative evaluations against traditional methods.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper introduces a unique noise scheduler inspired by the natural behavior of sound wave propagation, which is a significant departure from traditional diffusion models that do not consider the physical dynamics of ultrasound. This novel formulation allows for a more realistic simulation of how ultrasound images are generated in a clinical setting, mimicking the attenuation and echo patterns seen in actual ultrasound scans.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The reasoning for the methodology is lacking.

    • The explanation is lacking on why emphasizing areas close to the probe helps increase contrast and confidence.

    The qualitative assessment is lacking.

    • When conducting a qualitative evaluation, the absence of raw images and labels makes it difficult to determine at a glance whether the presence or absence of a b map leads to better performance.
    • This appears to be an even greater issue in the supplementary materials.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Do you have any additional comments regarding the paper’s reproducibility?

    The paper makes commendable efforts towards reproducibility, particularly through the availability of code and detailed methodological explanations.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    This paper proposes a diffusion model that significantly reflects the physics of ultrasound image formation, presenting a major novelty. The methodology is clearly described, accompanied by code, and it is evident that the proposed method effectively mimics ultrasound image formation. However, as previously mentioned, the paper lacks an explanation of why emphasizing areas close to the ultrasound probe results in better contrast and confidence. Additionally, there is a shortage of comparative analysis between original and generated images in the qualitative evaluation. Addressing these issues through a rebuttal could potentially lead to a higher rating.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I think the idea is incredibly interesting and innovative. The results have noticeably improved and could be groundbreaking in the field of ultrasound image synthesis in the future. However, the paper would be much improved with a more detailed explanation of why the proposed methodology leads to better contrast and confidence, as well as additional qualitative evaluation of the results.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper proposes a novel noise schedule theme for training diffusion models to generate US images. The proposed schedule is designed to mimic the physics of sound wave propagation through the scan. It is applied to three different datasets of US images, consistently improving the objective quality of generated images.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    • The proposal shows significant improvements in synthesising high-quality US images.

    • The proposed approach for generating US images was evaluated on three different US datasets with diverse attributes, such as scanners.

    • The paper has a clean writing style and is easy to follow. One exception is the first sentence of page 2.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • The introduction and explanation of B-Maps / B-mode images fall very short for potential readers without that background.

    *Providing the class-wise metrics could further highlight the performance gains from the method.

    • Some details on the experimental setup are missing, e.g. train/val splits for SegThy and Liver datasets and the number of total training steps/epochs. The incentive to augment CAMUS, but not the other datasets, is also unclear.

    • Without looking into the Guided-Diffusion and SDM papers, the method’s description is unclear, negatively impacting the work’s reproducibility and transparency. For example, it remains unknown if any changes were made to these models. From the text alone, one might also think that these are two distinct methods, but it is probably the same model with a conditioning mechanism.

    • While I can understand the motivation to use different features for the FID score, the “original” score should still be reported for a fair comparison with previous and future work. Further, the sample size for computing the FID was not reported but has a crucial role in the metric’s interpretability. The suggested sample size is in the thousands. From Figure 4 and the supplementary, a difference of 70 in FID scores is not really visible.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Do you have any additional comments regarding the paper’s reproducibility?

    Public code and two of three datasets are publicly available.

    The trained models are not provided.

    Some lacking details on the experiments limit the reproducibility.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Please have a look at the list of weaknesses and strengths for detailed comments. All of the weaknesses can be addressed easily before publication.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper shows good strengths and proposes an exciting idea for physics-based improvements of diffusion models. The weaknesses are minor and can be addressed until publication.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    This paper presents a novel approach to generating US images using diffusion models with a noise schedule to replicate the physics of US propagation. The generated images are evaluated qualitatively and quantitatively and present excellent results with different databases from different anatomical regions

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    This papers contribute to the quality enhancement of US images artificially generated. The methods are innovative and the authors demonstrated the clinical feasibility by testing the method with different databases, from different anatomical regions and acquired with different scanners. The paper is very well written and has its organization is excellent.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    I did not identify any weakness of the paper

  • Please rate the clarity and organization of this paper

    Excellent

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    The paper is already excellent and does not need further improvements

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Strong Accept — must be accepted due to excellence (6)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I believe the paper should be accepted due to the excellent organization and clarity, the innovative methodology, the completeness of the data analysis and the potential application of the methodology.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

We thank the reviewers for their thoughtful evaluations and constructive feedback. We are grateful for recognizing our diffusion model’s novelty and clinical applicability for generating ultrasound images, as highlighted by all reviewers. Reviewers found our work to be an “innovative method” and a “significant departure from traditional approaches” (R4) with “excellent organization and clarity” (R3), “clean writing style ” (R5) and “completeness of the data analysis” (R3), demonstrating “clinical feasibility by testing the method with different databases, from different anatomical regions and acquired with different scanners” (R3). Such feedback underscores the potential impact of our work on the field of ultrasound image synthesis, with one reviewer remarking that our results “have noticeably improved and could be groundbreaking in the future” (R4).

METHODOLOGICAL CLARIFICATIONS: We have revised our paper to better explain the emphasis on areas close to the probe (R4&R5), ensuring the reasons for enhanced contrast and confidence in the generated images are clearly understood. Our focus on regions closer to the probe is due to their inherent clarity and detail. By emphasizing these areas, we capture the most reliable details with higher fidelity as we allocate more denoising steps to these regions. Further, we have also refined the definition of B-maps (R5) to more clearly communicate their role and benefit in our model so that readers without an ultrasound background have a clearer understanding of the formation of ultrasound images. Thereafter, we highlight that our proposed scheduler is designed to mimic the natural attenuation of sound waves interacting with tissues to generate more plausible B-mode images.

QUALITATIVE ASSESSMENT: Acknowledging the suggestions from R4 regarding the qualitative evaluation, we have overlaid arrows on the images to directly highlight areas where our model has enhanced image quality. These arrows guide the viewer to specific improvements, facilitating a clearer comparison between the synthesized images with and without B-Maps.

EXPERIMENTAL DETAILS:

  • We have addressed R5’s comments regarding the missing total number of training steps/epochs: we have completed 28,000 training iterations for the CAMUS dataset, 36,000 for the Liver dataset, and 50,000 for the Thyroid dataset.
  • Moreover, as recommended by R5, we have incorporated the training and validation splits for the Liver and Thyroid datasets to ensure replicability and transparency.
  • In response to concerns about the clarity of modifications to the Guided-Diffusion and SDM models (R5), we have first incorporated brief descriptions of these baseline models and secondly clearly separated our methodological contributions: introducing our novel noise in the forward pass by our derived equation (5) and modifying the reverse process to incorporate the B-Maps with our derived equations.

FID SCORES: The choice to use those specific layers for assessing the FID scores, rather than the final layer, is deliberate. The selected layers offer a more appropriate feature recognition capability for ultrasound images, which differ significantly from natural images. Thanks to the feedback (R5), we have modified the paper to explain that we compute the FID score using the entire validation set to ensure a comprehensive representation of the model’s performance. Despite the apparent subtlety in visual differences, the statistical distribution captured by the FID scores confirms the efficacy of our model. We understand the concern and have made the qualitative analysis clearer, as suggested by R4 and R5.

MISC:

  • We will provide access to our trained models and the complete source code on our repository.

We trust that the revisions provided here effectively address the reviewers’ suggestions. Our revisions aim to enhance the clarity, depth, and scientific rigour. We will incorporate all the necessary changes into the final version of the paper.



Meta-Review

Meta-review not available, early accepted paper.



back to top