Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Cone Beam Computed Tomography (CBCT) finds diverse applications in medicine. Ensuring high image quality in CBCT scans is essential for accurate diagnosis and treatment delivery. Yet, the susceptibility of CBCT images to noise and artifacts undermines both their usefulness and reliability. Existing methods typically address CBCT artifacts through image-to-image translation approaches. These methods, however, are limited by the artifact types present in the training data, which may not cover the complete spectrum of CBCT degradations stemming from variations in imaging protocols. Gathering additional data to encompass all possible scenarios can often pose a challenge. To address this, we present SinoSynth, a physics-based degradation model that simulates various CBCT-specific artifacts to generate a diverse set of synthetic CBCT images from high-quality CT images, without requiring pre-aligned data. Through extensive experiments, we demonstrate that several different generative networks trained on our synthesized data achieve remarkable results on heterogeneous multi-institutional datasets, outperforming even the same networks trained on actual data. We further show that our degradation model conveniently provides an avenue to enforce anatomical constraints in conditional generative models, yielding high-quality and structure-preserving synthetic CT images.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1155_paper.pdf

SharedIt Link: https://rdcu.be/dV5Fq

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72104-5_62

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/1155_supp.pdf

Link to the Code Repository

https://github.com/Pangyk/SinoSynth

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Pan_SinoSynth_MICCAI2024,
        author = { Pang, Yunkui and Liu, Yilin and Chen, Xu and Yap, Pew-Thian and Lian, Jun},
        title = { { SinoSynth: A Physics-based Domain Randomization Approach for Generalizable CBCT Image Enhancement } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15007},
        month = {October},
        page = {646 -- 656}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    In this paper, a physics-based domain randomization method is proposed for generalizable CBCT image enhancement, which is named SynoSynth.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The experiments show the superiority of the proposed method by both qualitative and quantitative results.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. The training/implementation details are unclear, making it hard to reproduce the work. For example, what the relationship between the two losses and how to train the model?
    2. The used quality metrics are all for traditional natural images. Some specific quality metrics for medical CT images are suggested to be reviewed, such as RTN: Reinforced transformer network for coronary CT angiography vessel-level image quality assessment.
    3. Only one dataset is used for the experiments. However, the cross-dataset validation is important for learning-based models.
    4. Only two methods are compared. Both were published in 2021, which are insufficient.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Please find them in the weakness.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    See the weakness.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Reject — should be rejected, independent of rebuttal (2)

  • [Post rebuttal] Please justify your decision

    The authors do not well address my comments. Further validations would be helpful to verify the performance.



Review #2

  • Please describe the contribution of the paper

    This paper proposed a novel physics-based data augmentation method that generate different synthetic CBCT images with different combination of artifacts (e.g., Scanner Effects, Metal Artifacts, Extinction Artifacts). By using these synthetic images as a training dataset, the authors observed that various generative models achieved superior denoising performance compared to models trained on real CBCT data alone.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    This paper proposes a novel physics-based data augmentation method that simulates realistic variations in CBCT image noise and artifacts (e.g., scanner effects, metal artifacts, extinction artifacts). This innovative approach allows generative models to learn the underlying distribution of artifacts during training, potentially leading to superior denoising performance. Additionally, the authors demonstrate the effectiveness of their method through a comprehensive evaluation using multi-center clinical image datasets.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Lack of clarity. Better explanation of the preprocessing (e.g., how many synthetic images were generated and used for training) is needed. Additionally, it is unclear whether the size of the generated synthetic images that used for training could also affect the performance of the generative models (e.g., comparing 1000 vs. 100 synthetic training images).

    Statistics: it might be also to have statistic test to find out whether the improvement is significant.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    The study’s reproducibility is limited due to the use of a private database. There is no detailed information related to the hyperparameter settings for training different models. The implementation codes are also not available. It remains unclear whether the models were built from scratch or utilized publicly available code.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Please see the weaknesses section and improve the paper.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall, the paper is well-written. The idea of using different combinations of artifacts prior to generating synthetic images to train a better generative model for CBCT-CT image translation is novel. However, the evaluation could be strengthened by providing additional analysis, such as investigating how the size of the training dataset could affect the overall testing performance. Additionally, it might be better to provide additional statistical tests to support the results.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    In this work, the author introduced a physics-based domain randomization approach to address the inherent challenges associated with generating synthetic CT images from CBCT scans, including susceptibility to artifacts and limited generalizability of data.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Compared to networks trained with previous augmentation methods or actual data, SinoSynth-trained networks demonstrate significantly better zeroshot generalization ability and structure-preserving ability on challenging datasets collected from multiple hospitals.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. How did you implement Equation 1? A sinogram has only one G. What does it mean to integrate over G? Is it integrating over different Gs or integrating over different parameters within G?
    2. For metal artifacts simulation, adding noise directly to the projection data is unreasonable, in fact, the tissue parts in the CT image that are occupied by metal do not contribute to the sinogram. Furthermore, there is a significant discrepancy between simulated metal artifacts and real artifacts, which can easily lead to the issue of domain gap.
    3. Table 3 is confusing. How was the first insight derived?
    4. In Table 3, why might the results under the influence of multiple factors be better than those under a single interfering factor?
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    1. Provide the implementation details of Equation 1.
    2. More details about Ablation studies.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Lack of experimental details. Inaccurate simulation precedure.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #4

  • Please describe the contribution of the paper

    This paper proposes sinosynth, a data augmentation method for CBCTs inflicted by various artifacts generated from CTs. The authors propose to simulate various artifacts in sinogram domain, and use structural and generative consistency to train the generation (translation) module. Experiments are conducted to evalute the method on various denoising tasks.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Overall, the paper is well written and easy to follow. The topic is timely and important. The approaches seem to be novel and designed well, specifically a number of artifacts resulting from device variability, metal, quantum noise, zebra and motion can be integrated in the simulation. The method does not require matched CBCT-CT pairs, which alleviates the overhead of data collection. Simuation results look compelling.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    • Some of the architectural choices should be clarified:

    • In (8), artifcact mask M is applied in the loss. But shouldn’t G_theta(y) (which is supposed to be artifact-free synthetic CT) be close to x itself? Please explain why the loss used masked-out CTs.

    • Unsure what (4) means: it is the probability expression for Poisson distribution. Does this mean that sinogram value of x_s is replaced (or added) by this probability y_s?

    • The simulation of various artifacts is done in sinogram domain. However except for metal artifacts, the artifacts are more or less implemented in the image domain then Radon transform is applied. It would be good if there is some explanation on specific benefits of the simulation on sinogram domain.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    • For the main comments for potential improvement, see Weaknesses.

    • It is confusing that symbol “D” is used for deformation in Motion Artifacts subsection, but also as the entire CBCT simulation model in 2.3.

    • The proposed method in Title and Abstract is called”Syno”synth, whereas in the paper it is called “Sino”synth, maybe the latter is correct?

    • Some of fonts in math mode should be fixed, e.g. in Eq (1), you should use \exp instead of simply exp in math mode, this makes exp, ln appear italic font (which should not).

    • This is a minor point but if the scheme is called “physics-based” it may be a bit misleading. I would prefer naming without such statement, because it can be a redundant statement: most engineering problems are trivally based on physics.

    • typo: “signogram” in page 4 under eq 1 meta-affected region above (7): should be “metal”-affected?

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall the paper is well-written, although there needs some clarifications and minor corrections (mentioned above). The idea and approach to the data augmentation of CBCTs from unpaired CTs seem to be quite useful, which is backed up by the experimental results.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #5

  • Please describe the contribution of the paper

    The paper describes a AI based technique to improve the CBCT images from the several artificacts that arise during CBCT imaging.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper is exceptionally well written in terms of introduction, literature review, methodology, results and discussion. There happens many artifacts during CBCT imaging and reducing them is a image pst processing method. The authors propose a new methodology that improves the image quality and free from the artifacts (artifacts reduction). This has potential application in clinical settings.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    There is little less explanation of image acquisition parameters and the range of values of the images in the resource pool. The literature review should have been little more critically evaluted.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    I could not see the code and the data deposition along with this work. I believe authors had submitted the details during submission.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    1. Do mention multiple references together like [7, 19, 8] and [1, 15, 25]. Mention the methods and then cite separately. E.g.: X[7], Y[19} and Z[8]

    2. As mentioned in paragraph 1, after Fig 1, it is not really true that there is a difference in the scanning protocol when your dataset is from a country. Offcourse the protocol differs across countries. It is unclear that you are making the protocol variability statement w.r.t dataset from one country or from several countries. Also, technically as the noise in the image mainly depends on mA and since there is lot of advancements in either acquisition or in the image reconstruction from projection data. Hence, it is little difficult to accept your statement about diagnostic quality. Difference in the quality can be there offcourse if there are any artifacts such as motion, PVE, beam hardening etc.

    3. Page 2, last paragraph: Contrast and sharpness adjustment is okay, but how the brightness is adjusted and how it has helped is not clear.

    4. In Dahiya et al, what do you mean by “extracting artifacts”. Which type of artifacts were extracted? What was the purpose of mapping the artifacts to other datasets and what was the objective?

    5. It is good to mention the 5 artifacts at the same place where it is mentioned for the first time.

    6. What is “domain knowledge”? It is just mentioned at two places but not described anywhere. Is it the knowledge of the radiologist who interprets the images on natural slices? or is it something related to the lexicons? geometrical shape analysis?

    7. In 2.1, is it HU converted to linear attenuation coefficients or linear attenuation coefficients to HU?

    8. As mentioned in 3.1, 256x256 is image size not the pixel size. Pixel sizes are measured in mm in x and y direction. Its value remains same if it is square detectors and different if rectangular.

    9. As Fig 3 images looks too small thumbnails, it is difficult to zoom and check the individual images where we lose spatial resolution. Move the 4 column of images shown in green color to next line and zoom all the images adjusting to the width of the paper.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper is well written and the method looks new. Technically the content is good with few comments.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

We thank all reviewers for their valuable time and feedback, which helped improve this manuscript. We are glad that the reviewers appreciate novelty (R1, R3, R5, R6), superior performance (all reviewers), clinical usefulness (R5, R6), and writing (all reviewers).

We will release the code upon acceptance.

R1 Q1 Since we applied our method to existing frameworks, all models were trained with their original losses (e.g., GAN losses) alongside the proposed structure-preserving losses L_struc and L_sino Q2 PSNR/SSIM/MAE are widely used image quality metrics in medical imaging ([17,18,19]). The metric mentioned is tailored for vessel-level IQA that requires coronary artery region detection and vessel-level labels, which is hardly applicable to our head and neck data Q3 Our test set comprises 10,500 CBCT images from 70 patients collected from 5 hospitals spanning Europe and the US Q4 To the best of our knowledge, the two CBCT data augmentation methods we compared are the latest

R3 Our method simulates CBCT images on the fly during training instead of pre-generating them. Specifically, each model is trained for 200 epochs with 37,500 iterations in each epoch, and thus, there are 7,500,000 simulated CBCT images during the training process. This could well cover the degradation types and prevent overfitting, as demonstrated through quantitative and qualitative evaluation on 10,500 image pairs

R4 Q1 We apologize for a typo - “dG” should be “dt” and “x_mu” should be “x_mu(t)”, where t denotes the position along the X-ray beam path within cone beam geometry G. Eq 1 integrates the 3D object’s attenuation coefficients over t along the X-ray path to obtain the 2D projection data. We used the ASTRA Toolbox and Operator Discretization Library to implement Eq 1 Q2 Metal artifacts result from X-ray polychromaticity [29, 2]. As the polychromatic projection P is linear, following [29], we added the polychromatic projected metal implants to projection data, which amounts to adding metal implants to the CT image followed by P. We experimentally show that our simulation enables high performance on a highly heterogeneous CBCT dataset. We leave more realistic simulations for future work Q3-4 Tab.3 indicates which kind of artifact is simulated or constraint is applied. The results show that greater diversity in simulated artifacts leads to better performance, and the gains in simulating a particular kind of artifact may reflect its occurrence frequency in the dataset, i.e., the gain is the largest when that artifact is the majority

R5 Q1, Q5, Q8, Q9 We appreciate your suggestions and will revise our manuscript Q2 As our test data were acquired from 5 European and US hospitals, the protocols could be different. The DICOM header shows different mAs used in scanning (10, 12, 20, and 50). Although CBCT imaging technology has advanced, many hospitals have not upgraded to the latest machine Q3 Brion et al. adjust brightness by adding a random offset to the CBCT image (sec. 3.3 in [4]), in order to reduce the gap between CT and CBCT intensity distributions Q4 Dahiya et al. share the same goal as ours, except that they do not explicitly simulate the CBCT artifacts as we do, but directly extract them from other existing CBCT images. They used “power-law adaptive histogram equalization to extract scatter/noise artifacts” Q6 “Domain knowledge” refers to 1) CBCT scanning configurations and 2) physical causes of the artifacts, which we translated into algorithms simulating CBCT artifacts (Eqs 1-6) Q7 We converted HU to linear attenuation coefficients

R6 Q1 Since some CT scans can have metal artifacts, we exclude such affected regions via a mask. Q2 Yes, it means the Poisson-noise-contaminated sinogram Q3 Applying the Radon transform to the simulated images allows for inducing device-related effects (Sec.2.2 “Scanner effects simulation”), influencing the final appearance of the simulated artifacts Sec 10 Thank you for indicating the typos - we will correct them



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    The rebuttal adequately addresses all reviewers’ comments.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    The rebuttal adequately addresses all reviewers’ comments.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    The paper introduces SynoSynth, a physics-based domain randomization method for CBCT image enhancement, demonstrating superior performance through extensive evaluations. Reviewers praised the novelty, clinical relevance, and writing quality but raised concerns about training details, dataset variety, and comparisons with state-of-the-art methods. The authors’ rebuttal effectively addressed these points, clarifying implementation specifics, justifying the choice of evaluation metrics, and explaining artifact simulation techniques. Given the paper’s innovative approach, promising results, and thorough responses, it makes a significant contribution to the field and should be accepted for MICCAI 2024.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    The paper introduces SynoSynth, a physics-based domain randomization method for CBCT image enhancement, demonstrating superior performance through extensive evaluations. Reviewers praised the novelty, clinical relevance, and writing quality but raised concerns about training details, dataset variety, and comparisons with state-of-the-art methods. The authors’ rebuttal effectively addressed these points, clarifying implementation specifics, justifying the choice of evaluation metrics, and explaining artifact simulation techniques. Given the paper’s innovative approach, promising results, and thorough responses, it makes a significant contribution to the field and should be accepted for MICCAI 2024.



back to top