List of Papers Browse by Subject Areas Author List
Abstract
Contrast agents in dynamic contrast enhanced magnetic resonance imaging allow to localize tumors and observe their contrast kinetics, which is essential for cancer characterization and respective treatment decision-making. However, contrast agent administration is not only associated with adverse health risks , but also restricted for patients during pregnancy, and for those with kidney malfunction, or other adverse reactions. With contrast uptake as key biomarker for lesion malignancy, cancer recurrence risk, and treatment response, it becomes pivotal to reduce the dependency on intravenous contrast agent administration. To this end, we propose a multi-conditional latent diffusion model capable of acquisition time-conditioned image synthesis of DCE-MRI temporal sequences. To evaluate medical image synthesis, we additionally propose and validate the Fréchet radiomics distance as an image quality measure based on biomarker variability between synthetic and real imaging data. Our results demonstrate our method’s ability to generate realistic multi-sequence fat-saturated breast DCE-MRI and uncover the emerging potential of deep learning based contrast kinetics simulation. We publicly share our accessible codebase at https://github.com/RichardObi/ccnet and provide a user-friendly library for Fréchet radiomics distance calculation at https://pypi.org/project/frd-score.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3434_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: N/A
Link to the Code Repository
https://github.com/RichardObi/frd-score
https://github.com/RichardObi/ccnet
Link to the Dataset(s)
https://www.cancerimagingarchive.net/collection/duke-breast-cancer-mri/
BibTex
@InProceedings{Osu_Towards_MICCAI2024,
author = { Osuala, Richard and Lang, Daniel M. and Verma, Preeti and Joshi, Smriti and Tsirikoglou, Apostolia and Skorupko, Grzegorz and Kushibar, Kaisar and Garrucho, Lidia and Pinaya, Walter H. L. and Diaz, Oliver and Schnabel, Julia A. and Lekadir, Karim},
title = { { Towards Learning Contrast Kinetics with Multi-Condition Latent Diffusion Models } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15005},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The work introduces a new multi-conditional latent diffusion model specifically engineered for the synthesis of MRI images from pre- to post-contrast phases, it simulates time-dependent contrast uptake in imaging data, and also developed a new metric, Fréchet Radiomics Distance (FRD), a radiology-specific evaluation method to assess the quality of both 3D and 2D synthetic images, with a particular focus on biomarker variability.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Interesting work and architecture using diffusion model, the FRD metric should be of interests to a group of readers of this venue. The methods are extensively compared in the results session.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
FRD depends on certain suggested radiomic features, the performance consistency could be an issue in this case, as radiomic features can be significantly influenced by the format and quality of images.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Do you have any additional comments regarding the paper’s reproducibility?
No
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Would be good to give more evidence to support the consistency of FRD metric.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Bring new architecture and new metric, extensive experimental results.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This work proposes a multi-conditional latent diffusion model (LDM) for pre- to post-contrast magnetic resonance imaging (MRI) synthesis. A Frechet radiomics distance (FRD) is proposed which is a radiology-specific quality evaluation method for both 2D and 3D synthetic images based on biomarker variability. The work discusses the use of the variability in the contrast uptake as biomarkers to quantitatively measure the quality of the generated images from the LDM.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The work proposes a novel way of using contrast uptake as biomarkers for evaluating the quality of the generated images using LDM.
- The paper proposes a novel Fréchet radiomics distance (FRD) method that performs radiology-specific quality evaluation of the generated images based on biomarker variability.
- The paper is well written and organized.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The main weakness of this method is that it is evaluated on only one dataset.
- Results for CC-Net_{Any+Txt}, CC-Net_{Any+Txt+LDM}, CC-Net_{Any+Txt+LDM+CG1.6}, CC-Net_{Any+Txt+LDM+LT}, CC-Net_{Any+Txt+LDM+CG1.6+LT}, are not reported for P2, P3 and P4. Also, there is no discussion as why the performance varies across metrics for different model trainings on P1.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The authors do not clearly how they compose the heterogeneous radiomics features (First Order statistics, GLM, etc) and finally how do they calculate the Fréchet Distance between them.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed method of using contrast uptake as biomarkers to assess the quality of the generated images using LDM is novel. This can be an important downstream application like unsupervised brain tumor segmentation. However, the main weakness of the paper is that the results are shown on only one dataset. Overall, the strengths outweigh the weaknesses.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The authors use a multi-conditional latent diffusion model to generate synthetic dynamic contrast enhanced MRI images. This has great clinical potential particularly for patients requiring frequent gadolinium based contrast exams or those unable to receive contrast (i.e. during pregnancy).
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is well written with clear quantitative and qualitative results.
As a radiologist, I find the synthetic images and contrast kinetics persuasive. Particularly in the setting of breast MR where the pattern of enhancement is far more important than the absolute value.
As contrast enhanced breast MRI is being increasingly used annually to screen young, high risk women, the ability to generate this information without gadolinium is exciting.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The authors put forth quantitative metrics of the synthetic images using Frechet radiomics distance but it remains unclear to me how this may translate into the final image interpretation. How much variability in terms of enhancement and kinetics curves is there from patient to patient or with changing parameters?
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Do you have any additional comments regarding the paper’s reproducibility?
The provided links for source code did not work; however I suspect this is only a technical error. I have no concerns about the paper’s reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- nephrogenic systemic fibrosis is more historical than anything else. With modern forms of gadolinium, there have been no reported cases of NSF even in those with renal dysfunction. In fact, as a hospital policy, we no longer check renal function prior to administering gadolinium. Adverse effects with gadolinium are also exceedingly rare. Strongest argument to avoid contrast in my opinion is pregnancy where we never give gadolinium and in patients requiring repeated doses (e.g. young women at high risk for breast cancer, BRCA, or multiple sclerosis patients) given known deposition in the brain with unknown longterm effects. Scanner time is also a persuasive argument especially given cost and the current economics of adopting AI into clinical practice as a largely non-reimbursable entity.
- additional ‘textual’ information to include maybe some clinical variables about cardiac function (history of heart failure, medications for heart failure, etc.). Often when we are seeing scans with contrast not behaving as we would expect it is because of poor cardiac function.
- I appreciate that FRD is a novel metric but I find myself not having a good sense of what to do with it. If this were to be deployed clinically at any point it would be essential to have an understanding of how much confidence can be placed in the synthetic image and curves. I think it would be reasonable to in fact report synthetic images with a confidence metric and radiologists could then decide if it is insufficient given lesion morphology etc. The patient could ultimately be referred for a repeat ‘real’ gadolinium enhanced exam if the synthetic contrast was insufficient given the clinical scenario and confidence metrics. -Although the approach was not designed to be ‘explainable’ I would be curious to hear the author’s input as to what information in non-contrast images makes this feasible. I suspect this approach would not work with CT or ultrasound for example. As radiologists, we use contrast enhancement patterns to guide our interpretation, but perhaps the information is already imbedded in the non-contrast images it is just too subtle or complex for us to detect. Which then begs the question is generating contrast patterns actually beneficial for the final goal of revealing a diagnosis/prognosis or is it possible to skip this step?
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
For me it is a ‘big idea’ that is well executed. To be able to generate synthetic contrast enhanced images that actually mirror what is generated with gadolinium is a bit like magic.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
We would like to thank all reviewers (R3, R5, R6) for their time and effort in reviewing our manuscript. We appreciate the reviewers’ positive assessment and kind recognition of our work as “novel”, “exciting”, and “well executed” with “great clinical potential” and “clear quantitative and qualitative results”.
The reviewers’ insightful comments and constructive feedback will allow us to enhance and extend upon our work such as including “clinical variables about cardiac function” as part of the LDM textual input (R5), or by providing a Fréchet Radiomics Distance (FRD) library to further investigate the composition (R6), consistency (R3) and variability (R5) of feature sets used to compute the FRD.
R6 remarked on the existence of important clinical applications of our work such as “unsupervised brain tumor segmentation”, which motivates further exploring such downstream tasks using multiple datasets from different domains. Also, we concur with R5’s intriguing question that much of the required diagnosis/prognosis information may be already present (although subtle/complex) in non-contrast images as shown by their usefulness for generating corresponding contrast-enhanced images. However, apart from the information present in a particular non-contrast patient image, population-level statistical patterns (e.g. transferred into network weights during training) learned by mapping a non-contrast image to a contrast-enhanced image may likely be a necessary complementary source of information for diagnosis/prognosis in many scenarios. Generating contrast patterns can further enhance the visual interpretability of embedded (or predicted) diagnostic or prognostic information in non-contrast images, which can help to guide clinical assessment. For such assessments, we agree with R5’s idea that it would be useful to report a confidence metric alongside each synthetic image to determine cases where repeated ‘real’ gadolinium enhanced exams are beneficial.
Once again, we would like to sincerely thank the reviewers for their valuable feedback and thoughtful suggestions, which we will take into account and, to the best of our abilities, integrate as we advance our study.
Meta-Review
Meta-review not available, early accepted paper.