List of Papers Browse by Subject Areas Author List
Abstract
Blood vessel networks, represented as 3D graphs, help predict disease biomarkers, simulate blood flow, and aid in synthetic image generation, relevant in both clinical and pre-clinical settings. However, generating realistic vessel graphs that correspond to an anatomy of interest is challenging. Previous methods aimed at generating vessel trees mostly in an autoregressive style and could not be applied to vessel graphs with cycles such as capillaries or specific anatomical structures such as the Circle of Willis. Addressing this gap, we introduce the first application of \textit{denoising diffusion models} in 3D vessel graph generation. Our contributions include a novel, two-stage generation method that sequentially denoises node coordinates and edges. We experiment with two real-world vessel datasets, consisting of microscopic capillaries and major cerebral vessels, and demonstrate the generalizability of our method for producing diverse, novel, and anatomically plausible vessel graphs.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1908_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/1908_supp.pdf
Link to the Code Repository
https://github.com/chinmay5/vessel_diffuse
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Pra_3D_MICCAI2024,
author = { Prabhakar, Chinmay and Shit, Suprosanna and Musio, Fabio and Yang, Kaiyuan and Amiranashvili, Tamaz and Paetzold, Johannes C. and Li, Hongwei Bran and Menze, Bjoern},
title = { { 3D Vessel Graph Generation Using Denoising Diffusion } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15011},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper introduces an innovative method utilizing denoising diffusion models to generate 3D vessel graphs, addressing the limitations of previous tree-based generation methods by effectively handling cyclic structures. The authors have made their code publicly available, which, combined with the use of public datasets for experimentation, enhances the reproducibility and transparency of the research. This practice facilitates other researchers to verify the results and build upon this work, contributing to the scientific integrity and collaborative progress in the field.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper introduces a novel application of denoising diffusion models to generate 3D vessel graphs.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1) The motivation to build a diffsion model for the generation of 3D vessel graphs are not clear. The authors claims that it can be used to predict disease biomarkers, simulate blood flow, and aid in synthetic image generation, however, it is difficult to evaluate based on the description and experimental settings of this paper. 2) The evaluation seems to be unfair, as the experimental comparisons are only against the existing spatial graph generation methods but not the SOTA ones.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The paper introduces an innovative method utilizing denoising diffusion models to generate 3D vessel graphs, addressing the limitations of previous tree-based generation methods by effectively handling cyclic structures. The authors have made their code publicly available, which, combined with the use of public datasets for experimentation, enhances the reproducibility and transparency of the research. This practice facilitates other researchers to verify the results and build upon this work, contributing to the scientific integrity and collaborative progress in the field. My major review comments are listed below: 1) The motivation to build a diffsion model for the generation of 3D vessel graphs are not clear. The authors claims that it can be used to predict disease biomarkers, simulate blood flow, and aid in synthetic image generation, however, it is difficult to evaluate based on the description and experimental settings of this paper. 2) The evaluation seems to be unfair, as the experimental comparisons are only against the existing spatial graph generation methods but not the SOTA ones.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
1) The motivation of this paper is difficult to evaluate based on the description and experimental settings of this paper. 2) The experimental settings seems to be unfair, as the comparisons are only against the existing spatial graph generation methods but not the SOTA ones.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This work aims to introduce denoising diffusion models to the generate 3D vessel graphs. The key innovation includes a two-stage generation method that processes the node coordinates first, followed by generating edges.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Solid and extensive literature review. Presentation is clear and easy to follow. Experiments are well-designed and most results are clearly explained.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The title and targeted structure are about 3D vessels, but the design is quite general, and not specifically for 3D vessels. Human’s circulatory system, which includes blood vessels like arteries, veins, and capillaries, spreads throughout the body in a branching pattern that can resemble the branches and roots of a tree. However, this work doesn’t have any consideration of this major characteristics. It doesn’t promise to generate any vessel-like output if the data used are not vessels.
The comparisons with [24][25] seem unfair, considering [24][25] are designed for molecule generation, assuming the structure are permutation equivariant.
Table 2 is reported with very limited significant digits. For example, in the columns labeled ‘x, y, z’, there are essentially only one to two significant digits, instead of four.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Table 2 is reported with very limited significant digits. For example, in the columns labeled ‘x, y, z’, there are essentially only one to two significant digits, instead of four. Instead of reporting “0.010, 0.004, 0.002, 0.002”, the authors can consider taking a different unit and reporting more significant digits like “1.0xx, 0.4yy, 0.2zz, 0.2ww”.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This work is well-developed and well-written. The proposed technique has merit and novelty, which make the work deserving of publication. However, as I state in the ‘Weaknesses of the Paper’ section, the technique developed in this work is not specific to vessels and does not guarantee the production of vessel-like structures. Instead of MICCAI, this work may be more appropriately published at a general learning-oriented conference.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper introduces a two-stage method for generating 3D vessel graphs using denoising diffusion models. The method first generates node coordinates using a continuous diffusion model, and then generates edges between the nodes using a discrete diffusion model while keeping the node locations fixed. It addresses the challenge of simultaneously generating nodes and edges in vessel graphs and takes care of generating realistic edges for the application.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The two-stage approach is well-motivated and tailored to the unique characteristics of vessel graphs, where node locations strongly influence edge existence.
- The model demonstrates good performance in generating diverse, plausible vessel graphs across two real-world datasets of capillaries and major cerebral vessels.
- The paper makes a valuable contribution by applying diffusion models, a state-of-the-art generative modeling technique, to the under-explored problem of vessel graph generation.
- The experiments show the importance of key design choices like using a degree loss or edge types, and shows knowledge of the needs in the application.
- The work takes care of technical details, like the Gumbel-softmax trick for discrete data.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The evaluation metrics, while comprehensive, do not directly measure the realism or anatomical plausibility of the generated graphs from a clinical perspective.
- The baseline experiments lack sufficient detail, including information on whether the models were fine-tuned.
- The paper does not discuss potential applications or downstream tasks that could benefit from the generated vessel graphs.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
The paper provides an anonymous link to a GitHub repository, implying that the code will be made available, while the detailed descriptions within the paper suggest that the experiments are designed to be reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The design of the work and models appears well-thought-out, with specific applications in mind and technical decisions that seem appropriate. The writing is clear and concise. While the results show a significant improvement over other models, translating the metrics to realism in generated graphs can be challenging. I acknowledge that comparing generated images directly with ground truth may be difficult, but I suggest for future work that you select samples from each model’s output that closely resemble ground truth samples and compare which model generates more similar graphs.
Additionally, providing more details on experiments involving other models would be beneficial. Some specific observations include: Figure 1 mentions L_edge and the subsequent description appears much latter, while L_node is not mentioned; perhaps it would be better to omit or mention both. The term “learnable parameter γ” is introduced but left undefined. Furthermore, the phrase “edge type” and “edge class” are used interchangeably. Finally, the sentence “For both datasets, we have used a metric graph” appears out of place and could be rephrased for clarity.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
In general, the work appears to be very well-designed, with promising results evident, even though the comparative analysis primarily focuses on the distribution of graph elements. Nonetheless, this suggests that the approach is effective.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Accept — should be accepted, independent of rebuttal (5)
- [Post rebuttal] Please justify your decision
The authors’ response is good, although when they say “our models were trained from scratch on specific datasets”, I’m not sure if they also mean SOTA models, but even assuming that’s the case, I think the work is suitable for “Accept — should be accepted, independent of rebuttal”, not for “Strong Accept — must be accepted due to excellence”.
Author Feedback
Dear Reviewers and Area Chairs,
We appreciate the reviewers’ valuable comments and are encouraged by their acknowledgment of the novelty of our two-stage diffusion model (R1, R3, R4), which effectively handles cycles addressing the limitation of previous tree-based methods (R1). We are humbled by their remark on a well-motivated work with well-thought-out design choices (R4), well-designed experiments (R3, R4) with clear results (R3, R4), and good performance on two real-world datasets (R4). Below, we address the raised concerns.
Motivation & Downstream Tasks (R1, R4): Synthetic vessel graph generation is an important topic [20, 22] and can augment real datasets, where annotating large-scale samples is expensive (CoW) and tedious (VesSAP) for downstream tasks, such as blood flow simulation [21], vessel labeling, and synthetic image-label pairs for segmentation [10, 14]. Previous studies have demonstrated improved downstream performance [10] when generated graphs capture real data distribution. Our analysis (Tab 1, Fig. 4) shows that our model captures the underlying distribution well in terms of various graph and topology properties, ensuring the usefulness of the generated samples. We have now included this point in our manuscript.
SOTA Comparison (R1, R3): Kindly note that it is challenging for us to identify a SOTA baseline since no concrete references to prior work were pointed out beyond our SOTA baselines [24, 25]. Below, we answer, to the best of our ability, why we believe our comparison is fair. We aim to develop a generative model for complex vessel graphs, including cycles. Notably, previous SOTA vessel generation models [20, 22] can only generate tree structures and NOT anatomically meaningful cycles. Hence, these models are fundamentally NOT APPLICABLE as a baseline to the datasets considered in our paper. Nevertheless, CoW resembles a benzene molecule-like ring, and we could interpret VesSAP samples as a collection of molecules. Hence, we compare against SOTA molecule generation methods [24, 25], which can handle cycles. @R3, please note that both baselines [24, 25] and our model consist of permutation equivariant layers, making it a fair comparison.
MICCAI Suitability (R3): Although some large arteries and veins resemble tree-like structures, the human circulatory system also contains cycles at the capillary level and in anastomotic structures (CoW). Hence, we respectfully disagree with R3 on explicitly enforcing tree-like characteristics as it will hamper cycle generation—a limitation of previous works [20, 22]. The complex structures (trees and cycles) make it difficult to determine the complete topological characteristics of a vascular graph accurately, and incorporating inadequate inductive bias would limit the model’s capabilities. Instead, we opt for a data-driven approach without hand-crafted heuristics, and we make the following design choices to facilitate data-driven learning.
- Two-stage diffusion to learn the connectivity from bifurcation locations
- Removing rotation equivariance to learn dataset-specific vessel-orientation
- Degree-distribution loss to learn the branching pattern of the vessels
Although our method can not generate vessels if the training data is not vascular (a limitation of any data-driven approach), it can model complex topological structures, including arteries, veins, and capillaries, as shown in its performance on two real vessel datasets (Tab. 1), which is of interest to MICCAI community.
Metrics (R4): We acknowledge that the current literature lacks a ready-to-use metric to evaluate the anatomical plausibility of the vessel graphs. Hence, we opt for well-established topology and graph-structure metrics from vessels [4, 29] and molecules [25] for our evaluation.
Misc: @R3, we have included significant digits in Tab. 1. @R4, our models were trained from scratch on specific datasets. We have added baseline details and clarified other textual concerns.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The paper makes a valuable contribution to the conference. There is novelty and strong results. The authors addressed concerns of the most critical reviewers well.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The paper makes a valuable contribution to the conference. There is novelty and strong results. The authors addressed concerns of the most critical reviewers well.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
Although this work is niche, the author introduce a novel method based on denoising diffusion model. Moreover, the author has validated its performance on two real-world vessel datasets. This work may be a potential contribution to the community. Thus, I recommend acceptance.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
Although this work is niche, the author introduce a novel method based on denoising diffusion model. Moreover, the author has validated its performance on two real-world vessel datasets. This work may be a potential contribution to the community. Thus, I recommend acceptance.