List of Papers Browse by Subject Areas Author List
Abstract
3D reconstruction of cerebral vasculature from 2D biplanar projections could significantly improve diagnosis and treatment planning. We introduce a novel approach to tackle this challenging task by initially backprojecting the two projections, a process that traditionally results in unsatisfactory outcomes due to inherent ambiguities. To overcome this, we employ a U-Net approach trained to resolve these ambiguities, leading to significant improvement in reconstruction quality. The process is further refined using a Maximum A Posteriori strategy with a prior that favors continuity, leading to enhanced 3D reconstructions. We evaluated our approach using a comprehensive dataset comprising segmentations from approximately 700 MR angiography scans, from which we generated paired realistic biplanar DRRs. Upon testing with held-out data, our method achieved an 80% Dice similarity w.r.t the ground truth, superior to existing methods. Our code and dataset are available at https://github.com/Wapity/3DBrainXVascular.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1481_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: N/A
Link to the Code Repository
https://github.com/Wapity/3DBrainXVascular
Link to the Dataset(s)
https://public.kitware.com/Wiki/TubeTK/Data
https://brain-development.org/ixi-dataset/
BibTex
@InProceedings{Caf_Two_MICCAI2024,
author = { Cafaro, Alexandre and Dorent, Reuben and Haouchine, Nazim and Lepetit, Vincent and Paragios, Nikos and Wells III, William M. and Frisken, Sarah},
title = { { Two Projections Suffice for Cerebral Vascular Reconstruction } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15007},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes a method to reconstruct the 3D brain vasculature from two orthogonal X-ray projections. A Unet is used to postprocess a simple unfiltered backprojection as initialization, followed by iterative optimization of the voxel grid. This is a regularized Nerf-like optimization using the voxel grid as parameterization and a regularizer encouraging connectivity of the vessels.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper deals with a strongly ill-posed inverse problem, where the two available projection images allow for many possible solutions of the 3D vasculature. For that reason, the authors find a realistic initialization of the 3D volume using the Unet which implicitly introduces prior knowledge from the training data distribution. Then, starting from this initialization, the volume is further optimized with respect to data consistency and vessel connectivity. This increases the consistency of the final solution with the two measured projections. The connectivity regularizer also seems to be effective in closing missing segments of the vessels which is beneficial for downstream task (vessel analysis).
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
In the introduction, the authors claim that “DSA is often acquired as a set of anterior-posterior (AP) and lateral (L) projections of the vascular network.” with no further explanation or reference. I am mostly aware of single view 2D DSA acquisitions or full 3D acquisitions. I understand that acquiring just two projections instead of the full short scan might save time and dose, but I would be interested in a more elaborate describtion of the intended clinical use case and availability of such two-view DSA acquisitions in current clinical practise. For example, is there a workflow-based argument to restrict the number of projection to exactly 2 which is the hardest and most ill-posed case? I do not clearly understand the clinical purpose of this method in the current form of the paper. Technically, my main concern is the construction of the ground truth volumes as binary segmentation masks of the vasculature. In a real system, the projection images after subtraction will not be projections of a binary volume. Instead the measured absorption of the contrast agend filled vessels is not perfectly homogeneous and left-over artifacts will remain due to suboptimal alignment of filled and non-filled scan. With the idealized setting simulated in this study, the projection images are essentially perfect pathlength images which means that effects such as vessel foreshortening can easily be quantified because the vessels are assumed to have a constant gray value everywhere. This makes the problem easier since it constrains the possible solutions for the 3D volume. Further, rather small volumes are used to represent the vasculature which limits the minimal size reconstructed vessels. The authors mention the use of SDFs, but this is rather unclear. This aspect is not sufficiently addressed in the current version of the paper.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
The clarity of the decription of some aspects of the method could be improved, e.g., the section about clinical realism as well as the use of SDFs is not clear to me. Further, the projection geometry used for forward and backprojection operations is not mentioned at all.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The iterative finetuning of the Unet-based solution seems to be an effective step ensuring data consistency. Especially the connectivity prior shows promising results (e.g., Fig.3). To me, the clinical motivation of this method is not quite clear, i.e., is there already two-view DSA procedures likes this? How would the clincial workflow change? Why would we not take indermediate steps instead of going from a full 3D acquisition to 2 views only? Furthermore, it should be clarified why the modeling of the ground-truth 3D vasculature as binary volumes is not over-simplified. The manuscript itself could be improved in some places, e.g., in Eq. 3, I do not see the negative sum or normalization that is mentioned in the text. In the results, it took me a while to figure out, what “coarse” is referring to.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
From my point of view, the introduced connectivity loss is a positive contribution of this paper since it is simple and seems to be effective for the given task. Other parts of the method are mostly combinations of existing tools. My major concerns are clinical motivation and a simplified data setup as well as slightly unclear descriptions for parts of the method.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
The authors have clarified the clinical motivation of their method. They also acknowledge the simulation approach for their data setup as a limitation of the current study, and explain why limiting the ground-truth 3D vasculature to a binary volume is a simplification, but still captures the “essential complexity” of the problem.
Review #2
- Please describe the contribution of the paper
3D reconstruction of vessel structures could enhance diagnosis and treatment planning in angiography. To reduce radiation exposure, it is desirable to minimize shots, but reconstruction from biplanar images is a challenging task due to ambiguities in overlapping projections. The manuscript proposes a two-stage approach, in which the immmediate backprojection of orthogonal images is processed by a U-Net to reconstruct the vessel structure. The output is then used to initialize maximum a posteriori likelihood optimization, using connectivity as a regularizing prior. The method is evaluated using digitally reconstructed radiographs from vessell segmentations of 700 MRIs of the brain, significantly outperforming related methods with an 80% DICE score.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The formulation of biplanar reconstruction of vessel structures as a Bayesian optimization with a regularization prior, using a U-Net to initialize the optimization from a volume with the backprojections.
- The results show significantly improved performance over relevant baselines, some of which were re-implemented from scratch.
- An ablation study demonstrates the contribution of all components of the method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- There is no evaluation on real X-ray images, only digitally reconstructed radiographs generated from MRI segmentations of vessel structures.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Do you have any additional comments regarding the paper’s reproducibility?
None.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
I quite liked this paper, but my main concern has to do with the lack of evaluation on real images. The U-Net on its own is insufficient for adequate reconstruction, but the optimization process here benefits from the fact that the DRRs are generated from exactly the same process used in reconstruction, namely projection through a volumetric segmentation. That is, it is possible to exactly recreate the input images from the vessel reconstruction. This is not immediately true, given two real fluoroscopic images obtained during digital subtraction angiography. The evaluation here is understandable, given the lack of real paired projection-3D volume datasets, but qualitative analysis should still be possible from unpaired datasets.
Additional comments:
- Eq 3 could be simplified by defining the neighborhood of voxel x. The sum of the neighboring voxels is a straightforward concept that is burdened here by notation.
Typos:
- Typo on page 2: “who that”
- Section 2, Par 1: “figure 1” should be capitalized.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This is overall a very promising approach, which is unfortunately not validated on real fluoroscopic images. The problem of biplanar reconstruction of vessel structures is well motivated, but so far comparable methods have fallen short due to the ambiguities that arise from overlapping projections. The use of a deep neural network here to initialize an optimization of the maximum a posteriori likelihood is novel and outperforms previous methods. However, due to the lack of evaluation on real images, it is highly uncertain whether this is likely to succeed. The optimization is based on MSE reprojection error through the reconstructed vessel structure, which is exactly the same process by which the testing images were created. A qualitative analysis of results obtained from real DSA images may alleviate these concerns.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Accept — should be accepted, independent of rebuttal (5)
- [Post rebuttal] Please justify your decision
The rebuttal outlines a path forward to validation on real data in a future journal extension, which will be added to the conclusion. Given the related work that validates in a similar manner, the work should be of interest at MICCAI.
Review #3
- Please describe the contribution of the paper
Visualizing vasculature in 3D in real time is important during invasive procedures (brain, in this case). However, a fully 3D imaging is not logistically feasible in such a scenario. This paper proposes a method to provide high-quality 3D reconstruction from only two orthogonal 2D projection views (in MRA?). Targeted application is Digital Subtraction Angiography. It uses a pipeline of backprojection, subsequently denoised with a trained UNet model, and further iterative reconstruction. The procedure is sound and it achieves a significant improvement over state of the art methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The work has a high clinical impact.
- Especially, the follow up Max-a-posteriori update seems quite innovative and the ablation study without this step shows its effectiveness. The authors claim it is particularly useful in tracking vascular junctions (Fig 3).
- Training with only simulation data attains zero-shot learning flavor.
- Multiple interesting tools are used. For example, using Ising model as connectivity prior for the tubular objects (vasculature) is quite appropriate. Another example is borrowing the concept of Signed Distance Fields from computer graphics.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Acronyms for the modalities should be introduced, e.g., TOF-MRA.
- Ground truth for validation is not very clear. Are they simulated, not real? If so, that puts a damper on the method’s cinical transition, and should be discussed in the conclusion. If they are not simulated, How the two projection views are obtained from the IXI dataset? Are they available there?
- Achieving clinical realism by mirroring half the vasculature (over full brain?) appears dubious to me.
- How does SDF improve GPU utilization?
- Acronym DRA in the abstract is not yet introduced.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Very well written paper. Above weaknesses can perhaps be addressed in the rebuttal.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
A 3D image from two orthogonal camera should be very useful during cerebral angiography. Methodology seems sound.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Accept — should be accepted, independent of rebuttal (5)
- [Post rebuttal] Please justify your decision
I am not sure if the post-rebuttal edit has taken place. I am new to the MICCAI review process. If not, the authors may like follow my suggestion in the weakness section to improve the draft. The paper is strong and should be accepted with minor revisions in proceedings.
Author Feedback
We sincerely appreciate the recognition of our work from all reviewers for the “important” (R1) and “strongly ill-posed” problem (R4) of 3D brain vasculature reconstruction from only biplanar X-Rays. They found our method innovative (R1,R4,R5), “well motivated” (R5) with “high clinical impact” (R1) and large improvement. All reviewers praised our “sound” (R1), “novel” and “very-promising” (R5) two-step approach, with our connectivity prior “appropriate” (R1) and “effective” (R1,R4,R5), thoroughly validated in our experiments. Also, reviewers found our paper very clear (R1,R5) and “very well written” (R1). Our paper provides the critical first step to show that deep learning can be used to address this highly ill-posed problem. While there are still challenges ahead for clinical translation, our approach improves the results of previous works by a large margin. It lays a valuable foundation for the MICCAI community by demonstrating the potential of this approach. We are confident that minor revisions will address most reviewers’ concerns. VALIDATION REALISM (R1,R5) We acknowledge that our method has not yet been validated on clinical images. Like previous works, we used synthetic DSAs derived from real vasculatures due to the difficulty in obtaining paired bi-planar DSA and 3D vascular imaging datasets. We are finalizing a dataset to address this and extend our work, adapting our method for clinical DSA with vessel enhancement and real-to-synthetic mapping, to provide both qualitative and quantitative results. We will discuss this in the conclusion. Given the substantial space needed to describe this extended methodology and the main focus of this paper on biplanar reconstruction, we will present them in a journal extension. BINARY GROUND TRUTH (R4) We agree that DSAs can exhibit inhomogeneous contrast agent distribution. However, DSAs are captured over short sequences such that all vascular regions have been filled with maximum density across time. By employing Maximum Intensity Projection across these frames, we can aggregate the contrast agent responses so that we mitigate most of these inhomogeneities. Motion artifacts are typically small during the ~8 second acquisition for structural brain DSA, unlike longer fluoroscopic images acquired during interventions. Current imaging systems have also built-in motion corrections. We hence believe that our approach captures the essential complexity of this highly ill-posed problem, and that modeling the ground-truth 3D vasculatures as binary volumes from segmented MRA is not over-simplified. BI-PLANE WORKFLOW (R4) Reviewers R1, R5 recognized our high clinical potential. Biplane scanners are commonly used, available in most medium to large hospitals for a variety of specialties, which ensures high clinical applicability. Biplane DSAs, unlike 3D rotational scanners unsuitable for real-time interventions and single DSAs limited to simpler tasks, provide an ideal balance of speed, anatomical constraints, reduced cost, and radiation. They are widely used by interventionists but still present ambiguities, which drives our work. We chose to use only two projections because employing more would necessitate rotation, even partial, negating these advantages. Our method integrates with existing clinical workflows. RESOLUTION (R4) AND SDF (R1,R5) Due to GPU limitations, we deliberately used downsampled volumes to maintain network complexity and demonstrate feasibility with complex branching. We are exploring patch-based, cascade approaches or bigger GPUs to reconstruct smaller vessels. Using SDFs as an intermediate representation helps maintain thin vessel integrity and reduce artifacts during downsampling, unlike binary images which can cause structures to break apart or disappear. In the final version, we will clarify the above points, detail the acronyms, and correct the few typos to enhance our paper’s quality.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The presented method that intends to solve an ill-posed problem holds great clinical value. The rebuttal has addressed the major concerns of R4 and R5.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The presented method that intends to solve an ill-posed problem holds great clinical value. The rebuttal has addressed the major concerns of R4 and R5.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
N/A