List of Papers Browse by Subject Areas Author List
Abstract
4D Computed Tomography (4DCT) is widely used for many clinical applications such as radiotherapy treatment planning, PET and ventilation imaging. However, common 4DCT methods reconstruct multiple breath cycles into a single, arbitrary breath cycle which can lead to various artefacts, impacting the downstream clinical applications. Surrogate driven motion models can estimate continuous variable motion across multiple cycles based on CT segments `unsorted’ from 4DCT, but it requires respiration surrogate signals with strong correlation to the internal motion, which are not always available. The method proposed in this study eliminates such dependency by adapting the hyper-gradient method to the optimization of surrogate signals as hyper-parameters, while achieving better or comparable performance, as demonstrated on digital phantom simulations and real patient data. Our method produces a high-quality motion-compensated image together with estimates of the motion, including breath-to-breath variability, throughout the image acquisition. Our method has the potential to improve downstream clinical applications, and also enables retrospective analysis of open access 4DCT dataset where no respiration signals are stored. Code is available at https://github.com/Yuliang-Huang/4DCT-irregular-motion.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3437_paper.pdf
SharedIt Link: https://rdcu.be/dVZiT
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72378-0_55
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/3437_supp.zip
Link to the Code Repository
https://github.com/Yuliang-Huang/4DCT-irregular-motion
Link to the Dataset(s)
https://doi.org/10.5522/04/26132077.v1
BibTex
@InProceedings{Hua_Resolving_MICCAI2024,
author = { Huang, Yuliang and Eiben, Bjoern and Thielemans, Kris and McClelland, Jamie R.},
title = { { Resolving Variable Respiratory Motion From Unsorted 4D Computed Tomography } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15001},
month = {October},
page = {588 -- 597}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper aims to incorporate surrogate signals into motion corrected 4DCT reconstruction. This should avoid sorting artefacts due to temporal slice misalignment. Some preliminary comparisons on phantom data for quantitative and public (real) 4DCT for qualitative evaluation are presented.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The use of “hyper”-gradients of fitted/measured surrogates to inform the deformable motion estimation is reasonably well motivated
- Ordering artefacts are commonly place in clinical practice and while directly obtaining respiratory signals, estimating a proxy is practical
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The comparison methods are very limited, only within the same method and a single groupwise registration with little details
- The method itself is rather simple and it does not directly become clear, why e.g. not volumetric estimates of lung (segmentation) volumes or airway inflation would provide a more accurate surrogate modelling compared to the 1D temporal gradient
- The evaluation is severely limited, only a small phantom experiment and visual results on real data without e.g. segmentation or landmark quality metrics.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
More details are required for the compared registration method and how hyper-parameters were chosen.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Since a more elaborate and detailed experimental validation as well as a more sophisticated method that e.g. jointly estimates lung volumes and motion are not feasible during rebuttal, I suggest to focus on explaining the missing details and providing a road-map to a more complete paper draft.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Reject — should be rejected, independent of rebuttal (2)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
the work is in my opinion very preliminary and not ready for publication at MICCAI.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
In response to the problem of irregular breathing, the paper proposes a method using hyper-gradient to solve the artifact problem in 4DCT image reconstruction.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper uses a hyper-gradient method to replace the surrogate signals required by the previous method, which has good compatibility with 4DCT images with irregular breathing, and has been verified through experiments.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
（1）Figure 3 is 2 pages away from the first mentioned location. （2）The comparison methods are relatively old and there is no comparison with recent work. （3）Figure 3 still feels like evidence of artifact elimination, but the author’s explanation is about estimation of respiratory motion, and this aspect still needs further explanation. （4）The explanation of the dataset corresponding to Table 1 is not detailed enough. What is the scale of the dataset?
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
It is better to open source the data sets and code, especially the quantitative corresponding dataset.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Please refer to the weaknesses.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is clear and logical. The proposed method was verified with three data sets, but it lacks comparison of methods in recent years.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The paper described a motion compensated reconstruction technique to generate 4D-CT images free of motion artifacts. The motion model was constructed as a bspline model fitted by time-dependent motion curves. The bspline model, the motion curves, and the reference volume were iteratively updated via a motion-compensated framework.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Compared with the previous methods, the developed method can update the surrogate signal during the motion estimation, and can achieve surrogate-free reconstruction if such signal is not available.
- The method has shown promising results in both phantom and patient studies.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The hypergradient implementation is interesting. However, it is unclear why the paper chose to view the surrogate signal as a hyper-parameter, rather than optimizing it parallelly with the b-spline model. Does using the hypergradient optimization offer better convergence? Please clarify.
- The paper mentioned that the surrogate signals were approximated as cos and sin functions derived from the respiration phases. Is it the same for the XCAT and the patient cases? And how sensitive is the algorithm to the initialization functions? Will a non-sinusoidal initialization work?
- Is the motion of the three spatial directions initialized as the same curves? During the optimization are they allowed to vary? The motion in the lateral direction is usually quite different from that in the superior-inferior or anterior-posterior directions.
- It is unclear whether the derived motion curves are accurate, especially for the CT slices with limited motion or of similar appearances between phases (for instance, those located near the superior and inferior edges). Can the curves be correctly learned at these locations? In addition, since the phantom simulation study has the ground truth available, a comparison between the true and solved curves should be performed.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The authors should better explain the rationale of using the hypergradient approach, evaluate the sensitivity of this method to initialization conditions, and calculate the accuracy of the solved motion curves against the true curves (for the phantom study).
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper solved a common clinical problem with a well-designed approach, by deriving a motion model and the image volume concurrently.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
We thank the reviewers for their constructive feedback. They appreciate that our paper is solving a common clinical problem (R1&R5&R6), is “clear and logical, verified though experiments” (R1) and “shows promising results” “with a well-designed approach” (R6). We address the main criticism below: 1.Comparison methods (R1&R5): We would like to highlight that our method does something that very few other methods can do: estimate variable motion based on unsorted CT slabs at all timepoints during 4DCT acquisition. Most other methods focus on improving sorted 4DCT images and estimate images/motion that represent one arbitrary breathing cycle. This makes meaningful comparison with other methods difficult. We are only aware of surrogate-driven methods that aim to model variable motion and make for meaningful comparisons. We also compared to the simple groupwise registration method despite it not modelling variable motion, as this or similar methods are in clinical use at some centers. 2.Reproducibility (R1&R5&R6): We will open source the software, phantom data and relevant scripts to reproduce our results prior to publication.
For more specific comments of each reviewer, To R1: 1.Fig3 explanation: Fig3 shows our method can estimate variable motion as the mid and right images show the end-inhalation phase from two different breaths, and you can see that the diaphragm height differs between the images. 2.Dataset details: The phantom dataset consisted of images at 182 timepoints, each with size of 355x280x115 voxels and resolution of 1x1x3mm.
To R5: 1.Details of groupwise registration: It was performed as described in ref23. NiftyReg was used for registration. Based on experience default parameters were used except for velocity field transformation and SSD similarity metric. 2.Simple method: We think a simple method is preferable to an overly complex method if the simple method can produce good results as ours does. The reviewer suggests estimating lung volume/airway inflation is better, but only a few CT slices are available at each timepoint, which is insufficient to estimate lung volume or airway inflation. 3.Only visual results on real data: For real data only a few slices are acquired at each timepoint and the ground-truth volumes are unknown. This makes quantitative assessment using segmentations or landmarks very challenging. This is why we also evaluated with phantom data, where the ground truth volumes are known facilitating quantitative evaluation. Visual results of real data also clearly show that artefacts have been removed, and the supplementary videos show the estimated motion is plausible.
To R6: 1.Clarify hypergradient: We further clarify discussion on page 8 of our paper as below. Simultaneous optimization can be unstable due to the interplay between S and C (Eq 3). Alternative direction methods are commonly used to make the optimization more stable, but these require the gradient to be calculated separately for S and C. The hypergradient method allows us to reuse most of the gradient calculation, i.e. gradient up to M, for both S and C. 2.Initializing surrogates: The surrogate free method uses cos and sin to initialize the surrogates for both XCAT and real data. The surrogate optimized method uses the measured chest signal to initialize the surrogates. The surrogate optimized method has the best results (table 1), showing the method is sensitive to the initialization, but the surrogate free method can still achieve good results when a measured signal is not available. 3.Motion curve accuracy: Comparison between GT and solved motion curves is easy and will be done in the updated version. We agree that the motion estimates will be less accurate for CT slices with limited motion, and this is why the surrogate free method has higher RMSE than the surrogate driven/optimized methods (table 1). These slices can be detected and the motion estimates for these timepoints can be discarded prior to downstream applications.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The authors have adequately addressed all the concerns.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The authors have adequately addressed all the concerns.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
A clear and well-prepared paper that describes an interesting solution for a problem that might not be as well-known as the typical benchmarks. This will be of interest to the community.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
A clear and well-prepared paper that describes an interesting solution for a problem that might not be as well-known as the typical benchmarks. This will be of interest to the community.