List of Papers Browse by Subject Areas Author List
Abstract
3D reconstruction of the liver for volumetry is important for qualitative analysis and disease diagnosis. Liver volumetry using ultrasound (US) scans, although advantageous due to less acquisition time and safety, is challenging due to the inherent noisiness in US scans, blurry boundaries, and partial liver visibility. We address these challenges by using the segmentation masks of a few incomplete sagittal-plane US scans of the liver in conjunction with a statistical shape model (SSM) built using a set of CT scans of the liver. We compute the shape parameters needed to warp this canonical SSM to fit the US scans through a parametric regression network. The resulting 3D liver reconstruction is accurate and leads to automatic liver volume calculation. We evaluate the accuracy of the estimated liver volumes with respect to CT segmentation volumes using RMSE. Our volume computation is statistically much closer to the volume estimated using CT scans than the volume computed using Childs’ method by radiologists: p-value of 0.094 (> 0.05) says that there is no significant difference between CT segmentation volumes and ours in contrast to Childs’ method. We validate our method using investigations (ablation studies) on the US image resolution, the number of CT scans used for SSM, the number of principal components, and the number of input US scans. To the best of our knowledge, this is the first automatic liver volumetry system using a few incomplete US scans given a set of CT scans of livers for SSM. Code and models are available at https://diagnostics4u.github.io/
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3416_paper.pdf
SharedIt Link: https://rdcu.be/dV5Ej
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72104-5_42
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/3416_supp.pdf
Link to the Code Repository
https://diagnostics4u.github.io/
Link to the Dataset(s)
https://diagnostics4u.github.io/
BibTex
@InProceedings{Siv_LiverUSRecon_MICCAI2024,
author = { Sivayogaraj, Kaushalya and Guruge, Sahan I. T. and Liyanage, Udari A. and Udupihille, Jeevani J. and Jayasinghe, Saroj and Fernando, Gerard M. X. and Rodrigo, Ranga and Liyanaarachchi, Rukshani},
title = { { LiverUSRecon: Automatic 3D Reconstruction and Volumetry of the Liver with a Few Partial Ultrasound Scans } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15007},
month = {October},
page = {436 -- 445}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper introduces a method for 3D reconstruction of the liver. The uniqueness of this approach lies in using three segmentation masks of incomplete sagittal-plane ultrasound scans of the liver , coupled with a statistical shape model (SSM) established from a set of liver CT scans, to achieve 3D reconstruction of the liver. This method addresses the reconstruction challenges arising from inherent noise, boundary ambiguity, and limited visibility of portions of the liver in ultrasound scans. Additionally, the authors introduce a new database comprising 134 sets of ultrasound scans annotated by radiologists, providing an alternative pathway for subsequent 3D reconstruction tasks of the liver.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The authors use three ultrasound scan slices obtained from different positions to accomplish the 3D reconstruction and volume estimation tasks of the liver, demonstrating a high level of overall completeness in their work. 2.The figure clearly describes the framework proposed by the authors, and the paper provides a detailed description of the functions of the various modules within the framework.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1.The framework proposed by the authors lacks innovation. In comparison with （3D Organ Shape Reconstruction from Topogram Images | SpringerLink）, the main difference lies in the input data. However, this distinction is not sufficient to substantiate its novelty. Furthermore, the modules within the framework operate independently rather than undergoing end-to-end training. It remains unclear how to perform overall optimization, and this aspect requires clarification. 2.The medical segmentation models utilized in the experimental section are outdated, with the most recent being from 2021, lacking advancement. They should be compared with more recent models from the past two years, such as （LeViT-UNet: Make Faster Encoders with Transformer for Medical Image Segmentation | SpringerLink）&（H2Former: An Efficient Hierarchical Hybrid Transformer for Medical Image Segmentation | IEEE Journals & Magazine | IEEE Xplore）. 3.The visualization results presented in Figure 3 of the experimental section lack reliability. How can the actual difference between the reconstructed results (green) produced by the proposed method and the ground truth (yellow) be effectively demonstrated? The current visualization results suggest errors in both the green and yellow regions. A correct representation should use gradient colors to indicate the magnitude of errors in different regions. 4.The data in Table 1 of the supplementary material does not demonstrate the superiority of the proposed method over Childs’ method. On the contrary, the errors appear to be larger with the proposed method, which requires further explanation.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
Releasing the code and weights will greatly enhance the transparency and reproducibility of the research, allowing other researchers to validate and build upon the findings. This information should be made available not only to the authors and reviewers but also to the wider research community. By sharing these resources, the authors can contribute to the advancement of the field and foster collaboration among researchers.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
-
It is necessary to revise the tables in the experimental section to enhance the clarity of the paper. Specifically, in Table 1, comparisons with the latest segmentation models should be added. In Table 2, comparisons with other models, such as （3D Organ Shape Reconstruction from Topogram Images SpringerLink） should be included. Additionally, align the metrics and data and explain the evaluation criteria used in table. - In Table 3, the four ablation experiments do not seem closely related. It is necessary to clarify that the RMSE values are obtained from models under specific parameter settings. Additionally, an explanation is needed to regard why the number of principal components used for the Statistical Shape Model (SSM) is 50 rather than 20.
-
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The overall score for this paper is influenced by several major factors. Firstly, the lack of sufficient details and the absence of open-source code and pre-trained weights significantly impact the paper’s reproducibility. Secondly, the paper falls short in terms of conducting a comprehensive literature review and providing an adequate experimental comparison with related works. Furthermore, the presentation of experimental results is not reasonable, as it fails to demonstrate the quality of the reconstruction adequately.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The authors present a new methodology that combines statistical shape modeling (SSM) and deep learning to perform 3D reconstructions and volumetric analyses of the liver using only a few partial ultrasound (US) scans. Validation of the technique is done against CT-based measurements for segmentation and reconstruction accuracy.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
S1. The framework is interesting, using deep learning and shape modeling to minimize the need for extensive scanning.
S2. The technique is validated against standard CT measurements, showing high accuracy, which is critical for clinical application.
S3. The statistical approach and machine learning algorithms are well-integrated and appropriately detailed. There are various comparing methods as well.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
W1. The novelty of the study is questionable. The authors need to clarify exactly what their contribution is, compared to references [12] and [17], other than changing the sample type (using liver) and the instrument (using ultrasound).
W2. The limitations of this framework need to be clearly stated. For example, this method relies heavily on statistical shape models, which may not be suitable for patients with atypical liver morphologies. What are the implications of such a scenario?
W3. The author mentioned that ultrasound images are more prone to artifacts and noise. However, there is insufficient discussion about how variations in the quality of ultrasound images could affect the accuracy and reliability of the 3D reconstructions.
W4. It is not clear why the authors used only three ultrasound scans for this study. How would the accuracy change using more or fewer scans?
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
This paper could be improved by addressing the weaknesses and questions that were brought up.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The novelty of the paper is limited. It seems like it is a combination of other methods applied to a specific task.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
I would like to thank the authors for their rebuttal. They have addressed some of my concerns. Still the paper lack sufficient novelty. But it is a well-described study. I increase my score to 4.
Review #3
- Please describe the contribution of the paper
The paper proposed a 3D liver reconstruction using three sagittal plane US slices. It also introduced a new US liver database with 134 annotated CT scans. Experiments are performed to show the effectiveness of the method.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
(1) The method that uses just three sagittal plane US slices where the liver is only partially visible to reconstruct 3D liver and estimate volume is interesting and effective, according to their experimental results.
(2) The paper writing is good and the experiment result seems promising.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
(1) I think some existing methods can be used to help evaluate the effectiveness of the proposed dataset. The authors can design some task based on the proposed dataset.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
According to the authors’ checklist, I believe the paper has good reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
(1) Some tasks in 2D or 3D can be utilized to help evaluate the proposed dataset.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The overall idea of the paper is good and promising. It can be further improved as mention above.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
I think current version can be improved further with the help of all the reviews. I keep my score for now.
Author Feedback
We found the feedback to be thoughtful and constructive. Some provided future directions. We are grateful to the reviewers. We appreciate the comments on “uniqueness,” ability to address reconstruction (recons.) challenges, good readability, dataset of radiologist annotated paired CT-US liver scans, and evaluations that “show the effectiveness of the method.” Reproducibility (R1, R4): We will release the code and weights for further research. We will make the dataset available and an API for 3-D visualization of the liver. Novelty (R1, R4) with references to Balshova et al. 2019, [12], and [17] (R1): [17] uses 13 full-view CT slices of the left ventricle. [12] and Balshova et al. use x-rays taken from CT machines, with full view of the liver. We use only 3 partial US scans of the liver and use an SSM constructed from annotated CT scans. Our method, therefore, is novel and has high utility in the clinic. Clinics use repeated liver volume measurements. While CT scans are accurate, their use is limited by radiation exposure: US scans are used despite their noisiness and blurred boundaries. The liver’s large size and partial visibility in US make 3D recons. useful for radiologists. We agree that the individual components of our system are not new. The novelty is in the 3D liver recons. and volume computation using only 2 or 3 US scans, a significant improvement over the commonly used Childs’ method [3]. Contributions: i. 3D recons. and volume computation using only 2 or 3 unclear US scans in which the liver is only partially visible, ii. dataset with paired annotated CT and US scans of the liver (134 scans), iii. better liver 3D recons. accuracy (CD and MSD) compared to Ground Truth (GT) CT volumes and volume accuracy better than Childs (US liver volumetry with highest accuracy so far). End-to-End Training (R1): Our MLP and TransUNet segmenter are end-to-end trainable. We will have a fully DL based end-to-end trainable system by replacing the SSM with a DL module in future. Segmentation models “are outdated” (R1): and requests to compare with LeViT-UNet and H2Former. The model that we use is TransUNet (2021), a well-cited transformer-based segmenter. We ran tests with LeViT-UNet and H2Fornmer: RMSE slightly worsened. We conclude that our method is agnostic to the segmenter so long as it is transformer based, as stated on p. 5. Tab. 1 justifies TransUNet use. Also, we wish to state that segmen. is not our focus. Why Three US Scans (R4): Our radiologists have carefully selected 3 scan planes–midline, mid clavicular, and anterior axillary line–as they can be easily replicable. Childs et al. use 2 of the planes (except anterior axillary): ablations show even 2 US scans are satisfactory. Supp. material Tab. (R1): We have made a mistake in Tab. 1 (liver volumes calculated by various methods): we have interchanged the headings of Our Vol. and US Vol. We apologize. With this correction, our volumes can be seen to be closer to CT. Visualization (R1): “vis. in Fig. 3 lack reliability” as we have just overlapped the GT with our reconstruction. Despite visualizations, our recons. are accurate (low CD & MSD in Tab. 1). However, we agree and have re-created Fig. 1 with recons. error. Ablation Clogged (R1): This was to conserve space. RMSE is not much affected by no. of principal comp. We will add descriptions. US image noise effect (R4): We experimented by adding Gaussian noise and observed that it did not affect the volume much. We will further investigate. Literature review and comparisons (R1): We will cite the papers (e.g., 3D Organ Shape Recon.) and compare results . Evaluation of Dataset (R3): We will design evaluation tasks. Intra- and inter‐rater reliability studies will also be made available. Limitation on Atypical Livers (R4): We have not tested with atypical livers (fatty, Riedell’s lobe, fibrosis, cirrhosis) and are collecting datasets. Note that our dataset too has variations (liver sizes), and our system handles them.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
N/A
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The authors has done a great job to improve their score by sufficiently answering all questions raised by reviewers. Well done!
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The authors has done a great job to improve their score by sufficiently answering all questions raised by reviewers. Well done!