List of Papers Browse by Subject Areas Author List
Abstract
3D pose estimation from a 2D cross-sectional view enables healthcare professionals to navigate through the 3D space, and such techniques initiate automatic guidance in many image-guided radiology applications. In this work, we investigate how estimating 3D fetal pose from freehand 2D ultrasound scanning can guide a sonographer to locate a head standard plane. Fetal head pose is estimated by the proposed Pose-GuideNet, a novel 2D/3D registration approach to align freehand 2D ultrasound to a 3D anatomical atlas without the acquisition of 3D ultrasound. To facilitate the 2D to 3D cross-dimensional projection, we exploit the prior knowledge in the atlas to align the standard plane frame in a freehand scan. A semantic-aware contrastive-based approach is further proposed to align the frames that are off standard planes based on their anatomical similarity. In the experiment, we enhance the existing assessment of freehand image localization by comparing the transformation of its estimated pose towards standard plane with the corresponding probe motion, which reflects the actual view change in 3D anatomy. Extensive results on two clinical head biometry tasks show that Pose-GuideNet not only accurately predicts pose but also successfully predicts the direction of the fetal head. Evaluations with probe motions further demonstrate the feasibility of adopting Pose-GuideNet for freehand ultrasound-assisted navigation in a sensor-free environment.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3555_paper.pdf
SharedIt Link: https://rdcu.be/dY6jK
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72083-3_65
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/3555_supp.pdf
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Men_PoseGuideNet_MICCAI2024,
author = { Men, Qianhui and Guo, Xiaoqing and Papageorghiou, Aris T. and Noble, J. Alison},
title = { { Pose-GuideNet: Automatic Scanning Guidance for Fetal Head Ultrasound from Pose Estimation } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15004},
month = {October},
page = {700 -- 710}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper proposes an approach to estimate the pose of a ultrasound frame with respect to a 3D fetal head atlas. The model is pretrained using slices sampled from the atlas and is then finetuned on real data. The approach is intended to be used as guidance to find standard clinical planes.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The idea of pretraining the model on simulated data and finetuning it on real data is valid
- The proposed approach effectively makes use of existing data without pose annotations
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The experimental results clearly favor the “supervised” methods. But the paper doesn’t explain why the proposed method should nonetheless be preferred.
- The task of the model is to predict a pose, so I would expect some pose distance to be measured during validation. It is not clear why KL divergence is used instead.
- The atlas has a central role in this work but there are no details about how it was constructed
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Please work on enhancing the clarity of the paper. At its present state, it is challenging to discern the reasoning behind certain choices and to determine the validity of the method’s evaluation.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the proposed method appears to be interesting, the clarity of the paper needs to be improved. Furthermore, the choices of validation metrics needs to be explained, in order to show the soundness of the results
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
The authors have addressed my concerns and provided clarifications, therefore I am changing my decision to “weak accept”.
-
Changing the terms ‘supervised’ and ‘unsupervised’ to ‘sensor-free’ and ‘sensor-based’ avoids the confusion and makes the paper and the value of the contribution clearer.
-
The authors have justified the use of KLD for evaluation due to the different coordinate systems of the probe and the fetal head positioning. They explained that the motion dynamics, rather than absolute distances, are more relevant in their context.
-
The authors have provided information about the 3D fetal head atlas.
-
Review #2
- Please describe the contribution of the paper
The main contribution of this paper is the development of an automatic scanning guidance system for fetal head ultrasound. This unsupervised deep-learning-based technique accurately estimates the pose to align freehand 2D ultrasound images with a 3D anatomical space, eliminating the need for specialized transducers for 3D imaging.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main strength of this paper is its clear description of the research purpose, the proposed method, and the experimental processes and results that validate their method. Their framework utilizes pose localization on a 3D Atlas and cross-dimension, cross-modality alignment on 2D ultrasound images, differing from the approaches of Yeung et al. It specifically focuses on TVP and TCP, which are areas medical professionals typically examine to assess fetal head development. The experimental results demonstrate that their proposed method achieves accurate pose estimation.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The manuscript lacks detailed descriptions of the encoder structures (encoderE and SemanticE), which affects its reproducibility. Although the results of the unsupervised method approach those of supervised methods, they do not surpass them. Considering the minimal motion involved in fetal brain applications, acquiring multiple frames to obtain a reference image for a supervised framework is not challenging. It is unclear why an unsupervised approach is preferred here.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Providing a more detailed description of the network structure would greatly enhance the paper’s reproducibility. I am curious whether the method would also be effective in atypical motion scenarios beyond the standard TVP or TCP scan techniques.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the formulation itself is not novel, it distinguishes itself from prior works through its approach. The paper successfully demonstrates experimental feasibility and provides a robust evaluation, proving its effectiveness in practical applications. This combination of demonstrated utility and strong evaluation supports a recommendation for weak acceptance.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
The authors addressed issues. My decision remains unchanged.
Review #3
- Please describe the contribution of the paper
The authors present Pose-GuideNet, a model that can help guide fetal head biometry acquisition. In particular, the model estimates the 3D pose from freehand 2D ultrasound images as well as its transformation towards the standard plane in 3D anatomy, effectively avoiding a manual registration. The authors introduce a supervised geometric-guidance and an unsupervised semantic awareness for in-plane and out-of-plane alignments to improve the automatic registration to the standard plane. The proposed model outperforms existing unsupervised literature on the described dataset that focuses on TVP and TCP scans.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Novelty - including both geometric and semantic information on the fetal brain can help guide the model in its image registration to the standard plane automation. Performance - through the novel methodology, the authors consistently improve the state-of-the-art for unsupervised models. Clarity - the paper is well-thought and presented extremely well.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Missing or misleading information in the experimental section: the manuscript is extremely clear, however, information on the hardware used to run the proposed approach should be reported to help with reproducibility. Furthermore, Table 1 can be misleading since the geometric alignment is actually supervised as an approach.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
The authors describe the acquisition methodology. Therefore, given one is able to collect said images, the implemented model can be re-implemented and applied to the new collection.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The manuscript is well-organized and presented in a clear way. The only misleading part is associated with Table 1, where the geometric in-plane guidance is actually supervised. On a different note, Fig.3 could be improved by removing the ‘…’ on the x axis as they are not representing anything. One last suggestion: it would be interesting to apply your methodology to other brain images requiring registration to see how well the model behaves.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The manuscript is well-organized and the authors were thorough in their research. The model is also novel through its geometrics and semantic fetal brain characteristics and improves the sota for unsupervised approaches.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Accept — should be accepted, independent of rebuttal (5)
- [Post rebuttal] Please justify your decision
The authors addressed all raised issues. My decision remains unchanged as accept.
Author Feedback
We thank the reviewers for their valuable comments and suggestions on our paper. Here are the responses:
R1R3R4 - “About the clarification of ‘supervised’ and ‘unsupervised’ methods used in Table 1”: We apologise for the confusion. In Table 1, ‘supervised’ and ‘unsupervised’ are not referring to supervised and unsupervised learning for model training. ‘supervised’ in Table 1 refers to the approaches using an external motion sensor as the supervision signal for pose estimation, and ‘unsupervised’ refers to sensor-free approaches. Sensor-free approach is usually preferred as it is more applicable in a clinical environment without the need of an external sensor to record the 3D motion. To make it clear, the terminology used in the result Table will be changed to ‘sensor-free methods’ and ‘sensor-based methods’.
R1 - “..lacks detailed descriptions of the encoder structures (PoseE and SemanticE).”: The pose encoder PoseE employs ResNeXt50 [1] to learn the pose embedding and SemanticE employs SonoNet [2] to learn the semantics of fetal US. This will be clarified in the main text. - “Whether the method would be effective in a typical motion scenario beyond TVP and TCP”: The method can be readily extended to other cases where a target plane is defined. However, if the search is not for a neurosonography plane, a 3D atlas specific to the anatomy under examination would be required.
R3 - “Hardware information should be reported”: The experiments were run with PyTorch 1.10.1 on a 32GB NVIDIA Tesla V100 GPU. - “Fig3 could be improved by removing ‘…’” Thanks for suggesting this and we will remove the ellipses. - suggestion “it would be interesting to see applying to other brain images..”: Since the method is designed for US images, directly applying it to other types of brain images (e.g. MRI, CT) may face cross-modality challenges. However, the method could be adapted to analysis of fetal brain MRI scans given that a MRI-based 3D atlas (e.g. [3]) is available.
R4 - “Evaluation on KLD rather than pose distance”: The method performance is measured based on the probe’s orientation. However, since the probe and the 3D fetal head positioning are operated under different coordinate systems, their angular or positional distances are not directly comparable. Despite this, their motion dynamics during scan are related: a significant transformation in 3D fetal head will correspond to a relatively large probe movement. We thus evaluate on the distribution of the motion dynamics during scan using the statistical metric (i.e., KLD), rather than the absolute pose or angular distance. We will clarify this in the manuscript. -“missing details about atlas”: The 3D fetal head atlas used is open-sourced from [4]. 1059 US brain volumes collected from 899 fetuses across 8 countries were used to generate the 3D fetal head atlases for different gestational ages. In this paper, we use the atlas at 20 week pregnancy as the median description of the fetal head in the second trimester.
[1] Aggregated residual transformations for deep neural networks, CVPR 2017 [2] SonoNet: real-time detection and localisation of fetal standard scan planes in freehand ultrasound, TMI 2017 [3] A normative spatiotemporal MRI atlas of the fetal brain for automatic segmentation and analysis of early brain growth, Scientific reports 2017 [4] Normative spatiotemporal fetal brain maturation with satisfactory development at 2 years, Nature 2023
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The paper has minor deficiencies that were addressed in the rebuttal and can be incorporated in the final version.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The paper has minor deficiencies that were addressed in the rebuttal and can be incorporated in the final version.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
Reviewers agreed to accept the paper. Most concerns have been addressed in the rebuttal. Additional clarifications need to be included in the revised manuscript.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
Reviewers agreed to accept the paper. Most concerns have been addressed in the rebuttal. Additional clarifications need to be included in the revised manuscript.