List of Papers Browse by Subject Areas Author List
Abstract
Sensorless freehand 3D ultrasound (US) reconstruction poses a significant challenge, yet it holds considerable importance in improving the accessibility of 3D US applications in clinics. Current mainstream solutions, relying on inertial measurement units or deep learning, encounter issues like cumulative drift. To overcome these limitations, we present a novel sensorless 3D US solution with two key contributions. Firstly, we develop a novel coupling pad for 3D US, which can be seamlessly integrated into the conventional 2D US scanning process. This pad, featuring 3 N-shaped lines, provides 3D spatial information without relying on external tracking devices. Secondly, we introduce a coarse-to-fine optimization method for calculating poses of sequential 2D US images. The optimization begins with a rough estimation of poses and undergoes refinement using a distance-topology discrepancy reduction strategy. The proposed method is validated by both simulation and practical phantom studies, demonstrating its superior performance compared to state-of-the-art methods and good accuracy in 3D US reconstruction.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1045_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/1045_supp.pdf
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Dai_Advancing_MICCAI2024,
author = { Dai, Ling and Zhao, Kaitao and Li, Zhongyu and Zhu, Jihua and Liang, Libin},
title = { { Advancing Sensorless Freehand 3D Ultrasound Reconstruction with a Novel Coupling Pad } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15004},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
To summarize, a new tracking method and device is proposed for 3D ultrasound reconstruction from a set of 2D ultrasound images created by sweeping the probe over a region of interest. The novelty is to use a gel-based phantom between the probe and the skin that contains multiple “N-wires” whose geometry provides a way to calculate the pose of the probe at each image location. The pose is calculated using a similar-triangles geometry calculation from the dot-like depictions of the wires as they cross through each ultrasound image.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The strength of the paper is the method’s ability to do ultrasound tracking without any additional sensors, so it can be applied to any ultrasound machine in market with all the processing done on the imaging output. It can also be done in real-time for live pose-estimation of the probe. The inspiration is that the appearance of the dots gives a unique configuration for different translations and rotations of the ultrasound probe. The same N-wire calibration has been used for decades in offline calibration of tracked probes (probes with an additional tracker such as optical markers seen in cameras, electromagnetic trackers or inertial-measurement units) that determine the offsets between the tracker and the ultrasound image. So instead of an external tracker and calibration offline before tracking, this paper puts the N-wires on the subject’s skin and leaves it there during probe scanning. Tests are performed in simulation and phantom studies but not any tests on human subjects. In short, this paper combines a number of existing ideas in order to remove the need for external trackers or, usually with less accuracy, the need to do image-based pose estimation directly from the ultrasound imaging data. The N-wire phantom is also easy to make and inexpensive making it accessible to all. The method is susceptible to noise so additional regularization terms are added to optimization to make the tracking more reliable and accurate. The method does appear to work reliably and another strength is that the method does not depend on assumptions about the scanning, i.e. no assumption on probe movement speed, angular range, translation range, as long as the N-wires still appear clearly in the top of the ultrasound image. It also looks easy for ultrasound operators with no need for careful placement (except for “this side up” on the N-wire device).
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The main weakness, and this is not mentioned or addressed in the paper, is the degradation of ultrasound image quality that the N-wire device adds. Ultrasound, like all imaging modalities, is often pushed to its limits of resolution, contrast and depth penetration. Adding a relatively thick gel-like device between the probe and the body will reduce the ultrasound acoustic energy transmitted into the body that will reduce depth penetration, increase beam spreading that will reduce resolution, and add additional sources of error from multi-path reflections, refraction and speed of sound variations that will affect both resolution and contrast. Sacrificing image quality to perform tracking is a sacrifice few clinical applications are willing to make. Operators have always had access to “stand off pads” which are similar to the N-wire device proposed here and are used to put superficial (e.g. near the skin surface) features down into the middle of the image where they are better seen. But sonographers do not use such stand off pads when not necessary because o the loss of image quality. The additional of wires in the authors’ device adds an unknown further degradation that should be quantified. The second main weakness, and this is not mentioned or addressed either, is the fact that such an N-wire device must be pliable to conform to the various curvatures of the human body, but such pliability will affect the location of the N-wires, and their known locations are essential to the accuracy of tracking.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
To be reproducible, other researchers will need to build their own N-wire device. Many groups already have done so because they use N-wires for calibration (as explained above) but they are usually in rigid boxes so some redesign is needed to make them removable from the box and placed on the skin of the subject. It would be helpful if the authors provided details on the fabrication steps of their device, not just ingredients, and how to accurately place the wires. Otherwise the mathematics are fairly easy to reproduce.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Detailed comments start with the need to quantify the loss of image quality from the proposed N-wire device. This could be done fairly easily using an existing quality assurance phantom for point targets and contrast targets. Usual metrics of point-spread function, contrast to noise ratio, maximum depth can all be compared with and without the N-wire device. If it can be demonstrated that the loss of image quality is small, especially if a specific clinical application is proposed with it specific requirements, then it would help make this a credible solution for that application. Another limitation is that no tests are shown for human subjects. Figure 2 does appear to show real tissue, but quality of the 3D reconstruction is not provided (nor any mention of ethics approval for use on human subjects).
It is also necessary to answer how much the N-wires in the proposed device move as the device is pliably placed against a non-flat surface. What errors does this introduction into the tracking? What gel and wire material are used and how flexible is the N-wire device?
The tests in simulations and phantoms are good with beautiful graphics and illustrations to clearly show what was done. I have only minor questions: what is the full list of variables that are estimated in the pose-tracking? Is it 6 degrees-of-freedom (three translations and three rotations) or are there additional variables such as the scale factor in the ultrasound image (pixel size which may be affected by the speed of sound) that is also often estimated in N-wire calibration? I appreciate the need to add regularization of both “distance” and “topology” but it is not clear how to best weight these two objective functions. I do appreciate the breakdown of the results into sub-categories that test whether each of these are needed. Doing so gives confidence that the method is working reliably and needs both such optimization functions.
A challenge facing all such research on calibration and tracking is the need and definition of the gold standard for the true pose of the probe. I appreciate the authors building a test phantom to obtain this gold standard. It should be noted that the gold standard is again an N-wire phantom, so their N-wire tracking (in the near field near the probe face) is being tested again N-wire tracking in the far field (near the middle of the image). Errors in the gold standard should be reported if possible. If room, it would also be good to report the range of motion (translations and rotations) of the probe during the tests since a larger ranges stresses the method more than smaller ranges and allows better comparision to other papers. Lastly, it is not clear what effect the choice of additional parameter d_thr in equation 7 has on the results.
As a less important point, the authors could state that a limitation of using the N-wire device for tracking is that it only provides relative tracking of the images to each other; if tracking is needed to place the ultrasound images in an external coordinate system such as in image-guided surgical systems, then further steps (such as adding additional fiducials) are needed to localize the N-wire device too.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Although there are questions remaining about how well the device will work on human subjects given the need to deform the pliable N-wire device against the skin surface (adding errors) and how much the image quality is degraded by the N-wire device, the basic method is sound and it should work in some applications. 3D ultrasound is an often useful extension over 2D so this should be of interest to others at MICCAI.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The paper’s main contribution is in the introduction of a novel concept for 3D ultrasound image reconstruction, that eliminates the need for external tracking systems or sensors. This approach has also been validated in the manuscript through both simulations and physical experiments.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The concept of performing a 3D ultrasound reconstruction without any tracking systems or external sensors, and using only an N-wire phantom (or gel pad as authors refer to it in the manuscript) is an interesting concept that I haven’t seen before. Wire phantoms are typically used for performing ultrasound calibration, but not been used for reconstruction purposes. They’re pretty intuitive, cheap to fabricate, portable, and the reconstruction solution is formulated using a simple optimization problem that is independent of what is actually being imaged under ultrasound (which is a big advantage compared to deep learning solutions where it can prove to be difficult to extend to previously unseen environments). The gel pad also opens up the possibility to incorporate different wire patterns for different applications to potentially even further improve the reconstruction quality. Overall, the idea is interesting and the authors did a good job proving its preliminary feasibility (showing that it’s actually a promising solution).
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The first problem with the manuscript is the lack of clinical motivation for this specific solution, which raises questions about the clinical relevance and feasibility. 3D ultrasound reconstruction certainly remains an open problem, but the solution proposed by the authors is unlikely to work for all types of ultrasound work. Some organs that are looked at under ultrasound may not require moving the probe as much as is done in the manuscript. For example in eFAST scans, the probe is typically positioned in a single location at a time, and US images are collected using a sweeping motion (so only angular motion, no linear motion). In this scenario, a wire phantom may not be sufficient to get a good quality reconstruction. So understanding when can this method be adopted in practice is really important.
The second problem is the effect of the wired gel pad on the quality of ultrasound images. Ultrasound is pretty sensitive to changes in medium it traverses (now we’re adding extra boundaries and thickness), and can produce artifacts with the wires inside the gel pad. If you’re looking for certain artifacts (like the “bat” sign in lung ultrasound for instance), how are these wires going to affect the quality of the data? Other concerns are maintaining contact with the patient body at all time which is unlikely to be flat as is the gel pad. Can the wires (layered too) end up obstructing some critical structures the physician is trying to image?
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
Authors provide enough details on the optimization formulation and libraries used, although an open source repository would greatly help the community :)
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Here are some more detailed comments on the manuscript (food for thought?):
-
It would be interesting sometime in the future to look at a comparative study conducted with physicians to perform a 3D reconstruction of ultrasound data using the wired gel pad and more traditional methods such as using optical trackers or affixing the probe to a robotic system. Although deep learning is making leaps in the field, traditional methods are still somewhat the gold standard to getting things to “actually work”. Would also be good to get the physicians’ input on this.
-
How many ultrasound images are actually needed to be able to just get “a” reconstruction, and how spaced out can these images be?
- There are some logistical issues that I can see happening with the pad.
- What if it slides a bit during the ultrasound scan? There will be ultrasound gel everywhere creating lots of slippery surfaces.
- What happens when the pad moves with the patient? For example during breathing motion, patient movement, etc…
- Depending on the operator, ultrasound scans are collected with different pressure applied onto the surface. Can the wires deform at all because of this?
-
It is unclear in the manuscript how the gel pad wires are detected in the US images (manually?). Should this become a solution this obviously would need to be automated.
-
A good reconstruction will likely depend on the scaling factors used for the US image (mm/pixel), which can be different in the x and y directions, as well as dependent on the selected depth of the US image.
-
The comparative analysis in Table 1 is a bit confusing. DCL-Net and RecON have already been compared to each other in ref. [9]. DCL-Net and MoNet have also been compared to each other in ref. [7]. DCL-Net seems to underperform every single time. Not clear why that network was selected for comparison in the current manuscript.
-
The metrics used for the key parts (Fig. 3) need to be supplemented by a few more to take into account different dimensions, such as the height of some key features, etc.. The diameter metric (27mm) is a bit confusing since you won’t get a perfect circle reconstruction (are you averaging some numbers out?).
- It is generally a good idea to always acknowledge the limitations of the work in the manuscript at the end.
-
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This is a new concept for medical imaging 3D reconstruction that can be extended to other applications as well (not just ultrasound), so I think this manuscript can serve as a good reference point and inspiration to other researchers in the medical field. However the practicality of the solution needs to be addressed by the authors.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This work proposes a cost-effective and easy-to-use 3D ultrasound reconstruction solution that does not rely on any external trackers. In addition, the results demonstrate that this proposed solution achieves state-of-the-art reconstruction accuracy. Overall, this work exhibits its novelty and holds great clinical value.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This cost-effective design (the coupling pad) and the performance of 3D US reconstruction demonstrate great practical value. Compared to current solutions (sensor-based, and sensorless), this solution holds greater potential for future clinical applications.
The paper is well-written and organized, effectively delivering its content to readers.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Identifying the intersection points of Z-bar (N-shape) lines in ultrasound images is critical for ensuring the robustness and accuracy of 3D reconstruction. In particular, when increasing the US imaging depth, the intersection points (in the near field of US beam) may be difficult to localize accurately. The authors should provide further demonstration or discussion on this aspect.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
None
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The evaluation metrics (FDR, ADR, MD, SD and HD) of pose accuracy need to be defined in the paper.
Please clarify the equation (2), why does the first term need to divide by NL?
For presenting number results, typically, two decimal places are sufficient for the measurement in mm. Table 2 used three different ways (1, 2 and 3 decimals) to show the results, which is not scientific. And Figure 4 (C) has this problem as well. For the 9-line phantom, the authors need to provide a detailed description of the layout of these lines. From Figure 3, it seems that most lines are in parallel. If so, the elevational direction may not be sensitive to 3D reconstruction, potentially introducing bias. Therefore, the authors should explain how they addressed this issue to ensure an unbiased evaluation of the reconstruction accuracy.
For the last paragraph of Section “Experiments and Results”, the authors better put it to section Discussion.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed 3D ultrasound holds great clinical value, which is promising to bring to the clinical.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
N/A
Meta-Review
Meta-review not available, early accepted paper.