Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Accurate identification of arteries and veins in ultrasound images is crucial for vascular examinations and interventions in robotics-assisted surgeries. However, current methods for ultrasound vessel segmentation face challenges in distinguishing between arteries and veins due to their morphological similarities. To address this challenge, this study introduces a novel force sensing guided segmentation approach to enhance artery-vein segmentation accuracy by leveraging their distinct deformability. Our proposed method utilizes force magnitude to identify key frames with the most significant vascular deformation in a sequence of ultrasound images. These key frames are then integrated with the current frame through attention mechanisms, with weights assigned in accordance with force magnitude. Our proposed force sensing guided framework can be seamlessly integrated into various segmentation networks and achieves significant performance improvements in multiple U-shaped networks such as U-Net, Swin-unet and Transunet. Furthermore, we contribute the first multimodal ultrasound artery-vein segmentation dataset, Mus-V, which encompasses both force and image data simultaneously. The dataset comprises 3114 ultrasound images of carotid and femoral vessels extracted from 105 videos, with corresponding force data recorded by the force sensor mounted on the US probe. The code and dataset can be available at https://www.kaggle.com/datasets/among22/multimodal-ultrasound-vascular-segmentation

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3161_paper.pdf

SharedIt Link: https://rdcu.be/dY6jG

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72083-3_61

Supplementary Material: N/A

Link to the Code Repository

https://github.com/evelynskip/artery-vein-segmentation

Link to the Dataset(s)

https://www.kaggle.com/datasets/among22/multimodal-ultrasound-vascular-segmentation

BibTex

@InProceedings{Gen_Force_MICCAI2024,
        author = { Geng, Yimeng and Meng, Gaofeng and Chen, Mingcong and Cao, Guanglin and Zhao, Mingyang and Zhao, Jianbo and Liu, Hongbin},
        title = { { Force Sensing Guided Artery-Vein Segmentation via Sequential Ultrasound Images } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15004},
        month = {October},
        page = {656 -- 666}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper proposes to use force sensing guided segmentation to segment veins and arteries in ultrasound frames. The proposed approach uses a force sensor to measure the pressure induced by the ultrasound probe, automatically identifies key frames with significant vascular deformation and fuses the information coming from different frames using an attention mechanism.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Clincally, vessel compressibility is used distinguish between veins and arteries. Therefore using it in an automated system is a good idea.
    • The authors promise to publish the data used for the paper
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • The proposed idea is not novel. Existing work uses compressibility even without the need for a force sensor. For example see “AutoDVT: Joint Real-Time Classification for Vein Compressibility Analysis in Deep Vein Thrombosis Ultrasound Diagnostics” (Tanno et al, MICCAI 2018) and the other papers listed here https://thinksono.com/research-2/
    • Using multiple frames with different forces should make it easier to distinguish between veins and arteries. I would expect the experiments to reflect this and show that the models with force sensing are better at distinguishing the two classes and not at segmenting in general. Looking at Figure 3 I don’t see the “FG” models being better at distinguishing the two classes.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    The idea behind this paper is valid but it should be better validated and the relevant literature cited

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Reject — should be rejected, independent of rebuttal (2)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    While this idea is clinically sound, its not novel and not properly validated in the experiments. Therefore I don’t think this paper would bring significant value to the community

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Reject — should be rejected, independent of rebuttal (2)

  • [Post rebuttal] Please justify your decision

    While the authors have clarified the distinction between their work and prior efforts, the novelty of the proposed approach is still not entirely convincing. The authors claim the introduction of force data as a key contribution. However, the practical usefulness of this addition remains questionable. The increased accuracy must be weighed against the added complexity and cost of incorporating a force sensor. The results show more improvements coming from using larger and more powerful architectures rather than from the inclusion of force sensing. The comparison is not entirely fair, as only UNet was evaluated without force information across multiple frames. For a fair assessment, all models, especially the best performing Transunet, should be compared including multiple frames without force data. Furthermore, training all the models with the same hyperparameters and number of epochs is not a fair comparison. The training of different models should be tuned separately. From my experience of training ultrasound segmentation models, some of the errors shown in Fig. 4 are suggesting of subpotimal training. In particular the incompleteness of the segmentation of Transunet in the first row is surprising given the uniformity of the grayscale values inside the vessel. Having two segmentation classes mixed together, i.e. the red spots within the blue segmentation in the third and fourth row, is a signal that the models have not been trained for long enough.



Review #2

  • Please describe the contribution of the paper

    A force sensing guided segmentation technique has been introduced to improve deep learning-based artery-vein segmentation accuracy by leveraging their distinct deformability. Force magnitude has been utilized to identify frames with dominant vascular deformation in an ultrasound image sequence.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The proposed idea of force sensing guided segmentation is novel
    2. Well-written Introduction section
    3. Ablation study has been performed
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. Data acquisition details have not been provided
    2. Minimal performance boost compared to the complicacy added by the proposed approach
    3. No information on parameter optimization criteria
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    1. Abstract: Please report some quantitative results in the abstract to make it a proper digest of your findings.
    2. Abstract: Please state the anticipated clinical impact of your contribution.
    3. Introduction: The authors have claimed a publicly available ultrasound artery-vein segmentation dataset as one of the contributions. However, the dataset has not been published yet. I think at least a small part of the dataset should be published as a demo before making this claim. The complete dataset can be published after the acceptance of the manuscript.
    4. Section 2: This section looks shallow. Healthy volunteers/patients? How many: What are their demographic/age details? What were your recruitment criteria?
    5. Section 2: Please list the ultrasound imaging parameters.
    6. Along the same line of the proposed idea, do you think vascular elastograms in conjunction with ultrasound images can improve artery-vein segmentation?
    7. Section 4.1: How did you optimize different training parameters?
    8. Table 1: Do you think the improvement is minimal compared to the computational load added by the proposed technique?
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Reject — should be rejected, independent of rebuttal (2)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The major factor that led me to my decision is the minimal performance improvement achieved by the proposed technique. However, this work has good potential to be published in other venues once a substantial performance improvement is ensured through technical amendments, such as optimizing the training parameters or training on a larger dataset.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Accept — should be accepted, independent of rebuttal (5)

  • [Post rebuttal] Please justify your decision

    The authors have clarified my performance-boost-related concern. Therefore, I would vote for the acceptance.



Review #3

  • Please describe the contribution of the paper

    The authors propose a method for artery-vein segmentation in 2D ultrasound images. The method exploits the fact that veins show much stronger deformations upon applied forces than arteries. The main contribution is to combine an ultrasound probe with a force-measuring sensor so that deformations in the vessels can be visualized and related to the force, to develop a novel attention module using key frames for DL that can be used in conjunction with different U-Net architectures for artery-vein segmentation, and to provide a multimodal ultrasound/force dataset.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Novel idea of combining ultrasound probe with force sensor
    • Integration of key frames as a novel attention module into U-Net schemes
    • Thorough quantitative and qualitative experimental analysis and comparison with different U-Nets
    • Very promising segmentation results and artery-vein separation
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • Key frame selection seems too simple for longer video sequences
    • State-of-the-art nnU-Net could be used instead of standard U-Net in experimental comparison
    • The abstracts states “achieves significant performance improvements” but p-values are not provided
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    No

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Overall, the paper is well written, the application and contribution are clearly described, the proposed method is sound, and the experimental results and comparison are convincing. I have some comments:

    The key frame selection for K_min and K_max considers the full sequence and simply chooses the frames with global minimal and maximal force. However, for a longer sequence where the ultrasound probe might (slightly) move or tilt over time, the chosen key frames might not be optimal for all frames. Why not choosing K_min and K_max in some temporal vicinity of the current frame instead of considering the full sequence?

    I recommend to use the state-of-the-art nnU-Net instead of a standard U-Net in the experimental comparison if possible since it reduces issues with hyper parameter fine tuning and usually achieves better results.

    The abstracts states “achieves significant performance improvements” but p-values are not provided. I suggest to perform an adequate statistical test (e.g., paired t-test) to prove significance. Alternatively, the formulation “notable performance enhancements” of the conclusion could be used in the abstract as well.

    I suggest to partly reformulate Sect. 3.3 and explain all used symbols and better relate it to Fig. 2. Some examples:

    • 1x1 and 3x3 convolutions are not indicated in the Figure
    • “key feature D_V” and “value feature D_K” seems to be mixed
    • C_K and C_V are not introduced
    • “E_K and E_V are transposed”, but in the figure only E_K is transposed

    Minor comments:

    • Fig. 1: Caption states that “Twelve points are selected” but only 11 are shown.
    • Last paragraph of Sect. 2: Data is split in training and validation. But I assume there is also test data that was used for the reported quantitative results. Please comment on this
    • FLOPs are listed in table 1 but not discussed
    • “dice coefficient decreases by 0.039” should read “… 0.0039”
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Well written paper with appropriate SOTA review, sufficient novelty and technical level of contribution, qualitative and quantitative experimental comparison with SOTA approaches, promising qualitative segmentation results, and novel multimodal dataset that will be made publicly available.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Accept — should be accepted, independent of rebuttal (5)

  • [Post rebuttal] Please justify your decision

    Except for my concern on a statistical measure on the significance of the results, the authors addressed my concerns. Thus, I still argue for accepting the paper.



Author Feedback

We sincerely thank the reviewers for their insightful comments and valuable suggestions. Q1: Novelty and comparison with existing work (R1) R: Thank you for the comment and bringing the reference for our attention. The term “not novel” is inaccurate. Though AutoDVT (Tanno et al, MICCAI 2018) uses compressibility to diagnose DVT, it is not a segmentation but a classification task. We are still the first work using vessel compressibility to segment ultrasound images. The ThinkSono App claims to be able to segment groin vessels, but the method and performance remain unclear. Meanwhile, we are the first to propose the idea of utilizing force data to assist in segmentation, which introduces a new problem setting with benchmark dataset. This is also a key contribution of the article. Our proposed force guided approach can more effectively capture and utilize compressibility. Especially when force changes irregularly, directly selecting multiple frames will result in higher computational costs. Q2: Ability of artery-vein differentiation (R1) R: Thanks for the comment. In the fourth row of the quantitative results (Fig.3), both Swin-UNet and Transunet misclassify part of a vein as an artery, while force guided networks does not encounter this error. In the third row, UNet also incorrectly identifies part of veins, which is mitigated by FG-UNet. We will conduct further experiments to quantify the models’ ability to distinguish between arteries and veins. It is notable that the proposed approach aims more than artery-vein differentiation. Precise vein segmentation is equally crucial. Comprehensive experiments on three baselines, along with visual results, confirm the effectiveness of the force guided approach. Q3: Performance boost compared to added complicacy (R3) R: Thanks for the comment. The main contribution of this article lies not in performance boost, but in introducing the problem setting of utilizing force data to assist in segmenting ultrasound videos. We put forward a force-guided segmentation approach along with corresponding dataset, for further exploration by other researchers. This task can provide inspiration to the field of medical segmentation. Additionally, half of the self-constructed ultrasound dataset consists of carotid vessels which are extremely difficult to identify, making the task highly challenging. Even current SOTA methods perform poorly on this dataset, yet we are still able to achieve some performance improvements. Further tuning and modifications (such as applying nnUNet, suggested by R5) may lead to better performance. Moreover, although our method increases the complicacy during training, we can achieve nearly the same inference time as baseline models during testing by computing key frame embeddings only once per video. This feature will be available upon code open-sourcing. Q4: Data acquisition and training details (R3) R: Thank you for pointing that out. Due to length limitations, these contents are omitted in the article. All of the mentioned details will be further illustrated when the dataset and code are released. Q5: Key frame selection (R5) R: Thanks for the insightful comment. During data collection, we consider one palpation (one compression and one dilation) as a video clip. Given the short length of ultrasound videos, we employ this key frame selection method. For longer videos, approaches like selecting within a certain time window can be utilized without altering the overall structure of the network. Q6: Possible usage of nnUNet (R5) R: Thank you for this valuable suggestion. In this article, we valid the force-guided approach can improve the performance of multiple U-shaped networks. We also notice that nnUNet may achieve better performance due to its self-adapting ability and will do more experiments in the future. Q7: Enhancement of the writing (R3&R5) R: Thanks. We will carefully revise typos, notations and add quantitative results to the abstract to enhance readability.



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    N/A

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    N/A



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    N/A

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    N/A



back to top