Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Needle guidance in thermal ablation procedures is challenging due to the absence of a free line-of-sight. To date, the needle trajectory is manually planned on a pre-operative CT slice, and then the entry point is transferred with a ruler on patient and needle. Usually, the needle is inserted in multiple strokes with interleaved control CTs, increasing the number of exchanges between OR and control room and exposure of the patient to radiation. This procedure is not only tedious, but also introduces a navigation error of several centimeters if the entry point was not chosen precisely. In this paper, we present Cryotrack, a pre- and intra-operative planning assistant for needle guidance in cryoablation. Cryotrack computes possible insertion areas under the use of a pre-operative CT and its segmentation, considering obstacles (bones) and risk structures. During the intervention, cryotrack supports the clinician by supplying intraoperative guidance with a user-friendly 3D interface. Our system is evaluated in a phantom study with an experienced surgeon and two novice operators, showing that Cryotrack reduces the overall time of the intervention to a fourth while being on par with the traditional planning in terms of safety and accuracy, and being usable by novices.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1240_paper.pdf

SharedIt Link: pending

SpringerLink (DOI): pending

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/1240_supp.zip

Link to the Code Repository

https://github.com/Cryotrack

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Kru_Cryotrack_MICCAI2024,
        author = { Krumb, Henry J. and Mehtali, Jonas and Verde, Juan and Mukhopadhyay, Anirban and Essert, Caroline},
        title = { { Cryotrack: Planning and Navigation for Computer Assisted Cryoablation } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15006},
        month = {October},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    Trajectory planning is one of the important topics for thermal ablation. This paper presents Cryotrack, a pre- and intra- operative planning assistant for needle guidance in cryoablation. The main contribution is that this tool is not only for planning but olso for navigation. The authors claim that this tool will be available in 3D slicer once the paper is accepted.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The paper was well structured and easy to understand.
    2. The authors developed a computer aided tool, which could be used for both preoperative planning and intraoperative needle guidance in cryoablation.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1. the main contribution of this work is developing a computer aided tool, but there is no novelty in methodology. All modules are based on existing algorithm.

    2. The evaluation seems weak. a. The authors compare the results of needle insertion by a surgeon and novices with cryotrack. It is expected the tool would help the novice, but the results still show a disparity between novice and skilled surgeon.

    b. The authors may use the wrong reference to analyze and explain their results. “In most percutaneous procedures, a targeting accuracy of less than 5 mm is recommended [2]”. No description about “targeting accuracy of less than 5 mm” could be found in ref [2].

    c. No results were given for needle insertion by novice without cryotrack, which is helpful to show the advantages of the proposed tool.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    The main strength of this paper is that the authors focused on the clinical application and developed a computer aided tool for pre operative planning and intra operative guidance. Moreover, this tool is a 3D slicer extension, which would be easy to access for the community. However, there is no novelty in methodology and the evaluation is relatively weak.

    1. The significance of this work lies in its application in clinics. Therefore, a more realistic evaluation is expected, such as animal or cadaver experiments, or at least the use of a phantom with risk structures that better simulate real conditions. The authors utilized phantoms and virtual scenes of risk structures, which differ significantly from real-world applications.

    2. “In most percutaneous procedures, a targeting accuracy of less than 5 mm is recommended [2]”. The authors may use the wrong reference, since no description about “targeting accuracy of less than 5 mm” could be found in ref [2]. Please find more solid evidence.

    3. No results were given for needle insertion by novice without cryotrack, which is helpful to show the advantages of the proposed tool.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Reject — should be rejected, independent of rebuttal (2)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    There is no novelty in the methodology and the evaluation is relative weak.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper introduces cryotrack, a planning and navigation pipeline for computer assisted cryoablation. The system finds shortest unobstructed insertion paths and helps the user guide the EM tracked needle to the target. The system is evaluated in phantom studies and compared against the standard of care of using repeat CT imaging. This is the first such system for Cryoablation and will be made public as a Slicer extension.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper is overall easy to read. The evaluation with expert clinician and novices on phantoms is designed well. The proposed method seems sound.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    ○ The paper derives novelty by being the first cryo guidance tool. However, there is nothing cryo specific in this study and there exist many different EM needle guidance applications. It would be better to compare the method to existing needle guidance approaches. The paper should answer the question if it is still novel for general percutaneous needle guidance or not? ○ It seems like that a core piece that makes this work interesting is an automatic segmentation of critical structures. But it is unclear from the paper how this was achieved and if this is novel. ○ The final accuracy numbers are not improved compared to the standard of care.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    • Novelty (not just for cryo) needs to be better justified.
    • Automatic flagging of risk structures needs to be described.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Novelty of this work is questionable and methods lack critical description.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • [Post rebuttal] Please justify your decision

    Critiques not addressed in rebuttal. Novelty over other needle guidance methods not described and segmentation details missing.



Review #3

  • Please describe the contribution of the paper

    The paper presents a pre- and intra-operative 3D-Slicer-base planning tool that can be implemented for cryoablation procedures to help clinicians select feasible entry points and paths that avoid risk structures and obstacles. The system is evaluated in a phantom study including novice and experienced users with the experienced user trials compared to conventional techniques. The system allowed novice users to produce near-acceptable plans and helped experienced users with risk structure avoidance with much lower time requirements, but did not appear to improve targeting accuracy.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1) Clarity: This paper was very well-written with a high level of clarity, good flow, and attractive figures, making it very easy for readers to follow and understand. 2) Detailed descriptions: This paper included detailed descriptions of all the steps, protocols, and processes, including the inclusion of acceptable accuracies, demonstrating a good commitment to reproducibility. 3) Experiments: This study included a good number of insertions for a proof-of-concept study including multiple targets and insertions at varying skill levels.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1) Representation of the results: The targeting accuracy did not appear to improve with the use of Cryotrack with the experienced users, and in one scenario had average errors above the acceptable threshold while the baseline technique was well within this threshold. While offering other advantages, this was deficiency in the proposed system was not explicitly acknowledged and identified as an area of improvement. 2) Limitations: No discussion of the limitations of the study was included (ex. targeting accuracy, single needle type, small number of experienced users, etc.).

  • Please rate the clarity and organization of this paper

    Excellent

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Do you have any additional comments regarding the paper’s reproducibility?

    The paper demonstrated a good commitment to reproducibility, including many experimental details, an anonymized link to the source code, and a claim to release 3D Slicer extension upon acceptance.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Major:

    1. Results/Table 1: It would be good for a more thorough description of the results, in particular, the Surgeon appeared to perform better with the Baseline techniques rather than Cryotrack in terms of tumour distance, especially in the single stroke OOP cases (2.38 +/- 2.74 vs. 7.14 +/- 5.67 with Cryotrack). This is particularly of note because the 7 mm average error Cryotrack is above the acceptability threshold of 5 mm. This seems to be glossed over in the first paragraph of the section.
    2. Results/Table 1: While it is reflected in Figures 4 and 5, an indicator of significantly better performance between the Surgeon cases should be included (ex. bolding or asterisk) in the table to allow for easy comparison by the reader.
    3. Results: No discussion of limitations of the system has been included and is necessary for an objective assessment of the proposed system.

    Minor:

    1. Page 3, Methods, Paragraph 2: “the obstacles (such as bones)” is repetitive of the phrase just prior and should be removed
    2. Figure 3: It would be helpful to provide a heatmap scale or a description of the heat map in the caption (i.e. identifying that the blue region represents the most feasible entry area). This is in the text but a quick mention in the figure caption would improve clarity (as the figure is referenced prior to the explanation in the text)
    3. Page 5, Experimental Setup, Paragraph 1: The sentence beginning with “A pre-operative CT…” appears to be missing a word after “allowed”. Please correct.
    4. Page 6, Experimental Setup: It would be useful to include the CT scanning parameters.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall, I found this paper very interesting and well-done and believe it would be of interest to other members of the MICCAI community, particularly in the CAI field.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

We thank the reviewers for appreciating the structure and clarity of the paper, and for their constructive feedback. The reviewers identified two common concerns, which are a lack of algorithmic / method novelty (R3,4) and a lack of accuracy improvement (R1,3,4). Before detailing our view on the particular concerns, we must highlight that the major novelty of our CAI work is the conception, implementation and validation of a full pipeline encompassing planning and navigation, to improve the workflow of a cumbersome and complex procedure in terms of safety and accessibility. This is a notable difference to typical deep learning MICCAI works, where accuracy and algorithmic novelty are considered to be the most important values. In CAI, safety and accessibility are as important, and they go hand in hand: A simpler procedure enabled by patient-side planning and decision-making improves safety by reducing overall OR/anesthesia time and exchanges between OR and control room. Typically, surgeons would scrub many times per procedure for a single insertion relying on CT-guidance. Depending on the setting, the operators might not even be able to leave the OR, requiring the control CT to be taken with them inside the room, despite protection, exposing themselves to radiation. Cryotrack enables safe, automated patient-side planning that makes intermediate control CTs unnecessary or optional, lifting a barrier to safe and successful treatment. Hence, the emphasis of our work is not on improving accuracy only, but on improving safety and accessibility. Though most parts of the proposed pipeline build upon existing methods and the algorithmic novelty is limited, Cryotrack innovates on automatic intra-operative risk avoidance with visual feedback, together with our reproducible clinical validation in phantoms. We are not aware of any other work that achieves this full integration.

Experimental Protocol (R3): R3 points out that the experimental setup is incomplete, as Cryotrack is merely validated on a phantom with virtual risks, rather than animals or complex phantoms, and there is no direct comparison of novices planning under CT. In CAI, a phantom experiment is usually the first step before attempting studies with animal or human subjects, validating the systems in animals without formal pre-validation would violate the 3R principles of animal experimentation, which is unethical at this state. Adding larger physical risk structures to the phantom would be interesting, but the expected benefit for planning is limited. This would have no impact on the planning and would only increase the complexity of our setup, limiting the reproducibility of our validation. Novice experiments are not done under CT as novices lack the proper medical training to interpret CT images for pre-op planning, and translate the plan to the phantom. Our intention behind the novice experiment is to show how our setup increases accessibility and ease of use, while staying at a safe distance to risk structures.

5 mm Accuracy Margin (R3): R3 noted that [2] does not explicitly state a 5 mm accuracy margin for cryoablation. We derived the 5 mm margin from [2] which recommends to ablate the tumor plus at least 5 mm around the tumor, i.e. within this margin, the ablation is successful. This is a simplification, though it is nearly impossible to find a well-founded upper bound for positional error in the literature.

Baseline Comparison (R4): It was noted that a comparison to other EM guidance systems is not given. As we need to plan with long booking times for the OR, we need to carefully gauge the relevance of evaluations, and a comparison to a system without intra-op planning would not add much value to our evaluation. Further, EM-guidance tools on the market are scarce, and research products are typically closed-source. We are aware of only one commercial solution (IMACTIS) which provides EM+CT guidance, but no intra-op planning.



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    The paper presents an application of a system for cryoablation, providing a pre- and intra-operative guidance tool. Despite some of the reviewers concern about the evaluation, it did involve both novice and experienced users, and their preliminary results showed that their system can significantly reduces planning time and assists novice users in producing near-acceptable plans. The authors also show a strong commitment to open science by providing an anonymized link to the source code and dataset. As a CAI paper I believe that the application of the system and the initial testing of the prototype would be of interest to the community and would lean towards acceptance.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    The paper presents an application of a system for cryoablation, providing a pre- and intra-operative guidance tool. Despite some of the reviewers concern about the evaluation, it did involve both novice and experienced users, and their preliminary results showed that their system can significantly reduces planning time and assists novice users in producing near-acceptable plans. The authors also show a strong commitment to open science by providing an anonymized link to the source code and dataset. As a CAI paper I believe that the application of the system and the initial testing of the prototype would be of interest to the community and would lean towards acceptance.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Reject

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    The primary concerns raised during initial review are limited novelty in the components and shortcomings in the evaluation (a limitation that even the most positive reviewer mentioned). Overall, the rebuttal highlights the challenges of CAI work compared to MIC efforts, which is a known issue and one that I am generally sympathetic to, however, the reviewer verdict is a) still for rejection with scores low for this ACs batch, and 2) the rebuttal emphasizes explanations over actions that would result in a stronger manuscript, and thus, does not sway the reviewers. For me similarly, the work is an integration work that derives innovation at the systems level, but then has to demonstrate convincing validation which is something that is missing. Thus, my recommendation at this time if for rejection.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    The primary concerns raised during initial review are limited novelty in the components and shortcomings in the evaluation (a limitation that even the most positive reviewer mentioned). Overall, the rebuttal highlights the challenges of CAI work compared to MIC efforts, which is a known issue and one that I am generally sympathetic to, however, the reviewer verdict is a) still for rejection with scores low for this ACs batch, and 2) the rebuttal emphasizes explanations over actions that would result in a stronger manuscript, and thus, does not sway the reviewers. For me similarly, the work is an integration work that derives innovation at the systems level, but then has to demonstrate convincing validation which is something that is missing. Thus, my recommendation at this time if for rejection.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    This is a CAI translational paper and so it should be judged accordingly. It is indeed the case that the novelty of the individual parts for the piepeline is limited. The main achievement is the integration of all the elements that enable the intra-operative risk avoidance with visual feedback. It is standard practice for CAI papers to fist be tested with phantom experiments. The visualization and audio feedback as demonstrated in the supplied video (supplemental material) demonstrates the system. It is indeed the case that the novelty and experimental results are limited as pointed out by the reviewers. This makes it for a bordeline paper. Since it is a CAI paper and we are interested in encouraging this type of work, I recommend acceptance as a poster.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    This is a CAI translational paper and so it should be judged accordingly. It is indeed the case that the novelty of the individual parts for the piepeline is limited. The main achievement is the integration of all the elements that enable the intra-operative risk avoidance with visual feedback. It is standard practice for CAI papers to fist be tested with phantom experiments. The visualization and audio feedback as demonstrated in the supplied video (supplemental material) demonstrates the system. It is indeed the case that the novelty and experimental results are limited as pointed out by the reviewers. This makes it for a bordeline paper. Since it is a CAI paper and we are interested in encouraging this type of work, I recommend acceptance as a poster.



back to top