List of Papers Browse by Subject Areas Author List
Abstract
Reconstructing neurons from large-scale optical microscope images is a challenging task due to the complexity of neuronal structures and extremely weak signals in certain regions. Traditional segmentation models, built on vanilla convolutions and voxel-wise losses, struggle to model long-range relationships in sparse volumetric data. As a result, weak signals in the feature space get mixed with noise, leading to interruptions in segmentation and premature termination in neuron tracing results. To address this issue, we propose NeuroLink to add continuity constraints to the network and implicitly model neuronal morphology by utilizing multi-task learning methods. Specifically, we introduce the Dynamic Snake Convolution to extract more effective features for the sparse tubular structure of neurons and propose a easily implementable morphology-based loss function to penalize discontinuous predictions. In addition, we guide the network to leverage the morphological information of the neuron for predicting direction and distance transformation maps of neurons. Our method achieved higher recall and precision on the low-contrast Zebrafish dataset and the publicly available BigNeuron dataset. Our code is available at https://github.com/Qingjia0226/NeuroLink.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0811_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: N/A
Link to the Code Repository
https://github.com/Qingjia0226/NeuroLink
Link to the Dataset(s)
https://github.com/BigNeuron/Data/releases
BibTex
@InProceedings{Yan_NeuroLink_MICCAI2024,
author = { Yan, Haiyang and Zhai, Hao and Guo, Jinyue and Li, Linlin and Han, Hua},
title = { { NeuroLink: Bridging Weak Signals in Neuronal Imaging with Morphology Learning } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15008},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper proposed a new neural segmentation method named NeuroLink, which guides the network to continuously learn the morphological features of neurons, enhancing segmentation in areas with faint signals. Specifically, the method can divide into 1. introduce Dynamic Snake Convolution (DSC); 2. propose a Weighted Local Connectivity (WLC) loss; 3. design a multi-task learning approach.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- I think the Weighted Local Connectivity (WLC) loss in this paper to be quite appealing and effective.
- The author’s illustrations are easy to understand.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The primary weaknesses are presented here, with detailed comments to follow in section 10.
- The experiments are insufficient and fail to adequately demonstrate the effectiveness of each proposed method.
- No repeated or cross-validation experiments were conducted, which diminishes the reliability of the experimental results.
- The presentation of the results is inadequate.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
The authors have not explicitly stated in the paper that their code will be made public. Additionally, they utilized two datasets, one of which is publicly available, while the other, the Zebrafish Dataset, seems to be private and includes original images, segmentation labels, probability maps, and distance transformation maps. There is also no clear indication from the authors regarding the release of this data. Therefore, I have concerns about the reproducibility of this study.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Level 1 (Major Comments):
- (Method) The authors claim to have introduced a novel data augmentation method that simulates weak signals caused by uneven fluorescence signal distribution by eroding local regions of the original image in Section 2.1. However, I have the following concerns: Firstly, if the input images already contain weak signals, is there still a benefit in simulating them? Secondly, if erosion affects areas that already have weak signals, wouldn’t this risk losing vital weak vessel signals, leading to miss segmentation these vessels? Additionally, I could not find experimental results validating the efficacy of this proposed data augmentation method.
- (Method) No ablation study was conducted to ascertain the effectiveness of each task within the multitask module.
- (Method) I find the results for +mT+WLC and NeuroLink (ours) in Table 1 to be very close. The lack of repeated experiments has led me to question the necessity of the DSC module. Furthermore, the authors have not presented any mean variance from multiple experiments or conducted cross-validation.
- (Experiments) What criteria were used to select 28 and another 75 images for experimentation from the gold166 dataset? The justification provided for not using the entire dataset, due to significant differences within it, is somewhat unconvincing. Isn’t it better to test the method’s robustness against varying images? Additionally, How to decide that the data divided into 61 images for training, 13 for validation, and 29 for testing, and what was the total dataset size?
- (Experimental Results) Why weren’t SD, SSD, and MES measured across all categories in Table 2? In addtion, since the design of the authors’ method could potentially cause the false positive problem, as indicated by a lower RRE compared to the best-performing method, despite having the highest REC in Table 2.
- (Experimental Results) Why are there no results for SGSNet in Table 3? There are only qualitative results for the SGSNet. In addtion, other deep learning methods lack qualitative analysis, which I believe could be more meaningful compared to classical methods.
- (Ablation Study) It’s unclear why the ablation study was only performed on the Zebrafish dataset, and why this dataset was not used for comparison with SOTA methods. Is there a specific reason for this?
Level 2 (Intermediate Comments):
- (Method) How many positive sample points were randomly selected for the proposed data augmentation method? How was the size of 10×20×20 determined, given the considerable variability in vessel thickness?
- (Method) In section 2.2, how was the decision made to ignore the voxels obtained from the first two operations to eliminate the influence of the neuron radius? Moreover, particularly in densely vascular areas, if vessels are connected after the first two operations, wouldn’t this lead to an incorrect calculation of weights for the subsequent loss computation as they would all be zero?
- (Related Work, Experiment) There has been no comparison with the losses mentioned in references [6] and [4]. These topology-related losses are new, introduced in 2021 and 2023, and since the main contribution of this paper is also a loss design, I believe it would be beneficial to consider an experimental comparison.
Level 3 (Minor Comments):
- (Formula) Some symbols are not defined, such as si in formula (1), gi in formula (2), and R in formula (5).
- (Formula) The ‘N’ in the phrase “N is the total number of voxels” should be italicized to maintain consistency with the notation in the formulas.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Reject — should be rejected, independent of rebuttal (2)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The authors proposed a novel method for segmenting neurons, addressing the issue of weak signals in certain areas of microscopic images that lead to poor segmentation results. However, the experimental details and presentation of results in the paper are somewhat insufficient. In addition, I have concerns about the reproducibility of this study.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
The author has addressed some of my concerns. Although I still have reservations about the effectiveness of the integration of multi-task modules, the presentation of qualitative experimental results, and the lack of comparison with SOTA methods on zebrafish data, considering the constraints of the manuscript’s length and the author’s clarification and commitment to the method’s effectiveness, I am inclined to change my decision to weak accept (4).
Review #2
- Please describe the contribution of the paper
The manuscript reports a network-based segmentation of neurons in 3D fluorescence microscopy data. The authors incorporate data augmentation, and assumptions on connectivity, shape and directionality of neurons. The model is then tested on two different publicly available datasets.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The manuscript is well-structured and concise. Clear neuron-morphology-oriented frameworks, such as mimicking “broken” neuron lings by erosion of training datasets, incorporation of connectivity by “directed” dilation of predicted neurons and taking account of neuron shape and direction, by using distance maps and others are thoughtful and well-chosen biases for neuron morphology. The strongest feature of this manuscript is the use of diverse and publicly available data sets, including the use of biologically relevant metrics to assess model accuracy.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Unfortunately, the manuscript lacks discussions on when the model fails or segment neurons not accurately. This type of information would help to identify potential improvements for the pipeline. Additionally, no details on how the model could be used or implemented by other groups are provided.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
I found the work extremely interesting and of great interest to the biomedical community. To fully profit from it, however, access to the code or ideally its easy-to-plug-and-play implementation would be critical.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The work presented in this manuscript is of great interest to the biomedical community, however, it lacks a critical assessment and does not enable further use by the potential user community.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This work proposes NeuroLink to reconstruct neurons from large-scale optical microscope images, which add continuity constraints to the network and implicitly model neuronal morphology by utilizing multi-task learning methods. The new method achieved higher recall and precision on the low-contrast Zebraffsh dataset and the publicly available BigNeuron dataset.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This work introduces a Dynamic Snake Convolution structure to dynamically adjust the convolutional receptive field to accommodate slender structures, and designs a Weighted Local Connectivity loss to reduce long-range false negative segmentation. A multi-task learning approach is further introduced to guide the network to learn relevant representations of morphology by predicting the local orientation of neurons and the distance to the central line. The new network achieves a state-of-the-art performance.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Efficiency is also an important dimension to evaluate a model, this work lacks such consideration.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
It is better to add a test to test the computational cost of this model comparing to previous models.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Strong Accept — must be accepted due to excellence (6)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The authors design new methods for neuron segmentation and reconstruction by considering different properties of neurons, the results demonstrate it significantly outperforms the existing methods.The paper is well-organized.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
To all We are delighted that all three reviewers have recognized the innovation of our work as it mitigates the issue of segmentation discontinuity. Before addressing specific questions, we would like to clarify the following points: 1) For ease of algorithm replication, the manuscript already includes a detailed description. Upon acceptance of the paper, we promise to make the model publicly available. The Zebrafish dataset comes from another concurrent work under review. We will release it following the publication of that work. 2) Our experiment and result evaluations adhere to the standards set by SOTA neuron segmentation works [27,22,23] on IEEE TMI. 3) We have conducted experiments and comparisons on two datasets for the three innovative aspects of our paper. Due to space constraints and the lack of open-sourced comparative work, we did not conduct comparisons with some less important methods as pointed out by R3.
To R1 We are grateful for your recommendation on our work and the insightful comments ‘clear neuron-morphology-oriented framework’ and ‘thoughtful and well-chosen biases for neuron morphology’. We also recognize the importance of further discussion and the quest for improvement methods. Here, we discuss two scenarios observed in our experiments where accuracy was lower and the potential solutions:
- Gold166 is partially labeled, focusing on the neurons of interest. The lower precision may be due to the segmentation of irrelevant neurons in the image. The actual PRE would be higher. It can be effectively filtered out through post-processing.
- Our model emphasizes recall and segments some non-neuronal tubular structures. This can be mitigated by incorporating relevant negative samples into training process. Regarding the reproducibility of our method, you could kindly refer to the “to all” part. We have employed a variety of methods to guide the network in learning neuron morphology, which has enhanced the segmentation performance for weak signals. As you have pointed out, the publication of this work and the provision of an ‘easy-to-plug-and-play implementation’ could have a significant potential impact on the biomedical community.
To R3 We would like to express our gratitude for your comments and effort dedicated to reviewing our manuscript.
Q1: Method Essential for DSC: Generating neuron tracing results with App2[24] can ignore minor segmentation disconnections. The DSC increases short-range continuity, which benefit algorithms not permitting discontinuities. It also enhances PRE, SD and SSD, indicating more precise centerline tracing. Design of WLC: Ignoring the voxels is to reduce the impact of potential inaccurate radius when generating segmentation GT from centerline annotations. If there are gaps, the dilation following the multiplication will not cause interconnection.
Q2: Data augmentation It diversifies the inadequate samples. The erosion on grayscale images does not affect weak signal. The radius of neurons is relatively uniform (e.g., Fig.3).
Q3: experiment 1.2 [22,27] have already demonstrated the effectiveness of each similar task separately. 1.4 The images are same as [22] for comparison. The strategy you referenced can result in overfitting for the scarcity of images within each subset. The division was completely random. 1.5 Table 2 presents comparison of neuron segmentation. But SD, SSD and MES are based on tracing outcomes (skeleton points). 1.6 Table 3 conducts ablation study. Quantitative comparison has already been presented in Table 2. Other methods struggle to trace weak signal areas as SGSNet. 1.7,2.3 Table 1 shows ablation studies on Gold166. Zebrafish dataset containing more weak signals, better demonstrates the effectiveness of our design. Most of these methods are not open-sourced, and only results on Gold166 have been reported.
To R4 Thank you very much for your recommendation. The efficiency concerns you have raised will be beneficial in improving the final version.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The reviewers found the rebuttal to be sufficient to address their concerns. Therefore, the decision is to accept the paper.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The reviewers found the rebuttal to be sufficient to address their concerns. Therefore, the decision is to accept the paper.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The proposed method, NeuroLink, introduces a significant advancements in the field of neuronal imaging and segmentation. The integration of Dynamic Snake Convolution (DSC) and the Weighted Local Connectivity (WLC) loss function represents a novel approach to addressing the challenges of weak signal regions in region segmentation. These innovations contribute to the enhancement of both recall and precision, particularly in low-contrast images, which is a notable achievement in this domain.
The manuscript provides a detailed and well-structured methodological framework. The use of multi-task learning to predict local orientation and distance transformation maps is an effective strategy to capture the morphological features of neurons. The comprehensive description of the DSC module and WLC loss function, along with their implementation in a classic encoder-decoder architecture, demonstrates the robustness of the proposed approach.
The authors conducted extensive experiments on two datasets, Gold166 and Zebrafish, to validate the effectiveness of their method. The results show significant improvements in standard evaluation metrics such as Precision, Recall, F1 score, Structural Distance (SD), and Substantial Spatial Distance (SSD). The ablation studies further reinforce the contribution of each component of the proposed method, providing a clear justification for its design choices.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The proposed method, NeuroLink, introduces a significant advancements in the field of neuronal imaging and segmentation. The integration of Dynamic Snake Convolution (DSC) and the Weighted Local Connectivity (WLC) loss function represents a novel approach to addressing the challenges of weak signal regions in region segmentation. These innovations contribute to the enhancement of both recall and precision, particularly in low-contrast images, which is a notable achievement in this domain.
The manuscript provides a detailed and well-structured methodological framework. The use of multi-task learning to predict local orientation and distance transformation maps is an effective strategy to capture the morphological features of neurons. The comprehensive description of the DSC module and WLC loss function, along with their implementation in a classic encoder-decoder architecture, demonstrates the robustness of the proposed approach.
The authors conducted extensive experiments on two datasets, Gold166 and Zebrafish, to validate the effectiveness of their method. The results show significant improvements in standard evaluation metrics such as Precision, Recall, F1 score, Structural Distance (SD), and Substantial Spatial Distance (SSD). The ablation studies further reinforce the contribution of each component of the proposed method, providing a clear justification for its design choices.