List of Papers Browse by Subject Areas Author List
Abstract
The serial section electron microscopy reconstruction method is commonly used in large volume reconstruction of biological tissue, but the inevitable section damage brings challenges to volume reconstruction. The section damage may result in imperfect section alignment and affect the subsequent neuron segmentation and data analysis. This paper proposes an aligning and restoring method for imperfect sections, which contributes to promoting the continuity reconstruction of biological tissues. To align imperfect sections, we improve the optical flow network to address the difficulties faced by traditional optical flow networks in handling issues related to discontinuous deformations and large displacements in the alignment of imperfect sections. Based on the deformations in different regions, the Guided Position of each coordinate point on the section is estimated to generate the Guided Field of the imperfect section. This Guided field aids the optical flow network in better handling the complex deformation and large displacement associated with the damaged area during alignment. Subsequently, the damaged region is predicted and seamlessly integrated into the aligned imperfect section images, ultimately obtaining aligned damage-free section images. Experimental results demonstrate that the proposed method effectively resolves the alignment and restoration issues of imperfect sections, achieving better alignment accuracy than existing methods and significantly improving neuron segmentation accuracy. Our code is available at https://github.com/lvyanan525/Aligning-and-Restoring-Imperfect-ssEM-images.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1172_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: N/A
Link to the Code Repository
https://github.com/lvyanan525/Aligning-and-Restoring-Imperfect-ssEM-images
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Lv_Aligning_MICCAI2024,
author = { Lv, Yanan and Jia, Haoze and Chen, Xi and Yan, Haiyang and Han, Hua},
title = { { Aligning and Restoring Imperfect ssEM images for Continuity Reconstruction } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15002},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
A registration method to align for electron microscopy sections, capable of handling tissue defects such as tears and folds.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
A non-trivial method has been implemented and appears to be working.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The main problem is that the method is not described sufficiently clearly to be understood or independently implemented. The method is only described by a relatively simplified diagram in Figure 2.
The authors segment the image info fragments but it is not clear how exactly this is done. The interpolation described in eq. 1 and 2 seems inappropriate because it leads to a smooth deformation field, which is certainly not the case if there are tears or folds.
It is not clear how the Section-Prediction module works and how and by whom it was validated (Section 2.2). Does it actually invent the missing data? This does not seem to be appropriate in many applications.
In the experimental section, it is not clear how the ground truth for evaluation was obtained. Are the deformation synthetic?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
References (page 9) should in preference use reviewed sources, not arXiv. Superfluous links in [9] can be eliminated. The names in [17] appear incorrect.
How can epsilon (eq.2) be “infinitely close to 0”?
What exactly should we see in Fig.3, how is the “effectiveness” demonstrated?
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
If I understand the method properly, I think it is flawed and not appropriate for this kind of deformation. I am also not sure whether the experimental evaluation is appropriate.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The paper presents a pipeline for aligning and restoring imperfect sections in images. It comprises three main modules: section alignment, section prediction, and mosaic. The alignment module identifies deformations in different regions and applies appropriate transformations to approximate alignment. Authors address discontinuities along category borders by considering the influence of all categories on each point and implementing smooth constraints. Following alignment, the prediction module generates a restored image of the imperfect section. Finally, the mosaic module integrates the restored section onto the damaged region, completing the restoration process.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper addresses an interesting topic and thoroughly discusses the associated challenges. It offers a comprehensive overview of existing methods and presents a detailed explanation of its proposed solution. The paper provides comperhensiperformance analysis and shows superiority of proposed approach compared to SOTA and shows it enhances the accuracy of neuron tracing and segmentation.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The explanations of section prediction module lack clarity and detail. For example, it is not clear what is I3, or how the restored image is generated. Authors are advised to provide more explicit explanations and rewrite this section for improved comprehension. Also, the paper could benefit from including statistical significance in the comparison to validate the effectiveness of the proposed method.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
paper uses publcly available dataset but not mention open access to source code.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Please address weaknesses
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper introduces an interesting problem and offers a novel approach to address it. While it includes performance analysis and comparisons with SOTA, certain aspects lack clarity, and the statistical significance of the analysis needs to be incorporated for better validation.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Accept — should be accepted, independent of rebuttal (5)
- [Post rebuttal] Please justify your decision
Authors have addressed my concerns.
Review #3
- Please describe the contribution of the paper
Summary: Relevance: High. This paper addresses one of the hardest challenges in EM image restoration: Registration of EM sections that have been physically damaged or largely deformed, with folds that amount to irreversible data loss. The goal of such analysis is to preserve the information that can be salvaged, preserve an otherwise good registration of the volume, and infer the missing information for seamless downstream analysis. The authors demonstrate methodology that addresses all these aspects, and show it’s efficacy on real-world public EM datasets that suffer from these data issues.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The method is compared to the most well-known restoration methods in the literature that attempt to achieve the same restoration goal. It outperforms all of the current methods, which has been demonstrated convincingly.
- The method is demonstrated to work on a sufficiently large real-world dataset, with images that exhibit the data pathology well.
- The figures are well chosen to demonstrate all key aspects of the methodology in action.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The discussion could benefit from a demonstration of limitations of the method; for example, what is the largest crack/fold/starting-misalignment after which restoration performance decreases significantly.
- Minor: Figure 3 could be made more clear with a panel showing the regions in red and purple post-unfolding/restoration.
- Minor: EA and the proposed method seem to perform well (See Figure 3). It would be beneficial if the authors discussed briefly their thoughts on EA vs. the proposed method.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- A statement about computational cost/scalability of the method would be in order. For example: The FAFB dataset is large and contains many of the pathologies addressed by the proposed method. Maybe the authors can discuss briefly whether (and how) it would be practical to refine and restore a dataset of such scale.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Strong Accept — must be accepted due to excellence (6)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper solves one of the most challenging problems in EM restoration elegantly and transparently. A large body of already existing public data would immediately benefit from application of the method, with the immediate effect of elevating the value of many datasets.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
To all: We thank all for their valuable feedback. About open access and reproducibility: Due to anonymity requirements, we have not included source code links in our submission. The source code will be publicized and detailed algorithm descriptions will be provided if accepted.
To R1: Q1: The statistical significance A1: Thanks for your kind suggestion. We conducted our experiment on 100 image pairs and reported the average values. Box plots with p-values or confidence intervals will be incorporated later. Q2: Descriptions of section prediction module A2: Refer to Fig.2(a). I3 is the adjacent next section of the damaged section. Due to space limitations, we have not included further details. As an auxiliary part, this module follows the previous work STDIN (reference paper [12]), which is used to replace damaged regions by combining with the mosaic module and aid subsequent continuity reconstruction(Sec 3.2).
To R3: Thanks for your comments. We will public source code to provide more details. Besides, we would like to first clarify technical soundness of our method and then show the validity of experimental evaluation as follows.
About Method Q1: Eq. 1 and 2 may lead to a smooth deformation field A1: The deformation field near tears or folds is not smooth actually. Considering this, we did not set di (in Eq. 2) as a linear distance directly passing through the fold/tear, which may lead to smoothness. Instead, we set di as a path distance that bypasses tears/folds. In our method, as an extreme case, if the tear/fold runs through the entire image, the path distance between the points on either side will be infinite. Then, the probability (in Eq. 2) that a point on one side of a tear/fold belongs to the other side will be 0. This reinforces that the deformation field near tears/folds is not smooth. Q2: About epsilon (eq.2) A2: Epsilon (e.g. 1e-10) is a constant defined to prevent Eq.2 from being meaningless when di=0. Q3: About segmenting image info fragments A3: Refer to Sec 2.1. Image info fragments are categorized based on the deformations that different regions conform to. Specifically, we uniformly sample the coordinate points on the imperfect section and align them on the reference image. Points that conform to similar transformations are then grouped into the same category. Q4: About description of Sec.2.2 A4: As an auxiliary part, the section prediction module follows the previous method STDIN (reference paper [12]). We have not included more details due to space limitations but have cited the reference. STDIN [12] is designed to predict intermediate images from adjacent images and works well for EM images. We employed it to generate the missing data in damaged regions of imperfect sections by combining it with the mosaic module (Sec 2.3), which aims to help subsequent continuity reconstruction. Experiments in Sec 3.2 verify the effectiveness of adding this module.
About experiment Q5: About ground truth A5: While in the field of electron microscope image registration, the ground truth of the registration results is unobtainable. Therefore, comparing the registration result with the reference image (adjacent image) is a common practice. We measure the similarity between the registered image and the reference image using NCC and SSIM. Besides, the deformation is real because the public datasets we used contain images of real folds/tears. Q6: About Fig.3 A6: Refer to Sec 3.1. In Fig. 3, more artifacts in the visualization result and colors closer to blue in the heatmap indicate lower alignment accuracy. Our method, which shows fewer artifacts and less blue, demonstrates higher accuracy and effectiveness.
To R4: Thanks for your high praise. Your suggestions are very constructive, and we will expand and modify them based on your suggestions in the future, including the largest crack/fold/starting-misalignment that can be handled, computational cost/scalability, how it works feasible on large-scale data, and so on.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
N/A
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The paper presents a novel and effective method for addressing the significant challenge of aligning and restoring imperfect serial section electron microscopy (ssEM) images, which is crucial for accurate large-volume biological tissue reconstruction. The proposed approach integrates an improved optical flow network with guided position estimation sub-module and a section prediciton module, providing a robust solution for handling discontinuous deformations and large displacements due to section damage. The reviewers appreciated the detailed performance analysis, the superiority over state-of-the-art methods, and the comprehensive rebuttal that addressed their concerns effectively.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The paper presents a novel and effective method for addressing the significant challenge of aligning and restoring imperfect serial section electron microscopy (ssEM) images, which is crucial for accurate large-volume biological tissue reconstruction. The proposed approach integrates an improved optical flow network with guided position estimation sub-module and a section prediciton module, providing a robust solution for handling discontinuous deformations and large displacements due to section damage. The reviewers appreciated the detailed performance analysis, the superiority over state-of-the-art methods, and the comprehensive rebuttal that addressed their concerns effectively.