Abstract

Synchrotron Radiation micro-Computed Tomography (SR-microCT) is a promising imaging technique for osteocyte-lacunar bone pathophysiology study. However, acquiring them costs more than histopathology, thus requiring multi-modal approaches to enrich limited/costly data with complementary information. Nevertheless, paired modalities are rarely available in clinical settings. To overcome these problems, we present a novel histopathology-enhanced disease-aware distillation model for bone microstructure segmentation from SR-microCTs. Our method uses unpaired histopathology images to emphasize lacunae morphology during SR-microCT image training while avoiding the need for histopathologies during testing. Specifically, we leverage denoising diffusion to eliminate the noisy information within the student and distill valuable information effectively. On top of this, a feature variation distillation method pushes the student to learn intra-class semantic variations similar to the teacher, improving label co-occurrence information learning. Experimental results on clinical and public microscopy datasets demonstrate superior performance over single-, multi-modal and state-of-the-art distillation methods for image segmentation.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3328_paper.pdf

SharedIt Link: https://rdcu.be/dV51o

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72114-4_37

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/3328_supp.pdf

Link to the Code Repository

https://github.com/isabellapoles/LOTUS

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Pol_Letting_MICCAI2024,
        author = { Poles, Isabella and Santambrogio, Marco D. and D’Arnese, Eleonora},
        title = { { Letting Osteocytes Teach SR-microCT Bone Lacunae Segmentation: A Feature Variation Distillation Method via Diffusion Denoising } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15009},
        month = {October},
        page = {383 -- 393}
}

Reviews

Review #1

Please describe the contribution of the paper

A novel method for bone microstructure segmentation in SR-micro CTs based on a histopathology-enhanced disease aware model and unpaired data.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The approach does not rely on paired data, Has the potential to be used on other image types/modalities (as the authors show) - this is not common! Tackles an interesting problem of using a more available type of data with a model that has information from lesser available type of data for improved segmentation, where the data is not necessarily paired. Evaluation is thorough i.e. comparison to multiple existing methods, statistical analysis, ablation study.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Fairly solid. Please see below question regarding why one data type is considered a noisy version of the other.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

In the introduction the authors state that “histopathologies provide information on active osteocyte distribution and oblateness, which correlates with physiological processes, osteoporosis, or COVID-19, while SR-microCTs reveal early signs of the consequent bone microarchitectural damages”, and “we see the student model as a noisy version of the teacher having sub-optimal training ability to learn discriminative features;” where the student models is SR-micro CT and teacher is the histopathology. From the first statement it seems that the two modalities measure different characteristics of tissue. Can the authors explain a bit more why they consider the student model a “noisy” version of the teacher model? One would assume that a ‘noisy version’ would measure similar features or characteristics, however with some non-useful signals added to it.

“Similarly, a SRmicroCT model learns from SR-microCT images“: you mean “Similarly, a student model…”
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Accept — could be accepted, dependent on rebuttal (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Important problem with an interesting approach, strong evaluation.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

In this paper, the authors introduced a new deep learning architecture for the histopathology-enhanced bone lacunae segmentation. Overall, the architectural approaches introduced in this work are fairly interesting and sound, and the experimental results are indeed convincing.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The main strengths of this work spans across interesting architectural solutions, convincing experimental validation and comparison with the state of the art.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Although the manuscript reads well and presents valid ideas, it also suffers from several shortcomings which should be, in my opinion, thoroughly addressed before the manuscript could be considered for publication and presentation at MICCAI:
1. We are currently facing the reproducibility crisis in the machine learning field. It is especially visible and, at the same time, important in the area of medical data analysis. To address this gap, the authors should provide a link to the repository containing their implementation (as reproducing this approach would not be trivial). Also, the authors should include the scripts showing how to reproduce the numerical results reported in this work.
2. If possible, I would encourage to expand the number of visual examples hence the qualitative analysis presented in this work (I am aware of the space constraints). It would be useful to investigate the visual examples elaborated using different methods – are there any interesting insights here?
3. It might be useful to announce the structure of the manuscript in the introductory section.
4. I encourage the authors to investigate the non-functional characteristics of the investigated machine learning algorithms, with a special emphasis put on their inference time.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not provide sufficient information for reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

Please see my detailed comments in other parts of this review.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

Please see my detailed comments in other parts of this review.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Accept — could be accepted, dependent on rebuttal (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The authors presented a solid deep learning architecture, and the experiments are indeed convincing. There are, however, several shortcomings which should be thoroughly addressed before the manuscript could be considered for publication. I listed those issues in other parts of this review.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

The authors present a method for bona lacunae segmentation from SR-microCT images that incorporates knowledge from a histopathology teacher model. The proposed approach follows a teacher - student design and does not require explicitly paired multi-modal images. The method combines denoising diffusion with feature variation distillation methods. The results show that histopathology related information from the teacher model can be successfully inferred from SR-microCT data after training. The authors evaluate their approach based on a custom hand-annotated dataset as well as additional publicly available datasets. A particular positive aspect is also the 5-fold cross validation of models to obtain information about statistical significance of results in the benchmark comparison.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

A key strength of the publication is the design of the training pipeline via teacher - student that does not require matched image sections but instead learns to match segmentation and intra-class semantic knowledge via feature denoising and variation distillation, respectively. The resulting pipeline and training concept is novel and the achieved results are significant. As the authors point out, the concept can be applied to other multi-modal segmentation tasks which increases its potential impact. The authors also present extensive validation experiments based on their own dataset as well as publicly available data. The results support the claims of the authors. An ablation study is presented that supports the design choices proposed.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

I do not see many weaknesses of the proposed approach. It would be great if the authors were to release their custom dataset to the community.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

The methods are well described and the approach is evaluated also on publicly available datasets.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

As mentioned already above, I think the main concepts and claims are well described and supported by the details and data presented in the manuscript. The main paper would also benefit from having image examples that are currently only found in the appendix. A complementary release of the custom dataset would strengthen the contribution. I would like to encourage the authors to evaluate their approach on different multi-modal segmentation use cases maybe in a follow-up publication. If the concept generalizes well to other use cases, this would increase the impact and significance of the proposed work.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Accept — should be accepted, independent of rebuttal (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The authors address an important issue of multi-modal knowledge distillation that can have impact in clinical practice. The proposed approach significantly improves on the state-of-the-art. The results justify the claims of the authors.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Author Feedback

We thank AC and the Reviewers for their constructive comments. All Reviewers recognize our proposed histopathology-enhanced bone lacunae segmentation method as “novel” (R3, R6), “interesting and sound” (R4), and find “the achieved results significant” (R6), “convincing” (R4), and “thorough” (R3). Unanimously, they appreciate that “the approach does not rely on paired data, has the potential to be used on other image types/modalities” (R3) and that it “can have an impact in clinical practice” (R6). Yet, they require clarifications about (i) modalities measuring different features and noise (R3), (ii) visual insights (R4) and model generalization (R6), (iii) model nonfunctional characteristics (R4), and (iv) code-data release (R4, R6).

i) We apologize for any misunderstanding. Our aim was not to suggest complete diversity between histopathologies and SR-microCTs but to highlight their multifaceted yet interdependent functions in downstream applications exploiting osteocytes/lacunae segmentations. Discussions with biomechanical researchers and a review of related literature underscore how osteocyte variations can yield insights into bone remodeling or disease processes (Dreyer Vetter et al.). Differently, lacunae can predict healthy/diseased bone microcrack generation and progress (Buccino et al.). Besides, Carter et al. observed that osteocytes and lacunae morphology and distribution depend on each other, as osteocytes lie inside lacunae. This brings, from the functional side, dependence between their different roles, while, from the image feature side, similar semantic characteristics. Therefore, R3’s assumption is correct. The student’s role is to approximate the teacher’s prediction, although trained on similar yet harder-to-understand features (Hinton et al.). The resulting non-zero noise between their features could manifest as reduced prediction certainty of the student compared to the teacher due to low informative features or student prediction errors caused by some “non-useful signals” from the student image modality. Table I monomodal results prove this: despite the teacher and the student models being the same, the SR-microCT model performs worse than the histopathology one. ii) We agree that characterizing the predicted segmentation across methods could provide insights and enhance model interpretability. Conversely, a qualitative evaluation would be arduous due to microscale morphological differences among segmentations and lacunae/osteocyte density distribution that would require sample-level analysis and quantitative parameters (i.e., stretch, oblateness, and density) extraction to prove the qualitative findings. Besides, showcasing the effectiveness of our method across diverse image domains holds significance for broader adoption and community validation. Regrettably, time and page constraints precluded us from conducting additional experiments for this paper. Nonetheless, we are committed to comparing our predicted segmentations qualitatively and quantitatively against other approaches and testing our strategy on different domains (i.e., radiology) in follow-up publications. iii) In the revised manuscript, we will add the details on model trainable parameters (3,016,883) and average inference time for patch lacunae segmentation (0.0063±0.0198s), for SR-microCT WSI (12.27±0.02s), and DeepLIIF image patches (0.0054±0.0190s). We employed an AMD Ryzen 7 5800X @3.8 GHz with a 24 GB NVIDIA RTX A5000 GPU for development and evaluation on 512x512 size patches and 3100x3100 size WSI. iv) Recognizing the importance of reproducibility, we will add a link to the code repository in the final manuscript. While we cannot disclose our clinical bone dataset yet, we are working to facilitate its accessibility. Nevertheless, our method demonstrates versatility with the DeepLIIF dataset (Ghahreman et al.), which is publicly available. Finally, we will fix the typo (R3) and include the visual results (R6) in the final paper.

Meta-Review

Meta-review not available, early accepted paper.

back to top

Letting Osteocytes Teach SR-microCT Bone Lacunae Segmentation: A Feature Variation Distillation Method via Diffusion Denoising

Author(s):