List of Papers Browse by Subject Areas Author List
Abstract
Hyperspectral imaging (HSI) shows great promise for surgical applications,
offering detailed insights into biological tissue differences beyond what the naked eye can perceive.
Refined labelling efforts are underway to train vision systems to distinguish large numbers of subtly varying classes.
However, commonly used learning methods for biomedical segmentation tasks penalise all errors equivalently and thus fail to exploit any inter-class semantics in the label space. In this work, we introduce two tree-based semantic loss functions which take advantage of a hierarchical organisation of the labels.
We further incorporate our losses in a recently proposed approach for training with sparse, background-free annotations.
Extensive experiments demonstrate that our proposed method reaches state-of-the-art performance on a sparsely annotated HSI dataset comprising 107 classes organised in a clinically-defined semantic tree structure.
Furthermore, our method enables effective detection of out-of-distribution (OOD) pixels without compromising segmentation performance on in-distribution (ID) pixels.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2025/paper/0788_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{WanJun_Treebased_MICCAI2025,
author = { Wang, Junwen AND Maccormac, Oscar AND Rochford, William AND Kujawa, Aaron AND Shapey, Jonathan AND Vercauteren, Tom},
title = { { Tree-based Semantic Losses: Application to Sparsely-supervised Large Multi-class Hyperspectral Segmentation } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2025},
year = {2025},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15967},
month = {September},
page = {584 -- 594}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes two tree-based semantic loss functions for hyperspectral image (HSI) segmentation tasks, aiming to leverage label hierarchies to improve performance. Traditional methods treat all errors equally, disregarding the semantic relationships between classes. The authors incorporate their semantic loss functions into a method designed for sparse, background-free annotations. They demonstrate that their approach, applied to a sparsely annotated HSI dataset with 107 classes organized in a clinically-defined semantic tree, achieves state-of-the-art performance. Additionally, their method is capable of detecting out-of-distribution (OOD) pixels without affecting segmentation performance on in-distribution (ID) pixels. The paper presents an advancement for semantic segmentation in hyperspectral imaging, particularly in biomedical applications with limited annotations.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Sparse Annotation Handling: The method effectively deals with sparse, background-free annotations, which is a common challenge in medical and biomedical imaging tasks.
Out-of-Distribution Detection: The ability to detect OOD pixels without compromising performance on ID pixels is a notable advantage, especially for real-world deployment.
State-of-the-Art Performance: The method outperforms existing approaches on a challenging HSI dataset with complex class hierarchies.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Is the dataset used in this paper open-source? If it is, the open-source link should be provided. If it is not open-source, why were experiments not conducted on existing open-source datasets?
As the core expert knowledge relied upon by the proposed method, where does Figure 1 come from? Are there any relevant medical guidelines or expert consensus available as references?
Designing a hierarchical structure for labels to address issues like label sparsity and granularity in large multi-class problems is a rather general approach. The loss design presented in this paper is not particularly novel. Although the authors emphasize that “no previous work in the field of surgical imaging has leveraged the structure of the label space as a source of information,” the paper does not specify what makes this scenario different from others, nor does the proposed method show a specialized design for this particular scenario.
From Table 1, it can be seen that the performance of the two losses used in this paper heavily depends on the weight settings for the edges in Figure 1. However, the exploration of edge weights in this paper is rather superficial. The authors only manually set a few values without providing any reasoning for these settings. Is there a more scientific way to determine the weights? For example, could expert knowledge be used as a reference, or could an additional module be designed to dynamically learn these edge weights?
The references in the paper are formatted inconsistently. The citation format for conference papers is not unified (e.g., CVPR, ICLR, ICML), and some references do not specify the name of the conference or journal where they were published, such as citations 11, 19, 27, etc.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — could be rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Please refer to the weakness section.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The authors introduce two loss functions based on the Wasserstein distance and decision trees to perform sparsely supervised segmentation, enhancing class estimation in hyperspectral images.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper introduces the development of two loss functions for training neural networks, highlighting the following strengths:
- A well-structured and thoroughly explained mathematical framework
- Detailed and descriptive experimentation
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The paper is generally well-written; however, the following observations should be considered:
- The results are very similar, yet no statistical evaluation was performed to identify significant differences (refer to the author’s guidelines).
- Figure 1 lacks clarity and fails to effectively convey the intended information.
- There are some stylistic issues that could be improved to enhance the overall quality of the writing.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — could be accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is written in an accessible and clear manner, effectively highlighting the key points and advantages of the proposed approach.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
Based on the response provided in the rebuttal, I believe the authors have adequately addressed the reviewers’ comments. Therefore, I find it appropriate to recommend the acceptance of the paper for presentation at the conference.
Review #3
- Please describe the contribution of the paper
The authors proposed two tree-based semantic losses for sparsely supervised segmentation.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The authors provide a clear mathematical definition of the proposed losses.
- The authors provide a comprehensive cross-validation to validate the effectiveness of the losses.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- Lack of a simple explanation for sparsely annotated ground truth. Does sparsely annotated ground truth mean that only part of the objects have been annotated, as the qualitative results in Figure 3 show?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — could be accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I recommend that the paper be accepted. The proposed losses leverage data hierarchy into losses is novel and meaningful for tree-structured datasets.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
I recommend accepting the paper. The author mentioned in their feedback that they would clarify the definition of “sparsely annotated ground truth.”
Author Feedback
We thank all reviewers for constructive feedback as they acknowledged the clarity, methodology and detailed experimentation.
R1 and R2 question the clarity of Fig. 1. Fig.1 illustrates the clinically-defined label hierarchy established by consensus among three senior neurosurgeons, embedding expert knowledge at every level. We are the first to use such a refined hierarchy for intraoperative neurosurgical imaging. Providing in-depth clinical justification for this hierarchy is out of scope for this manuscript and will instead be covered in a clinically-focused publication. Nonetheless, we will clarify the consensus basis in the revision and provide an expanded caption in the revised manuscript: “From top to bottom, the hierarchy progresses from coarse object categories to specific classes. The colour coding matches the ground-truth mask at each level”.
R1 requested further analysis of statistical differences. We agree this is a valuable addition and therefore conducted paired-sample t-test on the test image scores. To limit the number of paired comparisons, we focus on F1-score and comparison between the final tree-based results (M_ℓ configuration for L_tce or L_wce) and their corresponding baselines (M_h configuration). Both comparisons were found statistically significant (p < 0.0001). This will be added to the revision.
R2 asks whether our data is open-access and why we did not also use an open-access dataset in our experiments. Our dataset cannot be released publicly as it is part of an ongoing clinical study and our ethics do not allow public data release. The closest available public dataset with sparse surgical annotations, the Dresden Surgical Anatomy Dataset [4], only features 11 categories and lacks a fine-grained label hierarchy. Improving reproducibility is nonetheless important. We plan to release our source code for the proposed losses. A journal extension is currently under preparation in which our tree-based losses are shown beneficial on public datasets featuring dense (rather than sparse) annotations and large label numbers (e.g. TotalSegmentator). Space forbids adding such analysis to this revision.
R2 requesting clarification for novelty of the proposed method. Novelty is along two axes. 1) Our novel loss formulation tackles segmentation using an expert-defined class hierarchy, which we use to encode the fact that some errors are worse than others. While some related work exists for classification in the ML community, our problem has received limited attention in image segmentation with no prior demonstrated performance gain in medical image segmentation. 2) Our loss is integrated in a sparse background-free label training pipeline. This is key to enable practical use cases in surgical imaging. This is the first work (classification included) to exploit label hierarchies in the context of sparse background-free labels.
R2 question about the rationale for the chosen edge weights. These weights intentionally encode expert knowledge: higher-level edges carry larger weights to reflect the greater clinical severity of errors made high in the hierarchy. For instance, misclassifying the ground-truth class “Vein (Normal vascular)” as “AVM Nidal Vessel (Abnormal vascular)” is more serious than predicting it as “Dural vessel (Normal vascular)” because the former is incorrect at both the child and parent levels. Ablation results M_t, M_e, and M_l (Table 1) confirm that embedding this prior knowledge improves performance. However, we acknowledge that better approaches exist for selecting the optimal edge weights. The revised discussion will highlight this limitation and the opportunity to explore hyperparameter tuning algorithms available, e.g. with Ray Tune.
R3 asks for clarification on sparsely annotated ground truth. This indeed means that only parts of foreground objects are annotated, as shown in Fig. 3 column 2. This will be clarified in the revision.
Formatting issues raised by R1 and R2 will be corrected.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- If your recommendation is “Provisional Reject”, then summarize the factors that went into this decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. You do not need to provide a justification for a recommendation of “Provisional Accept” or “Invite for Rebuttal”.
N/A
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A