List of Papers Browse by Subject Areas Author List
Abstract
Label variability is one of the major challenges in prostate lesion segmentation. In multi-site datasets, annotations often reflect site-specific contouring protocols. This can bias segmentation networks towards local annotation styles and reduce generalisation to unseen sites during inference. In this study, we treat each observed annotation as a noisy observation of an underlying latent “clean” lesion mask, and propose a hierarchical expectation-maximisation (HierEM) framework that alternates between: (i) inferring a voxel-wise posterior distribution over the latent mask, (ii-1) training a UNet that uses this posterior as a soft target, and (ii-2) estimating site-specific sensitivity and specificity with a logistic-normal hierarchical prior. The hierarchical prior decomposes label quality into global, site- and case-level components, which regularises site-level deviations, encouraging the model to avoid overfitting to centre-specific annotation styles, thereby promoting site-invariant segmentation.
Experiments on three-site datasets demonstrate that the proposed HierEM framework enhances cross-site generalisation compared to state-of-the-art methods. In the pooled datasets evaluation, per-site mean DSC ranges from 29.50% to 39.69% across datasets; for leave-one-site-out generalisation, it ranges from 27.91% to 32.67%, yielding statistically significant improvements over comparison methods (p<0.039). The method also produces interpretable per-site latent label quality estimates (e.g., sensitivity α ranging from 31.5% to 47.3% at specificity β~0.99). These results indicate that explicitly modelling site-dependent annotation can improve cross-site generalisation. Code is available via https://github.com/yanwenCi/HierEM.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1086_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/yanwenCi/HierEM.git
Link to the Dataset(s)
Dataset1 was obtained from UCLH with ethical consent. It is private dataset without link to it. Detailed description can be found in https://doi.org/10.1016/j.media.2023.103030
Dataset 2 is publicly available via https://doi.org/10.5281/zenodo.15683922
Dataset 3 is private dataset from Miami, FL with ethical consent.
BibTex
@InProceedings{YanWen_Deep_MICCAI2026,
author = { Yan, Wen AND Wang, Yipei AND Huang, Shiqi AND Thorley, Natasha AND Emberton, Mark AND Stavrinides, Vasilis AND Punwani, Shonit AND Hu, Yipeng AND Barratt, Dean C.},
title = { { Deep EM with Hierarchical Latent Label Modelling for Multi-Site Prostate Lesion Segmentation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16881},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper proposes a segmentation framework for prostate cancer lesion delineation in multi-parametric MRI (mpMRI) that explicitly models multi-site and multi-rater annotation noise. Rather than treating heterogeneous labels from different clinical sites as ground truth, the method introduces a latent consensus mask as the true underlying segmentation, with each site’s observed labels modeled as noisy transformations of this latent variable through site-specific confusion matrix-style noise models. The core clinical motivation is strong: real-world datasets are frequently annotated by single readers and existing methods rarely account for this structured label noise explicitly.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Clinically important problem formulation: The paper addresses a challenge: real-world clinical datasets are rarely annotated by multiple raters on the same image, yet label heterogeneity across sites is pervasive. Rather than the classical multi-rater setup where disagreement is directly observable, the framework treats each site as a proxy annotator, inferring noise distributions. This is a pragmatic and reasonable adaptation to how multi-site data is actually collected.
Principled generative model: The hierarchical formulation, latent clean mask as the unobserved truth, observed labels as site-conditioned noisy measurements, is theoretically coherent and draws meaningfully from the noise modeling literature (e. g. , confusion matrix-style annotator models), adapting it sensibly to the multi-site setting.
Leave-one-dataset-out evaluation: The inclusion of a leave-one-site-out experiment is a commendable and well-designed validation choice. It directly probes the risk of the model collapsing toward a single site’s annotation bias and provides meaningful evidence of generalization across institutional protocols.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Hyperparameter sensitivity not analyzed: Given that these hyperparameters directly shape the learned noise model and the inferred latent consensus mask, showing their influence on downstream segmentation performance can strengthen the paper.
Insufficient ablation study: Although it could be from limited pages, but the paper lacks ablations that isolate the contribution of the hierarchical structure from the noise modeling component. It is unclear whether the performance gains stem from the latent variable formulation, the site-specific noise model, or simply from multi-site data aggregation with soft labels. These components should be evaluated independently.
Dataset effect not fully characterized: It is not demonstrated whether the learned site-specific noise parameters actually capture meaningful protocol or cohort differences (e. g. , scanner type, field strength, annotation guidelines). Authors might be able to analyze dataset-level distributional shift in an uninterpretable way. PIRADS-stratified analysis: Given that annotation noisiness likely varies systematically across PIRADS grades (with grade 3 lesions being substantially more ambiguous than grades 4 and 5) a stratified performance analysis across PIRADS categories would strengthen the evaluation. (minor) The Dice metric is incorrectly reported with units of mm, which should be unitless. The typo “nn” instead of “mm” also appears and should be corrected throughout. (minor) data details needed (e. g. , preprocessing steps)
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper addresses an important problem in medical image segmentation: modeling structured annotation noise arising from multi-rater/site) clinical label heterogeneity. The hierarchical generative framework is theoretically motivated and the leave-one-site-out evaluation is a thoughtful design choice. However, the work requires more explanations and ablations. With revision addressing these issues, this work has the potential to make a meaningful contribution to the field.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors addressed the major concerns raised (especially the ones R2 raised) and promised that the discussion will be reflected in the camera-ready version.
Review #2
- Please describe the contribution of the paper
This paper proposes HierEM, a hierarchical expectation-maximization (EM) framework for multi-site prostate lesion segmentation that explicitly models site-dependent annotation variability. Treating each site’s observed contour as a noisy observation of an underlying latent ‘clean’ mask, the method estimates site- and case-level sensitivity/specificity parameters arranged in a three-level hierarchical prior (global mean, site offset, case offset). The E-step infers a voxel-wise posterior over the latent mask by combining the network’s image prior with the annotation likelihood; the M-step updates the UNet segmentation backbone, and re-estimates label quality parameters via MAP optimization. The framework is evaluated on three under both a pooled patient-level split and a leave-one-site-out (LOSO) protocol, compared against vanilla UNet, bootstrapping, and a flat site-EM without hierarchy.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The design that formulates each site’s annotation as a noisy observation of an underlying clean annotation is well-motivated. The decomposition of label variability into global, site-specific, and case-level components provides a natural mechanism for partial pooling via the hierarchical Gaussian prior. The leave-one-site-out (LOSO) experiment clearly compares whether uncertainty generalizes across sites.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.There are report discrepancies between the text and the table, which can undermine the reliability of the results: a. ‘Supervised Dice decreases to 25.50/24.66/33.09% across Sites 1–3’, but the table reported 31.20% for site 3. b. ‘HierEM consistently improves generalisation, yielding higher Dice on Sites 1 and 2 (28.11 and 27.40 mm)’, but mm is clearly not the unit for Dice, and it should be 28.11/27.91%. c. ‘with lower HD95 (20.51 and 20.31 nn)’, nn is incorrect unit, and it should be 20.51/21.12 mm. 2.Dice for HierEM in site 1 has a very large standard deviation (21.88), if this was not a reporting error, the authors need to explain this finding. 3.The work was an extension of STAPLE, where multiple annotations of the same case were used to estimate annotator sensitivity and specificity. In this study, each case has exactly one annotation from one site and the sensitivity/specificity was pooled performance across cases within a site. This potentially assumes that all annotators at a given site perform identically, which is hardly achievable even under the same reading protocol. The authors should discuss this further and the influence of theoretical/realistic aspects of the design on the estimation of alpha and beta. 4.Minors: a. Misspelling like ‘generalisation’ was used throughout the paper, they don’t significantly affect reading but it’s good to correct them. b. The risk-coverage curve in fig.2 will benefit from an area under the curve-like metrics to help interpretation.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This study addresses a major challenge: site-specific variability in annotation for prostate cancer segmentation. The theoretical foundations of the study were sound, extending STAPLE to a multi-site training scheme. However, the discrepancy between the metrics in the text and the table hinders understanding and raises questions about the reliability of the results. The discussion of uniform annotation behavior within sites needs to be addressed to further justify the design choice,
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
I thank the authors for the response. I find the explanation of u_k and v_k for case-level difficulty helpful. I’m raising my rating. I’d also suggest the authors to highlight the 3 components when introducing Eq.2 in the method section, since the point was actually made clear in the introduction but the reader may be somewhat lost when they reach Eq.2.
Review #3
- Please describe the contribution of the paper
The authors propose a hierarchical expectation–maximisation (HierEM) framework that alternates between: (i) inferring a voxel-wise posterior distribution over the latent mask, and (ii) training a CNN using this posterior as a soft target and estimate site-specific sensitivity and specificity under a hierarchical prior.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Multi site training an evaluation is a large problem in deep learning for medical image analysis. Performance typically drops when algorithms are validated in sites that they weren’t trained upon. HierEM is an elegant framework that addresses label bias between hospitals, to train a model that is more robust for site specific labeling. The method seems broadly applicable, on any scenario where multi institutional data is available. The sharing of the code, and extensive mathematical formulations are much appreciated.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The reported dices all seem rather low. As shown by the literature in the paper, this is because intra operator agreement is already topped at 0.4 dice for these type of annotations. As only an in house dataset is used, it is currently unclear if the method also extends to more moderate use cases, where annotations agree more.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(6) Strong Accept — must be accepted due to excellence
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The work addresses a relevant question, is strongly methodological and generic and elegant. By the methodological rigor, and addition of the code, the method can quickly be adapted by others after release.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
After reading the rebuttal, the other reviews, and the meta-review, I maintain my recommendation to (weakly) accept this work. Both R2 and the meta-review emphasize the alleged lack of modeling of inter-institute variance in HierEM; however, this does not appear to be the case. My understanding of the work is that HierEM is capable of capturing these sources of variability.
Author Feedback
We thank the comments from the reviewers and the clear guidance from the AC.
Reviewer 2: single annotation per case / STAPLE assumption We agree that annotators within the same site should not be identical, even under the same protocol, as suggested by R2.But we clarify that we do NOT assume uniform behaviour within sites. First, algorithmically, HierEM includes case-specific latent variables, u_k and v_k, in addition to the site-level variables (a_s, b_s) in Eq. 2, modelling the hierarchy of global, site-level, case-level variabilities. During training, the model specifies probabilities of observed labels given unknow latent clean label(Eq.1), and allows each case to move away from global and site averages by case variables (u_k, v_k) (eq.2). For reporting, site-level statistics can be estimated by pooled sensitivity and specificity over within-site annotators, but these do NOT assume all annotators at a given site perform identically. Second, from the data point of view, HierEM also addresses a different problem from classical STAPLE. STAPLE experiments used multiple annotations for every case, allowing individual-reader sensitivity and specificity to be estimated directly. Our HierEM is designed for multi-site data with single annotation for each case. It is noteworthy that, a site may involve multiple annotators, but each case is annotated only once. In practice, such single-annotation-per-case multi-site datasets are much more commonly available than datasets requiring multiple annotators to annotate every case. These within site annotators are expected to follow the same site-level annotation protocol, for example PI-RADS in dataset1 and Likert in dataset2, but their actual labelling behaviour may still differ due to individual judgement, experience. In HierEM, such residual case-wise discrepancies are not forced into a single site-level α/β. Instead, the case-level latent variables (u_k, v_k) allow the effective sensitivity and specificity, \alpha_{s,k} and \beta_{s,k}, to vary from case to case.
Reporting results We sincerely thank R2 for identifying several typos and formatting issues. We are grateful for the careful reading and humbled by the attention to detail. Among these, we clarify that 1) the current value for Dice in Site3 is 31.20% both in table and text; and 2) the correct standard deviation for Site 1 should be 11.88 instead of 21.88.We will correct these. Following the reviewer’s suggestion, we also added AURC to Fig. 2, with smaller area indicates lower risk.
Ablation concerns of contribution of the hierarchical structure We use the same strong and efficient U-Net backbone across all methods to ensure a fair and controlled comparison. Since our method is agnostic to the segmentation backbone, this controlled setting reduces architectural differences as a potential factor. The experimental design is progressive and aims to demonstrate the contribution of site- and case-level modelling: we start with U-Net baseline, then add soft-label bootstrapping, flat EM modelling without case-level variables, and finally the proposed HierEM. The experiments serve not only as comparisons against relevant baselines, but also as an ablation of the main methodological components. We agree that a more systematic comparison study could further strengthen the work and will consider in future work.
PIRADS-stratified analysis by reviewer1 This study included cases contain PI-RADS/Likert ≥3 lesions. We focus on comparing methods within this clinically meaningful lesion-positive population. We agree that PI-RADS/Likert-stratified evaluation would be clinically valuable, may potentially expand clinical applicability, although due to the fixed diagnostic patient cohort used in our test set, such stratification needs be carefully designed to reflect any intended real-world patient cohorts. Following the AC suggestion, we will acknowledge these.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes HierEM, a hierarchical expectation–maximization framework that treats each site’s annotation as a noisy observation of an underlying clean mask and decomposes label variability into global, site-, and case-level components. The reviewers agree that the problem is clinically important, the hierarchical generative formulation is theoretically coherent, and the leave-one-site-out evaluation is a clever design choice.
Scores split 4/3/6.R1 and R2 raise concerns that are specific and grounded. R1 notes the paper lacks ablations isolating the contributions of the hierarchical structure, the site-specific noise model, and soft-label aggregation, and that no hyperparameter sensitivity or PIRADS-stratified analysis is reported. R2 identifies several discrepancies between the text and the tables (including wrong units and conflicting Dice numbers), flags an unexplained large standard deviation on Site 1, and raises a more substantive concern that extending STAPLE to a setting where each case has a single annotation implicitly assumes uniform annotator behavior within a site. R3 gives a Strong Accept, but the review is short and does not engage with the concrete details of the paper nor it justifies why the paper is outstanding, so I gave it less weight. R1’s review is also superficial and raised concerns that are somewhat generic.
Given the split scores and somewhat unconvincing reviews (which is unfortunate for the authors and the community), I went through the paper myself and found R2’s point on the uniform-within-site assumption the most significant one; it touches the validity of the estimation that the whole framework rests on, and the authors should engage with it carefully, including possible clarification, implications, and any conditions under which the assumption is reasonable. The reporting discrepancies (units, conflicting Dice numbers, unexplained Site 1 standard deviation) are smaller in scope but should also be addressed, since they affect how the results are read. The ablation and sensitivity requests are harder to resolve in a rebuttal but are worth acknowledging. I invite the authors to respond before a final decision. Please refer to MICCAI rebuttal guidelines. Good luck.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
I thank the authors and all three reviewers for their active participation in the rebuttal phase. Following the rebuttal, the authors effectively addressed the significant concerns raised during the initial review round, including inconsistencies in results, underlying assumptions of the method, and potential limitations. All three reviewers agreed that the paper should be accepted. Considering the merits in the paper, authors’ effective rebuttal, and the unanimous agreement from the reviewers, I recommend acceptance.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Congratulations! Well done
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Reviewer consensus achieved.
