List of Papers Browse by Subject Areas Author List
Abstract
Automated recognition of the Critical View of Safety (CVS) in Laparoscopic Cholecystectomy is considered a deterministic classification task using majority vote labels. However, expert annotators’ disagreement does not equate annotation error, rather reflecting informative ambiguity in judgement. Reducing such labels to hard targets ignores the underlying sampling process and discards information about observer’s variability. Our work models annotator votes as Binomial observations of a latent visibility probability and learns its predictive distribution via a conjugate Beta-Binomial (B²) loss with evidential parametrisation. The network outputs a Beta distribution over the latent visibility probability, enabling direct estimation of the probability that a majority of experts would judge a CVS criterion as visible. On Endoscapes2023, the proposed approach significantly improves average balanced accuracy by +2.1% compared to the state-of-the-art. We also introduce a precision-critical video-level evaluation protocol reflecting surgical decision-making, reporting recall under a strict precision constraint (precision = 1.0). Under this, the model maintains meaningful recall while avoiding false positives. Modelling annotator disagreement probabilistically provides a principled and clinically aligned alternative to deterministic CVS classification and supports precision-focused evaluation for surgical deployment. Code available at: github.com/franeknowak/BetaBinomialLoss
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/5649_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/franeknowak/BetaBinomialLoss
Link to the Dataset(s)
Endoscapes2023 dataset: https://github.com/CAMMA-public/Endoscapes
BibTex
@InProceedings{NowFra_B²_MICCAI2026,
author = { Nowak, Franciszek M. AND Mazomenos, Evangelos B. AND Davidson, Brian AND Clarkson, Matthew J.},
title = { { B² Loss for Evidential Classification of Critical View of Safety in Laparoscopic Cholecystectomy } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16892},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
Authors propose to use a binomial formulation of the annotator judgments and then use the evidential uncertainty framework for the clinical view safety problem.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The approach and the use of the evidential uncertainty framework is justified for this problem and I believe novel. Formulations are correct and clear.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
I had to look up the CVS problem because the use of the terms “annotator agreement” is misleading: “reflecting annotator agreement, not objective anatomical truth. “ ın general, the problem introduction could be better made.
As for the evaluation, there is a single dataset (of 201 video procedures split into train/val/test sets) and a single comparison method. Furthermore, the results are given for two different thresholds showing good average scores but mixed scores wrt C1-3.Authors give some justification discrediting the performance metrics where the baseline model outperforms, I dont know how that would affect the clinical usage of the system.
Also paper does not provide results of the base model for video-level prediction, while presumably same aggregation could be applied over the 5 frames. ?
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The methodology is sound and relevant and sufficiently novel, however the experimental evaluation seems weak.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The authors approach variation across annotators that is a common source of variation in the ground truth. Using prediction of critical view of safety during cholecystectomy, the authors model the probability of a ground truth label across annotators, use a related loss, and a meaningful evaluation approach using the Endoscapes2023 dataset.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
An attempt to address inter-annotator variation with weak labels. A more sensible approach to evaluation of model than conventional frame-level evaluations.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Per-video evaluations are not accompanied by measures of variance. It is easier using standard statistics to estimate measures of variance for per-video estimands than frame-level estimands. This is a lost opportunity.
I don’t understand the Pred:GT ratio as a metric of model evaluation. This is not necessary because there are other well-established estimands for evaluating with the video as the unit of analysis.
Variance across models with different seeds is not necessarily of interest to understand the statistical variation we seek to understand.
Table 2 - paired t-tests are not the appropriate method here. If the authors insist on statistical testing, then a statistician should be consulted to use appropriate methodology.
Limitations in fitting the data to an empirical binomial should be acknowledged. The sampling mechanism for selecting annotators becomes important, and should be explained/emphasized in this paper.
The “strong class imbalance” in CVS visibility is not clearly explained. It is a consideration for frame-level estimands but not video-level estimands. The KL divergence term is not clearly substantiated.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
The font size of axis labels in Figure 3 is too small to read.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(2) Reject — should be rejected, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The approach is intuitive and interesting, but it is not well thought out in terms of when it may be applied and assumptions made. The model evaluation is weak.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors acknowledged limitations. Statistics still iffy, but within reason.
Review #3
- Please describe the contribution of the paper
The authors propose a statistically grounded Beta–Binomial evidential framework for CVS recognition that explicitly models annotator disagreement and integrates uncertainty directly into predictive probabilities.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Evidential deep learning is a well-suited approach for the medical domain, where inter-annotator variability is a persistent challenge. I find the idea of the paper both timely and well-motivated.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The authors state that “the final six transformer blocks are first adapted on majority-vote labels.” Could this design choice bias the model toward majority-vote supervision, potentially undermining the benefits of explicitly modeling annotator disagreement? Additionally, was training without this adaptation considered? The paper does not state whether the code will be made publicly available, raising reproducibility concerns.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper proposes a well-motivated Beta–Binomial evidential framework for CVS recognition that explicitly models annotator disagreement and uncertainty, which is an important and timely problem in medical image analysis. The use of evidential learning is appropriate and shows promising direction for handling label variability. However, the training strategy raises minor concerns, particularly the initial adaptation using majority-vote labels, which may partially bias the model and reduce the impact of explicitly modeling disagreement. Further clarification or ablation would strengthen the work.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
I am satisfied with the authors’ response, and I believe the idea of explicitly modeling different annotators outweighs some of the limitations in the evaluation details. I hope this work will inspire further research in the medical domain.
Author Feedback
We thank R1, R2, R4, MR for their constructive reviews. We are encouraged our work is “sound, relevant and sufficiently novel-R1”, “intuitive and interesting-R2”, “well-motivated, promising and addresses an important and timely problem-R4”. We have clarified annotation assumptions, evaluation design and statistical rigor. Replies will be incorporated in the final paper.
R1-Task definition: CVS criteria labelling is subjective, confirmed by annotators’ Cohen’s kappa 0.33-0.55.All past works use majority vote, collapsing partial agreement into binary ground truth. This assumes that the disagreeing expert is incorrect. We are the first to maintain fraction labels, thus not discarding information about ambiguity in the images. This will be clarified in the introduction.
R1-Single dataset/comparison: Endoscapes2023 is used as the most comprehensive and only publicly available benchmark with standardised splits and three-annotator CVS labels; Cholec80 lacks both. The CVS Challenge dataset was released too close to submission (28/01/26) for full integration, though we since added it to the codebase for future analysis. SwinCVS is chosen as the SOTA method on Endoscapes2023 using the same weakly annotated inputs.
R1-Mixed per-threshold scores; video-level baseline: Tab.3 shows that per-sequence metrics are clinically misaligned; a single false positive may authorise premature transection. We report video-level results only for B² because the evaluation critique concerns the protocol, not the architecture; both B² and SwinCVS process 5-frame clips. Thresholding addresses the argued sensitivity concerns.
R2-Statistical testing and variance: Confirmed with statistician, paired t-test is relevant to compare differences of two classifiers on same test set. We also report non-parametric Wilcoxon signed-rank test. Significance levels are unchanged - B² significantly outperforms SwinCVS at the same metrics. Seed-based variance is a standard to account for initialisation bias when comparing models.
R2-Per-video variance and Pred:GT: We applied stratified bootstrap resampling (1,000 resamples, preserving the positive/negative video ratio) to existing test predictions, and calculated 95% percentile CIs for all relevant metrics; which will be added to Tab.3.LCB in line with paper’s discussion. Pred:GT has been replaced with more established specificity.
R2-Binomial assumptions: We model CVS annotators as exchangeable, independent, expert observers whose disagreement reflects perceptual ambiguity rather than systematic error. The latent quantity is therefore the probability a randomly selected expert would judge a CVS criterion as visible, for which the Binomial likelihood is well suited. Independence and exchangeability are the minimal assumptions implicit in any majority-vote labelling scheme. These assumptions and constrained use-cases will be made clear.
R2-Class imbalance; KL divergence: We agree imbalance is a frame-level property (reported in Table 1); however, miscalibrated evidence estimates, caused by imbalance, propagate directly to video-level aggregation, so the regularisation benefits video-level performance indirectly through improved frame-level calibration. The KL term is a regulariser, penalising deviation from a prevalence-weighted prior Beta(πν,(1-π)ν), constraining evidence growth in data-sparse regions. We consider this a principled Bayesian approach that prevents overconfidence under reported imbalance.
R4-Pre-training on majority-vote labels: Pre-training addressed the perceived ImageNet-Endoscapes domain gap. Risk of bias was considered and tested before initial submission. Without adaptation on both models, B² retained significant improvement over SwinCVS (in BACC, p<0.05), confirming no meaningful bias was introduced. Adaptation was also deliberately conservative (lr=1e-6, early-stopping, classifier head discarded). Both variants are reproducible from our public codebase available upon acceptance; see Sec 2.3.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper presents a Beta–Binomial evidential learning framework for Critical View of Safety (CVS) recognition, explicitly modelling annotator disagreement and incorporating uncertainty into predictions. Reviewers agree that the core idea is timely and that evidential learning is an appropriate choice for handling variability in surgical annotations. However, notable concerns remain about the evaluation design and statistical rigor. The experiments rely on a single dataset with limited baselines, lack appropriate variance estimates at the video level, and employ evaluation measures and statistical tests (e.g., Pred:GT ratio, paired t‑tests) that are insufficiently justified. Additional issues include unclear assumptions behind the binomial modelling of annotations, limited discussion of the annotator sampling process, ambiguity around class imbalance in video‑level analysis, and training choices (such as initial adaptation to majority‑vote labels) that may dilute the intended benefits of explicitly modelling disagreement. Clearer problem framing, more coherent statistical evaluation, and a more explicit discussion of assumptions, limitations, and clinical interpretation of the uncertainty estimates is needed. Authors should address each concern raised by the reviewers.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The rebuttal directly addresses most major concerns, including clarifying the statistical assumptions behind the Beta–Binomial model, improving evaluation by replacing questionable metrics and adding confidence intervals, and better justifying the choice of tests and dataset. While some limitations remain (single dataset, moderate statistical rigor, residual concerns about assumptions), these are acknowledged and reasonably justified, and do not undermine the core contribution. The idea of explicitly modeling annotator disagreement remains novel and valuable. Reviewers are generally in favor of Acceptance.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The authors have responded in a satifactory manner to the points raised by the reviewers.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The rebuttal successfully addressed many concerns regarding statistical assumptions, evaluation design, and the rationale for modeling annotator disagreement. Although some concerns remained regarding the statistical methodology, limited evaluation on a single dataset, and the overall strength of experimental validation, reviewers considered these limitations acknowledged and reasonably justified. The post-rebuttal consensus suggests that the methodological contribution and clinical relevance outweigh the remaining weaknesses. therefore I recommend acceptance.