List of Papers Browse by Subject Areas Author List
Abstract
Deep neural networks remain vulnerable to adversarial perturbations, which can distort not only predictions but also confidence scores, undermining uncertainty calibration. While existing certification methods focus on preserving the predicted category, providing guarantees on how calibration behaves under adversarial attacks remains overlooked. In this work, we introduce CalCErt, a simple and efficient post‑hoc strategy that certifies bin‑wise confidence calibration for any pretrained differentiable classifier. Our approach combines empirical calibration estimates, statistical concentration bounds, and local Lipschitz estimates of the confidence function to derive data‑dependent upper bounds on worst‑case miscalibration within an l2-ball of radius R. We evaluate CalCErt across 11 medical image classification tasks and multiple adversarial perturbations, demonstrating substantially higher certified coverage than baseline strategies while maintaining competitive tightness.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2687_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/leofillioux/calcert
Link to the Dataset(s)
N/A
BibTex
@InProceedings{FilLeo_CalCErt_MICCAI2026,
author = { Fillioux, Leo AND Christodoulidis, Stergios AND Cournède, Paul-Henry AND Vakalopoulou, Maria AND Dolz, Jose},
title = { { CalCErt: Bin-wise Certification of Confidence Calibration in Medical Image Classification } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16882},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a novel post-hoc method to estimate an upper bound on calibration error under bounded adversarial perturbations. Their estimate relies on a local approximation of the confidence constants, using the gradient of the confidence function. Authors conduct extensive experiments using well-established datasets in the medical image classification domain.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper is generally well written and easy to follow. 2.Authors verify trustworthiness and usability of the proposed upper bound on calibration error under various bounded perturbations.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
I have concerns regarding evaluation metrics: 1.It seems like those high coverage rates come at the expense of tightness. Authors should use a principled metric that reflects the trade-off between tightness and coverage in the main table. 2.The problem with the “overshoot” metric is as follows: if your bounds are not tight, then naturally the overshoot will be small because the bounds are close to the rare bins where the bounds are violated. Thus, the “overshoot” metric is actually misleading: it will be lower for bounds with inferior tightness.
I also have remarks concerning notation and readability: 1.It would be beneficial if you had explained how these notions of uncertainty (statistical, empirical, and noise-based) relate to well-established uncertainty decomposition into aleatoric and epistemic parts 2.Notation would benefit from polishing. In your definition, Ak(δ) is a function of δ, but at the same time, you treat it as a scalar.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Table 1 is hard to interpret because it presents a trade-off between coverage and tightness, and the main table does not reflect how well the method balances coverage and tightness. The proposed “overshoot” metric is highly misleading because it is lower for methods with worse tightness. Based on this unreliable and unjustified metric, the paper tries to prove that CalCErt’s tightness is actually good.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
After authors’ response, I still have concerns with the metrics presented and how impactfull the paper would be.
Review #2
- Please describe the contribution of the paper
This paper proposes a method for providing bin-wise bounds for calibration errors under adversarial attacks. The method is based on local Lipschitz bounds and can be applied to any differentiable model. The result is an upper bound of the calibration error per confidence bin, under perturbations. To my knowledge, this is the first work on calibration certification. Alongside the proposed method, the authors propose a simple baseline and evaluate their method on MedMNIST, where the proposed method consistently outperforms the baseline.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This seems to be the first work on calibration certification. The authors compellingly motivated the problem and I agree this is a problem well worth solving.
- The paper (especially the introduction and methods section) are very well written, easy to follow, concise and clear – well done.
- The proposed method seems theoretically sound and elegant.
- The proposed method is posthoc and does not rely on any (computantionally heavy) adversarially robust training.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
-
I suspect that the evaluation is slightly biased. The proposed method is evaluated only on samples in robust bins (i.e. narrower bins), so it seems natural that the coverage is better. Would it make sense to evaluate the coverage on the same data (robust bins only) for fair comparison?
-
Along similar lines: Since the proposed method uses robust bins, what happens to the samples that are not in any robust bins because they are close to the bin boundaries? Are there any calibration bounds for them, and if not, could the authors justify this?
-
I believe an analysis of lambda, the Lipschitz constant R and the proportion of certified samples would be relevant and interesting, but this is currently missing from the manuscript.
Detailed comments:
-
In Sec. 4.2, the authors day “CalCErt certifies substantially more samples”. What is meant by this? Is this shown in the results anywhere?
-
“Bin-wise Certification of Confidence Calibration certification methods [11, 7, 11, 17, 36]”: Ref 11 is listed twice
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Certification of calibration is a completely novel but clinically meaningful contribution
- The proposed method is solid and seems to work well
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
I keep my accept recommendation as I still think it’s an innovative and timely idea, also after reading the other reviewer comments.
Review #3
- Please describe the contribution of the paper
This paper proposes CalCErt, a post-hoc framework that bounds worst-case calibration error under L2 perturbations using empirical calibration, concentration bounds, and local Lipschitz estimates.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper studies adversarial robustness from an underexplored problem which is certifying calibration robustness, not only prediction robustness. 2.The training-free nature of the method makes it post-hoc, furthermore it is architecture independent so it can be readily deployed on pretrained models 3.The number and extent of explorations are good in terms of architectures and datasets
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The guarantee relies on an approximate local Lipschitz estimate which is based on the gradient norm of the confidence function - the the certificate is not fully exact. 2.If I understand correctly, the method is calibration only, and does consider the robust accuracy, so a model may still be confidently wrong under perturbation. 3.The adversarial evaluation is somewhat limited, relying mainly on FGSM, BIM, PGD, and RFGSM, which weakens the empirical stress test - AutoAttach or the optimization based CW-L2 would be good additions
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The formulation is clean, and the empirical results are considerable especially in terms of certified coverage. However, the theoretical guarantee is weakened by the use of a first-order Lipschitz approximation, robust accuracy is not considered, and more advanced adversarial attacks are not considered
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
The reviewers have not properly answered the weaknesses I mentioned - specially their explanation on the calibration vs. accuracy case is very loose
Author Feedback
R1.Coverage/tightness tradeoff. We agree that both metrics should be read jointly. We could look at a tradeoff in the form of C=coverage-Ⲗ*tightness. We can report C in Tab. 1 for Ⲗ=1, and also add a Pareto plot for varying values of Ⲗ. The average value of C (Ⲗ=1) for 0.11 for L2-CAL, 0.24 for ASC, and 0.53 for CalCErt.
R1.Overshoot is misleading. This is a fair point, a method which predicts very high bounds, will have a high tightness, but no overshoot. However, we still believe that the fact in the cases where the bounds are violated, it is not violated as much as other methods is interesting. We argue that it is preferable to have a method which predicts looser bounds, but which rarely fails, and fails at a small margin, rather than a method which tries to get tight bounds, but get large errors when it fails.
R1.Aleatoric/epistemic link and notation. We will clarify in the refined version that the statistical deviation corresponds to epistemic uncertainty because it corresponds to a finite-sample estimation of \hat{A_k}, which would vanish with sufficiently large data. Similarly, the Lipschitz deviation reflects epistemic uncertainty, as it is model-induced. The empirical miscalibration corresponds to the residual epistemic uncertainty and the aleatoric uncertainty.
R2.Robust bins only. We agree that this is a fair point raised by the reviewer. However, we would like to specify that all methods are evaluated on the same dataset, and that the sub-bins do not impact the samples on which CalCErt is applied. We will make sure to modify the manuscript so that it is clear to the reader.
R2.Ablation for values of R. We agree with R2 that this analysis would be interesting for the reader. We propose to include an ablation sweeping for multiple values of R, and to report the mean Lipschitz value L_k, width of the sub-bin γ_k, coverage, and tightness. Due to lack of space, we could remove other results such as the ResNet 18 table.
R4.Approximate Lipschitz estimate. This is correct, we use a gradient-based estimation, which yields a first-order certificate. This allows to make the computation tractable. We will make sure to emphasize this point in the refined manuscript.
R4.Calibration vs accuracy. The method is indeed focused on certifying calibration and not accuracy. The model may then be confident about a wrong prediction, but if it is calibrated, marginally over the dataset, it should be confident on a wrong prediction only on a fraction of the samples.
R4.Other attacks. Our objective with CalCErt is not to defend against any specific attack, but rather to provide a post-hoc, model agnostic approach to calibration certification. The bound in equation (4) holds for any value of δ<R, so it covers any l2-bounded attack. CalCErt requires no retraining, or attack-specific finetuning. We will clarify this framing further in the refined version of the manuscript, and emphasize that this work is a first step towards calibration certification.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper proposes a novel post-hoc method to estimate an upper bound on calibration error under adversarial perturbations. The paper is a useful contribution to responsible AI direction; however, there are some concerns raised by the reviewers that must be addressed, e.g., evaluation of the trade-off between tightness and coverage, a fairer comparison, and experiments with advanced adversarial attacks.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
This paper is an early attempt in the direction of certification of calibration by proposing bin-wise bounds for calibration errors under adversarial attacks. The paper received mixed reviews. The reviewers’ concerns are genuine, especially the trade-off between coverage and tightness, and limited immediate impact. However, I feel that the paper should be of interest to the community working towards responsible AI and motivate future research in this direction.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation.
While the authors have provided a reasonable rebuttal, two of the reviewers remain unconvinced about the key metrics and tradeoffs involved. Given the theoretical nature of the contributions, the paper would benefit from a longer version with stronger proofs and justifications. Hence, the paper cannot be recommended for acceptance despite its technical contributions,
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The paper presents an interesting and underexplored study on adversarial robustness, focusing on certifying calibration robustness rather than only prediction robustness. This is a valuable direction, especially for medical imaging, where reliable confidence estimates are critical. The post-hoc nature of the proposed method further increases its practical relevance, as it can potentially improve robustness without requiring major changes to existing models. The empirical results appear strong enough to justify the benefits of the approach, and the concerns raised during review were addressed in an acceptable manner. Overall, the paper offers a unique and meaningful contribution, and I recommended for acceptance.
