List of Papers Browse by Subject Areas Author List
Abstract
Vision impairment affects 2.2 billion people globally, yet scalable screening is hindered by workforce shortages. We introduce OphFM-bench, a national-scale benchmark for early multi-retinopathy screening using color fundus photographs. The dataset includes over half a million adults from 1,010 hospitals across 27 Chinese provinces, capturing diverse devices and population structures. We evaluate three foundation model families—retina-specific models, general medical vision-language models (VLMs), and general-purpose backbones across three dimensions: Screening Efficiency (high-sensitivity regimes), Spatial Generalization (geographical shifts), and Algorithmic Fairness (age, sex, and residency). While retina-specific models like RETFound-DINO excel in ranking, medical VLMs like MedSigLIP offer superior transfer efficiency and robustness in high-recall scenarios. Notably, we identify a performance-calibration gap where high AUC masks significant false-positive burdens in low-prevalence areas. OphFM-bench establishes an actionable framework for equitable AI deployment in national-scale screening networks.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/4580_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{CheWen_Benchmarking_MICCAI2026,
author = { Cheng, Wenquan AND Guo, Jia AND Wang, Jinyuan AND Sun, Yihua AND Han, Haojie AND Chen, Fang AND Liao, Hongen AND Wong, Tien Yin AND Sun, Xiaodong AND Jia, Huixun},
title = { { Benchmarking Foundation Models for Early Multi-retinopathy Detection at National Scale: Performance, Generalization, and Bias } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16895},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper presented OphFM-bench, a national-scale benchmark for evaluating foundation models in early multi-retinopathy screening using color fundus photographs, in China. The study leverages a substantial dataset comprising over 500,000 adults from 1,010 hospitals across 27 Chinese provinces, representing significant geographical and demographic diversity. The main contribution: systematic comparison of three foundation model families—retina-specific models (e. g. , RETFound-DINO), general medical vision-language models (MedSigLIP), and general-purpose backbones—across three critical dimensions: Screening Efficiency, Spatial Generalization, and Algorithmic Fairness.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
Large-scale, diverse dataset: It’s a large study, providing substantial geographical and socioeconomic diversity rarely seen in ophthalmic AI research.
-
Systematic and thorough analysis: The paper conducts a comprehensive evaluation across three model families with rigorous assessment of linear probing versus full fine-tuning strategies. The analysis is well-structured, providing comparisons of screening efficiency, generalization, and fairness metrics.
-
Actionable clinical insights: The identification of adaptation regimes that remain reliable under national-scale distribution shifts provides practical guidance for screening use-cases. The analysis of the “fine-tuning paradox”, where full fine-tuning improves overall performance but frequently exacerbates parity violations at fixed thresholds, offers valuable deployment recommendations.
-
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
-
Limited methodological novelty: The paper does not propose new methods or novel evaluation frameworks beyond standard benchmarking protocols.
-
Dataset accessibility: The dataset will not be made publicly available, which limits the practical utility of the benchmark. While the scale is impressive, the inability to reproduce results on the exact dataset reduces the long-term impact of the work.
-
Conference targeting: While the analysis is thorough, the paper could be framed more specifically for the MICCAI audience by emphasising technical lessons about model selection and adaptation strategies, rather than primarily epidemiological findings.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
The paper provides a valuable systematic analysis of foundation model performance in large-scale retinopathy screening. To strengthen the contribution for the MICCAI community, consider emphasizing the technical lessons regarding model adaptation regimes and the trade-offs between specialist and generalist architectures. Additionally, while the dataset cannot be released, providing detailed protocol documentation or synthetic data samples could enhance reproducibility. The discussion on the “fine-tuning paradox” and equity trade-offs is particularly relevant for clinical translation and could be highlighted further.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
To me, this paper is a (borderline) accept due to its systematic and thorough evaluation of foundation models at an unprecedented national scale. While it does not introduce novel methodological techniques, it provides important empirical insights into the practical deployment of AI for retinopathy screening, particularly regarding the equity and generalization challenges faced by telemedicine networks. The work is well-structured, and addresses a significant public health need with a somewhat rigorous analysis. Although the dataset is not publicly available, the methodological clarity and scale of the study could provide value for the MICCAI community focused on clinical translation of medical image analysis.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The paper benchmarks multiple publicly available generic-, medical-, retinal-foundation models(FMs) by finetuning (full- vs linear probing) them on large national-scale dataset collected across china. These models were compared for various performance metrics, including screening relevant performance criteria - such as Recall@95%Specificity and Specificity@95%Recall, robustness to geographic variability, fairness across age groups, gender and residency. The national-scale dataset collected consists of ~605K images from ~1k Hospitals across 27 provinces in China. The benchmarking performance of these finetuned models show that Medical-foundation Models such as MedSegLIP are able to compete and even surpass SOTA retinal-specific foundation models such as RetFound-DINO. The performance on Medical Foundation models – specifically MedSegLIP - as pretraining base seem to be strong and consistently outperform other models both under Linear Probing and Full Finetuning training regime. Both MedSegLIP and RetFound-DINO seem to perform comparatively well compared to other architectures on geographic generalizability and fairness metrics. Different finetuning techniques also seem to change calibration in that full finetuning improves in-domain performance compared to linear probing but at the cost of reduction in fairness metric.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The large-scale benchmarking on national scale dataset provides considerable strength to the evidence generated. Additionally, experimental setup – with multiple FMs across generic-, medical- and retinal-based ones, task specific (screening relevant S@95R and R@95S) metrics, multiple finetuning strategies, in-domain performance as well as robustness to geographic variation, albeit country-specific, and fairness evaluation, has been well executed. The generated evidence and outcomes are very interesting including tradeoff between in-domain performance and fairness metric, the surprising effectiveness of generic medical-specific foundation model , in this case MedSegLIP, as a pretraining base for retinal fundus image-based screening, being competitive and even surpassing retinal fundus (and OCT) specific FMs, on certain metrics and tasks, is of value, interest and could be potential exploration opportunity.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Not necessarily a weakness, but additional visualizations and analysis of the reported results/table could be useful – specifically comparison of performance between linear probing and full finetuning, which currently are in two different tables, and slightly convenient to compare, - but understandable given the page limit. Additionally, exploration of why MedSegLIP performs well as a pretraining baseline could be useful.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
Any exploration on why MedSegLIP provides a strong pretraining baseline compared to other FMs – dataset size? Affinity? Training strategy?
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Given the statistical significance provided by the large national scale dataset, experimental setup, reporting of metrics, and generated findings, the paper provides – I think – substantial value to the community.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This paper introduces OphFM-bench, a large-scale benchmark for evaluating foundation models (FMs) on early multi-retinopathy screening. The dataset covers 503,689 adults from 1,010 hospitals across 27 Chinese provinces (605,224 images), with expert-graded labels for DR, AMD, PM, and RVO. The benchmark evaluates 11 foundation models across three FM categories (retina-specific, medical-general, domain-agnostic) under two adaptation regimes (linear probing and full fine-tuning), along three evaluation axes: (1) screening efficiency at clinically relevant operating points (S@95R, S@99R, R@95S); (2) spatial generalization under North–South geographic shift; and (3) fairness auditing across age, sex, and urban–rural residency subgroups. The paper reveals that AUC is a deceptive proxy for screening effectiveness, that geographic shift disproportionately degrades tail metrics, and that fine-tuning can improve separability while worsening equalized odds.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.Dataset scale and real-world fidelity: The dataset (500K+ patients, 1,010 hospitals, 27 provinces) is exceptional in scale and diversity. Importantly, no additional filtering was applied, preserving real-world complexity including class imbalance, label noise, and device heterogeneity. 2.Clinically grounded evaluation: Moving beyond AUC to S@95R, S@99R, and R@95S aligns evaluation with actual screening workflows and regulatory requirements. This is a significant methodological improvement over standard benchmarks. 3.Novel insights with practical implications: The identification of the AUC-vs-screening-efficiency gap, the geographic tail-metric degradation, and the fine-tuning fairness paradox are non-obvious, actionable findings that are well-substantiated. 4.Fairness analysis: The multi-axis fairness audit (age, sex, residency) is rigorous and reveals age as the primary bias axis — a finding with direct clinical relevance. 5.Broad foundation model coverage: Evaluating 11 models across 3 pretraining paradigms gives comprehensive insight into the FM landscape for ophthalmology.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.No new methodology: The paper is purely an evaluation study. While the dataset and findings are valuable, there are no algorithmic contributions. For MICCAI, this may limit its appeal relative to methodological papers. 2.Dataset not publicly released: The OphFM-bench dataset (503K patients) is described but no release plan or timeline is mentioned. Without data release, the benchmark’s utility for community-level comparison is significantly reduced. 3.Label quality: Labels are derived from telemedicine grading, which introduces inter-rater variability. The paper acknowledges label uncertainty but does not quantify it or analyze its impact on benchmark conclusions. 4.Geographic shift is North–South only: The chosen stress test (South→North) is well-motivated but limited to one axis of shift. Other confounders (device type, hospital tier) are not isolated. 5.Missing questions 15–17 in the review template (visible only to meta-reviewers) may contain confidence/expertise self-rating not evaluated here.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
The dataset includes 503,689 anonymized adults from routine health checks across China. While the data is described as anonymized, there is no explicit statement of ethics committee approval, IRB waiver, or patient consent mechanism in the paper. Given the scale (half a million individuals), this should be explicitly addressed.
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
OphFM-bench is a timely and important contribution that addresses a real gap in the evaluation of foundation models for ophthalmic screening. The scale of the dataset and the rigor of the evaluation protocol (especially the clinically-relevant operating points and fairness audits) are strong points. The main limitation is the purely evaluation-focused scope and the absence of a public data release plan. The findings about AUC vs. tail specificity and the fine-tuning fairness paradox are genuinely novel and impactful.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
OphFM-bench makes a substantial contribution to the ophthalmology AI community through its unprecedented dataset scale, its clinically-grounded evaluation protocol, and its actionable insights about foundation model behavior under real-world deployment conditions. The paper is well-written and the findings are novel and impactful. The absence of algorithmic novelty is expected for a benchmark paper, and the findings clearly justify acceptance. The main unresolved concern — ethics statement — should be addressed before final publication. The geographic generalization and fairness analyses in particular are exemplary and should influence how future screening benchmarks are designed.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
N/A
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
All reviewers agree that this work makes a substantial empirical contribution, given the unprecedented dataset scale, rigorous clinically aligned evaluation and insights into real-world conditions.
The is clear consensus on the real-world relevance of this work and appreciation of the exceptional dataset size, scope and complexity. The experimental evaluation is well-designed and comprehensive with multiple approaches and strategies considered, leading to significant clinically-relevant insights.
While it is disappointing that the dataset will not be made public, and there is a lack of technical novelty, this a strong benchmark study and the insights gained support acceptance.