List of Papers Browse by Subject Areas Author List
Abstract
Multimodal Large Language Models (MLLMs) demonstrate immense potential in medicine, yet existing ophthalmology benchmarks face significant limitations. They often neglect text-only clinical scenarios, restrict question formats, and lack structured evaluation systems that mirror real-world diagnostic workflows. To bridge these gaps, we introduce OphthFlowBench, the first comprehensive ophthalmology benchmark that integrates both multimodal VQA and text-only clinical QA pairs across open-ended and closed-form formats. Our benchmark features a pioneering workflow-aligned evaluation system that decomposes the clinical process into four sequential stages: visual perception, clinical interpretation, disease diagnosis, and clinical management, enabling granular, stage-by-stage assessment. Furthermore, to address the scarcity of domain-specific models, we developed Oph-RL, a specialized ophthalmic MLLM based on the Qwen3-VL-8B-Instruct architecture. Oph-RL is trained using a novel two-phase strategy comprising Supervised Fine-Tuning (SFT) and difficulty-aware Group Relative Policy Optimization (GRPO). Extensive evaluations on OphthFlowBench and the OphthalWeChat benchmark reveal that Oph-RL achieves exceptional accuracy and robust clinical utility. It markedly outperforms existing proprietary, general and medical MLLMs across multiple modalities, subspecialties, and languages, thereby paving the way for reliable ophthalmic decision-support systems. The benchmark and model are publicly available at https://github.com/HapisQJ/OphthFlowBench.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0815_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/HapisQJ/OphthFlowBench
Link to the Dataset(s)
https://github.com/HapisQJ/OphthFlowBench
BibTex
@InProceedings{ZheQia_OphthFlowBench_MICCAI2026,
author = { Zheng, Qiaojian AND Luo, Xiaoling AND Ding, Meidan AND Zheng, Ruli AND Dou, Xiaoyan AND Wen, Yingying AND Liu, Chengliang AND Shen, Linlin},
title = { { OphthFlowBench: A Unified Ophthalmology Multimodal Benchmark with Workflow-Aligned Evaluation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16895},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper highlights that the existing ophthalmology benchmarks do not cover text-only clinical scenarios and are also limited in terms of question formats. To overcome these limitations, the authors proposed OphthFlowBench, a comprehensive ophthalmology benchmark that includes VQAs as well as text-only questions. They also proposed Oph-RL, a domain-specific MLLM based on Qwen3-VL-8B-Instruct, and a two-phase training strategy: Supervised Fine-Tuning (SFT) followed by difficulty-aware Group Relative Pol- icy Optimization (GRPO). Additionally, the paper presents detailed results highlighting the significance of the proposed benchmark and model.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Comprehensive benchmark, OphthFlowBench, that combines VQA and text-only clinical QA pairs for both open-ended and closed-form formats
- A structured assessment system that divides the clinical process into four stages: visual perception, clinical interpretation, disease diagnosis, and clinical management
- A high data purity achieved through a two-stage extraction pipeline involving an MLLM and a human-in-the-loop verification step by a panel of professional ophthalmologists
- Developed Oph-RL, based on the Qwen3-VL-8B-Instruct architecture, which achieved state-of-the-art performance
- Supervised Fine-Tuning (SFT) followed by a difficulty-aware Group Relative Policy Optimization (GRPO) training strategy
- Detailed evaluation
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- While Oph-RL offers better performance in diagnosis and perception, it underperforms compared to proprietary MLLMs in the management stage
- The creation of OphthFlowBench relied heavily on models such as GPT-4o-mini and GPT-5 for initial processing. The impact of these non-medical-specific models on the quality of the benchmark should be clarified.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The authors proposed a comprehensive Benchmark, OphthFlowBench, for ophthalmology. Additionally they proposed a structured assessment method that divides the complete clinical process into four stages: visual perception, clinical interpretation, disease diagnosis, and clinical management. Moreover, the paper presents Oph-RL model that achieved state-of-the-art performance. Additionally, they presented detailed evaluation considering multiple SoA MLLMs.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors have addressed most of my comments to a satisfactory level. Additionally, the contributions of the paper are already strong. The authors have proposed a comprehensive Benchmark, OphthFlowBench, for ophthalmology. Additionally, they have proposed a Oph-RL model that achieved state-of-the-art performance compared to other comparable models.
Review #2
- Please describe the contribution of the paper
This paper introduces OphthFlowBench, an ophthalmology benchmark integrating both multimodal VQA and text-only clinical QA pairs across open-ended and closed-form formats. The benchmark is aligned with real-world clinical workflows, decomposing into four sequential stages: visual perception, clinical interpretation, disease diagnosis, and clinical management. Furthermore, the authors develop Oph-RL, a specialized ophthalmic multimodal large language model trained via SFT and GRPO. Extensive evaluation demonstrates that Oph-RL outperforms existing proprietary, general, and medical MLLMs.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.Workflow-Aligned Benchmark: OphthFlowBench integrates multimodal VQA and text-only QA, covering both open-ended and closed-form tasks, and aligning with real clinical workflows for stage-wise assessment. 2.Ophthalmology-Specific Model: The paper develops Oph-RL, a tailored ophthalmic MLLM with a two-phase training strategy, which improves domain-specific reasoning and addresses challenges in multimodal clinical understanding. 3.Extensive Evaluation Across Multiple Dimensions: The model is rigorously benchmarked against 23 representative MLLMs, including proprietary, general, and medical models. Oph-RL achieves superior performance across modalities, subspecialties, and languages.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Lack of Human Verification in Sec 2.1 (Refinement): The refinement phase appears to rely solely on GPT-5 without any human verification. Compared to the “Retrieval and Verification” and “Classification” steps, the “Refinement” phase is more challenging and critical. It requires not only type-level categorization but also a deep understanding of semantic meaning, clinical context, and medical knowledge. Relying entirely on MLLMs may introduce clinically inappropriate revisions, hallucination, or artifacts, compromising the accuracy and reliability of the refined QA pairs. 2.Unclear Handling of Ambiguous Cases in Sec 2.1 (Refinement): It is unclear how the authors handle ambiguous cases. Specifically, what procedures are used to identify and correct samples that violate the proposed three core principles? Since this step directly determines the quality of both training and test data, as well as the validity of the evaluation ground truth, the lack of human verification and the absence of a clear ambiguity-resolution protocol constitute significant limitations. 3.Unclear Definition of Clinical Stages: The paper does not clearly explain how the four stages are defined and separated. Clearer criteria are needed to ensure the rigor and reproducibility of the stage-wise classification. 4.Limited Workflow-Level Analysis: Although the paper highlights a workflow-aligned benchmark, the presentation of workflow-related results is limited. The paper lacks sufficient stage-wise comparisons across models, analysis of the contribution of each stage, and discussion of how different stages are connected into a coherent end-to-end clinical workflow. As a result, the workflow aspect is not convincingly demonstrated. 5.Limited Real-World Validation: The evaluation is based on public datasets rather than real-world clinical data, which raises concerns about whether the proposed benchmark and model truly reflect the complexity and distribution shifts encountered in actual medical practice. 6.Incomplete Comparison with Ophthalmology-Specific MLLMs: A more comprehensive comparison against existing ophthalmic MLLMs (e.g., VisionUnite, FundusExpert) would better demonstrate the advantage of Oph-RL. 7.Baseline Selection on OphthaWeChat Benchmark: The overlap between the baselines evaluated on OphthaWeChat and those used on OphthFlowBench is small. This makes it difficult to directly compare model performance across the two benchmarks. OphthaWeChat would benefit from including stronger and more competitive baselines, such as Lingshu, InternVL3.5, and Qwen3-VL. 8.Limited Performance on Common Settings: Oph-RL underperforms non-ophthalmic models on common modality combinations (e.g., CFP+FFA, CFP+FAF) and common diseases such as glaucoma on OphthWeChat. These results need further explanation, as it is unclear whether it reflects limited generalization or other factors.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
1.Lack of Human Verification and Unclear Handling of Ambiguous Cases in Sec 2.1 (Refinement) 2.Incomplete Comparison with Ophthalmology-Specific MLLMs 3.Limited Workflow-Level Analysis
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
Thanks to the authors for addressing several of my concerns. However, some issues remain and should be further clarified in the final version.
The interaction among different stages within the clinical workflow could be more clearly demonstrated. While the authors state that perception or interpretation errors may propagate to diagnosis and management, Table 1 mainly reports aggregate performance for each stage and does not provide clear evidence of error propagation.
The rebuttal regarding OphthaWeChat does not fully address the limited overlap and comparability of baselines across benchmarks, making it difficult to directly assess the generalizability of the proposed model.
Review #3
- Please describe the contribution of the paper
The paper introduces OphthFlowBench, a workflow-aligned ophthalmology benchmark covering multimodal VQA and text-only QA in both open-ended and closed-form settings. The benchmark decomposes clinical reasoning into four stages: visual perception, clinical interpretation, diagnosis, and management.
In addition, the authors propose Oph-RL, an ophthalmology-specialized MLLM built on Qwen3-VL-8B-Instruct, trained using a two-stage pipeline. The goal is to move from task-specific evaluation toward clinically grounded, end-to-end reasoning.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.Clinically aligned evaluation: Decomposing clinical reasoning into four sequential stages is an intuitive, practical way to diagnose where a model fails in a realistic diagnostic pipeline. 2.Smart RL sampling: Implementing a difficulty-aware data selection strategy for GRPO is a clever way to ensure the model receives stable, effective reinforcement learning signals, avoiding gradient noise from overly simple or impossible queries. 3.Comprehensive benchmarking: The benchmark includes multimodal and text-only tasks, open-ended and closed-form settings, and evaluates a large set of models.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Potential data leakage: The training data and the evaluation benchmark are derived from overlapping public sources. There is no description of a hash-based or embedding-based deduplication protocol across the train/test splits. Without strict decontamination, the performance results are questionable. 2.Limited evaluation for open-ended tasks: Relying solely on ROUGE-L for clinical reasoning tasks is insufficient. ROUGE-L measures n-gram overlap, not clinical semantic correctness or safety. The paper needs an LLM-as-judge safety rubric, entailment metrics, or clinician evaluation to prove the generated reasoning is actually valid. 3.Insufficient ablation for GRPO design: The contribution of the difficulty-aware sampling strategy is not isolated; comparisons to standard GRPO or alternative curricula are missing. 4.. Incomplete baseline coverage. The comparison lacks several competitive medical MLLMs such as GMAI-VL-R1, RadFM, UniMedVL and BiomedGPT, making it difficult to assess whether the proposed method advances the state of the art.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper addresses an important problem and proposes a reasonable benchmark design with a clear clinical motivation. The staged evaluation and broad experimental coverage are strengths, and the reported improvements are encouraging.
Overall, while the direction is promising, the current evaluation does not yet meet the level of rigor expected. However, with these additions and clarifications, a higher score could be warranted if concerns are fully addressed in the rebuttal.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
It’s an interesting work and the authors have shown some efforts in their work for this area of medical research. In my opinion, overview should be accepted after rebuttal.
Author Feedback
We appreciate valuable comments from reviewers and will consider them carefully in the final manuscript.
Meta concerns on verification and ambiguity handling (R1Q2, R2Q1Q2), leakage (R3Q1), workflow analysis (R2Q4), open-ended evaluation (R3Q2), baselines (R2Q6, R3Q4), and GRPO ablation (R3Q3) are addressed below.
R1Q1: Lower management-stage performance As an 8B model, Oph-RL is much smaller than proprietary MLLMs. In addition, we made relevant analysis in Sec. 3.2. R1Q2: Benchmark quality GPT-4o-mini/GPT-5 were used only for preprocessing and standardization. As stated in Sec. 2.1, all retained samples were further verified by four ophthalmologists.
R2Q1Q2: Handling in refinement GPT-5 refinement was rule-constrained rather than unrestricted rewriting, following the protocol in “Adaptive Reinforcement for Open-ended Medical Reasoning via Semantic-Guided Reward Collapse Mitigation (CVPR 2026-Findings)”. It only ensured questions were complete, unambiguous, and semantically sufficient, while keeping their form, scope, detail level, and structure aligned with the answers. Ambiguous cases were corrected under these rules, unresolved cases were discarded, and ophthalmologists verified both pre- and post-refinement results.
R2Q3: Clinical-stage definitions Stages were defined by the dominant cognitive operation: visual perception identifies visible findings; clinical interpretation explains their meaning; disease diagnosis determines disease category; clinical management selects diagnostic or treatment actions. All labels were assigned accordingly and verified by ophthalmologists.
R2Q4: Workflow-level analysis Table 1 reports stage-wise results. It reveals clear stage-specific bottlenecks: strong perception/diagnosis does not always imply strong management. The workflow design also captures cascading dependency, where errors in perception or interpretation may propagate to diagnosis and management.
R2Q5: Real-world validation Multi-center real-world validation is important future work but requires substantial expert effort across workflow stages.
R2Q6: Ophthalmology-specific MLLMs We thank the reviewers for the suggestion. We have included representative ophthalmology-specific MLLMs such as Med-R1-Fundus, and evaluated VisionUnite and FundusExpert, as suggested by the reviewers (The results will be shown in the final version). The result is consistent with our existing findings: current ophthalmology-specific MLLMs still show only moderate performance on workflow-level tasks.
R2Q7: OphthaWeChat baselines OphthaWeChat is supplementary, not our main evidence. Since it mainly contains classification-oriented questions, it tests generalization to simpler ophthalmic VQA settings.
R2Q8: Common-setting performance Glaucoma is not underperformance: Oph-RL ranks first, 0.549 vs. 0.547 second-best. Remaining common-modality gaps mainly reflect scale/data coverage, as Oph-RL is 8B while strong baselines such as Qwen2.5-VL-72B are much larger.
R3Q1: Data leakage SHA-256 hashing found no duplicate sample content.
R3Q2: Open-ended evaluation ROUGE-L provides a reproducible metric but is not the sole evidence. As shown in Sec. 3.3, OphthaWeChat also uses DeepSeek-V3 as an LLM judge for factual correctness, where Oph-RL achieves the best Open-ended-CN/EN results (language subsets), supporting its open-ended ability.
R3Q3: GRPO ablation The manuscript compares the SFT with Oph-RL, showing GRPO gains. Our design follows the standard GRPO paradigm using answer and format rewards. Difficulty-aware selection does not change the RL objective. It only filters medium-difficulty samples, since extremely easy/hard cases provide weak learning signals, as also motivated by OralGPT-Omni (CVPR 2026).
R3Q4: Baselines We evaluated 23 representative MLLMs, covering proprietary, open-source, medical-specific, and ophthalmology-related models such as Lingshu and Med-R1-OCT. More baselines will be considered in future work.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The reviewers see the paper as a timely contribution. The main concerns are about rigor rather than direction: reviewers ask for clearer human verification and ambiguity handling, stronger decontamination against data leakage, better workflow-stage analysis, stronger open-ended evaluation beyond ROUGE-L, fuller ophthalmology/medical MLLM baselines, and more ablation of the GRPO design.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The main concerns were about how reliable the benchmark is. In particular, the reviewers asked for clearer details on human checking, how ambiguous cases were handled, whether there was any overlap between training and test data, and whether open-ended answers were evaluated well enough. Some reviewers also felt that more ophthalmology-specific baselines and better workflow-level analysis would strengthen the paper. After rebuttal, all reviewers moved to Accept. Some issues remain, but they can be addressed in the final version.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
This paper introduces OphthFlowBench, a comprehensive ophthalmology benchmark that structures clinical assessment into four stages (visual perception, clinical interpretation, disease diagnosis, and clinical management), along with the Oph-RL model that achieves state-of-the-art performance across multiple SoA MLLMs. Reviewers acknowledged the strong contributions and that the authors addressed most concerns during rebuttal, though some noted remaining issues. Although the initial reviewer scores diverged, all reviewers now agree on acceptance after rebuttal, and I recommend acceptance.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Reviewers agreed that most raised questions were addressed in the rebuttal and consistently recommended acceptance. They acknowledged the contributions of the work, including a very relevant benchmark for ophthalmology and strong empirical results. R2 suggested further refinements for the camera-ready version.
