List of Papers Browse by Subject Areas Author List
Abstract
Dermatological diagnosis requires integrating fine-grained visual perception with expert clinical knowledge. Although Multimodal Large Language Models (MLLMs) facilitate interactive medical image analysis, their application in dermatology is hindered by insufficient domain-specific grounding and hallucinations. To address these issues, we propose DermAgent, a collaborative multi-tool agent that orchestrates seven specialized vision and language modules within a Plan-Execute-Reflect framework. DermAgent delivers stepwise, traceable diagnostic reasoning through three core components. First, it employs complementary visual perception tools for comprehensive morphological description, dermoscopic concept annotation, and disease diagnosis. Second, to overcome the lack of domain prior, a dual-modality retrieval module anchors every prediction in external evidence by cross-referencing 413,210 diagnosed image cases and 3,199 clinical guideline chunks. To further mitigate hallucinations, a deterministic critic module conducts strict post-hoc auditing via confidence, coverage, and conflict gates, automatically detecting inter-source disagreements to trigger targeted self-correction. Extensive experiments on five dermatology benchmarks demonstrate that DermAgent consistently outperforms state-of-the-art MLLMs and medical agent baselines across zero-shot fine-grained disease diagnosis, concept annotation, and clinical captioning tasks, exceeding GPT-4o by 17.6% in skin disease diagnostic accuracy and 3.15% in captioning ROUGE-L.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0286_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/YizeezLiu/DermAgent
Link to the Dataset(s)
https://huggingface.co/datasets/redlessone/Derm1M
BibTex
@InProceedings{LiuYiz_DermAgent_MICCAI2026,
author = { Liu, Yize AND Yan, Siyuan AND Hu, Ming AND Ju, Lie AND Li, Xieji AND Tang, Feilong AND Feng, Wei AND Ge, Zongyuan},
title = { { DermAgent: A Multi-Tool Agentic System for Evidence-Grounded and Traceable Reasoning in Dermatology } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16879},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes DermAgent, a collaborative multi tool agent for comprehensive skin disease image analysis. The main contribution is: (1)The paper proposes a Plan-Execute-Reflect framework, which integrates seven professional visual and language tools to provide a step-by-step and verifiable reasoning process. (2)The paper designs a dual-modality retrieval module combining Case RAG over 413,210 diagnosed cases with Guideline RAG over 3,199 clinical guideline chunks, grounding predictions in traceable external knowledge. (3)The paper introduces the a deterministic Critic module with confidence, which establishes three threshold levels: confidence, coverage, and conflict detection. It can automatically identify conflicts between different tools and trigger targeted self correction mechanisms. (4)The paper was validated on five dermatology benchmarks, and the results showed that DermAgent outperformed GPT-4o by 17.6% in skin disease diagnostic accuracy and 3.15% in captioning ROUGE-L.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
(1)The paper organically combines visual perception, external knowledge retrieval, and reflective verification to solve the key bottleneck of the current MLLM in the field of dermatology, which lacks domain specific foundation and is prone to hallucinations. (2)By using an explicit Evidence Chain and a Critic driven self correction mechanism, a step-by-step reasoning path is provided, with intermediate output results available for physicians to review, meeting the explanatory requirements for medical AI decision-making. (3)The paper conducted experiments on three different tasks: classification, annotation, and description generation, as well as ablation experiments, clearly demonstrating the contribution of each component to the final performance.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
(1)Due to the use of a multi round iterative “plan execute reflect” loop and the involvement of multiple large visual language models (such as Qwen3-VL-8B) being called in parallel, the paper did not report inference latency and computational resource consumption, which may limit its deployment feasibility in real-time clinical scenarios. (2)The confidence thresholds in the Critic module (such as PanDerm’s 90% cosine similarity and RAG’s 80%) are empirically set and lack adaptability to highly variable clinical images captured by different types of lesions, lighting conditions, skin tones, or devices. The generalization ability and robustness of these static thresholds may be insufficient. (3)The performance limit of DermAgent is largely limited by its seven integrated professional tools. If underlying visual models such as PanDerm or MAKE have systematic biases on certain rare skin diseases, even with reflective mechanisms, the stability of the agent will still be affected. (4)The experiments in the paper are all based on static datasets, but clinical applications involve more complex doctor-patient conversations and dynamic queries. Therefore, the interactive performance of DermAgent in real medical processes still needs further verification.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper has made substantial progress in addressing the two core pain points of “illusion” and “lack of evidence chain” in dermatology AI, but the methodology of the paper is mainly based on engineering heuristics rather than a strict theoretical framework. There are still some potential limitations in the theoretical architecture and design logic of DermAgent. The comparative experiments in the paper cover a wide range of benchmark tests, but there is still room for improvement in the depth of ablation experiments. (1)The Plan-Execute-Reflect loop proposed in the paper sets the maximum retry count kmax=2, but it has not been proven that all Critic gating conditions must be satisfied within a finite number of steps, and lacks a formal definition of the “termination state”, which cannot guarantee the logical completeness of the output results. (2)The three gate mechanism of the Critic module relies on hard thresholds, and the statistical significance of these thresholds has not been verified through hypothesis testing or confidence interval theory. The impact of prior probability differences in different disease categories on threshold sensitivity has not been considered, and there is a lack of theoretical analysis on threshold robustness. (3)The process of dynamically selecting tool combinations for LLM controllers lacks optimality theory guarantees, and it has not been proven that the selected tool combinations have optimality or approximate optimality in terms of information gain or diagnostic accuracy. The assumption of conditional independence between tools has not been established, which may lead to redundant calls, such as the failure to quantify the functional overlap between DermoGPT and Qwen3-VL from an information theory perspective. (4)As a multi module cascade system, the paper did not establish a theoretical model for error propagation, did not analyze the cascading impact of upstream tools (such as the incorrect annotation of the MAKE concept) on downstream diagnosis, and lacked quantitative limits for error accumulation in the “evidence chain”, such as the relationship between total error and errors in each module. The Critic module itself serves as an error detector, and its false negative/false positive rate is not theoretically defined. (5)There is a risk of circular argumentation in the definition of “illusion”. In the theoretical model, if PanDerm and Case RAG both provide the same but incorrect predictions, the Critic module will determine it as “conflict free” and directly pass the validation. This means that the system’s reflective mechanism may fail when dealing with “collective errors” or diagnostic Mimics with highly similar appearances. This mechanism is more inclined to detect “uncertainty in the calculation process” rather than “objective truth” in the medical sense. (6)Lack of semantic alignment analysis for knowledge retrieval, Case RAG relies on 512 dimensional embedding vectors for cosine similarity search. In dermatology, statistical similarity of images (visual underlying features) is not equivalent to diagnostic consistency in medicine. The paper did not delve into how to ensure that the retrieved ‘similar cases’ have reference value in medical logic. (7)The experiment lacks a comparison of inference efficiency (cost/latency), and DermAgent adopts a multi round iterative Plan-Execute-Reflect loop. Compared with the MLLM baseline of single inference, its response delay and API call cost are inevitably higher. The paper does not provide comparative data on average diagnostic time or computational resource consumption, which is crucial for real-time clinical applications. Suggest adding a comparison of the average token consumption and inference time between DermAgent and GPT-4o when processing a single image. (8)The ablation experiment only demonstrated the overall difference between the presence and absence of Critic, but did not separately validate the effectiveness of each of the three major gating mechanisms. For example, how much will the system performance decrease if only conflict detection is retained? Suggest splitting the three gates of the Critic module for separate testing to verify whether conflict detection is the key to resolving illusions. (9)In addition to showcasing successful self correction cases, it is recommended to analyze situations where Critic is unable to recognize errors or where negative corrections such as “getting more and more wrong” occur.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
It proposes a collaborative multi-tool agent that follows a plan-execute-reflect loop to iteratively gather and refine evidence, and generates final outputs by integrating information from multiple sources. The framework is further adapted to the dermatology domain. At the same time, the paper evaluates the proposed method across a range of tasks and datasets.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This work shows some novelty in applying LLMs. It combines multiple tools into an iterative workflow. The critic-based feedback and checking step can re-check the outputs and help reduce hallucinations in multimodal models compared with baseline methods. The application of a general multi-tool agent framework to the dermatology domain for clinical image analysis tasks has practical value. The framework is evaluated on multiple tasks and datasets, and compared with multiple baseline methods including LLMs and agents.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The paper treats Case RAG as an important contribution and says it helps improve performance a lot. But since it is built on Derm1M, and Derm1M comes from web data and several public datasets, it would be good to explain how overlap with test data is avoided. It is not clear whether test images could be inside the retrieval database. So this point needs more clarification. The thresholds in the critic module (like 90% for PanDerm and 80% for RAG) are set in an empirical way. Since the critic plays an important role in the whole pipeline, it would be better to explain how these values are chosen, or provide a simple sensitivity study.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This work shows some nice application ideas. It combines multiple tools into an iterative workflow, and uses a critic step to re-check the results, which helps reduce hallucination. And the evaluation is comprehensive, covering different tasks and datasets. However, there are still some concerns. It is not clear whether the Case RAG overlaps with the test data, which may affect the reliability of the results. Also, the thresholds in the critic are set in an empirical way and lack clear explanation.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This paper proposes DermAgent, a multi-tool agentic framework for dermatological image analysis that integrates seven specialized vision-language modules within a Plan-Execute-Reflect loop. The system combines visual perception, dual-modality retrieval (case-based and guideline-based), and a deterministic critic module to produce traceable, evidence-grounded diagnostic reasoning. Extensive experiments across five dermatology benchmarks demonstrate improved performance over existing MLLMs and agent-based approaches.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Well-motivated problem: The paper addresses important limitations of current MLLMs in dermatology, particularly hallucination and lack of grounding. Novel system integration: The combination of multi-tool orchestration, dual-modality retrieval, and a critic-driven self-reflection mechanism is well-designed and aligned with clinical reasoning workflows. Interpretability: The explicit evidence chain and stepwise reasoning improve transparency compared to black-box models. Strong empirical evaluation: The method is evaluated on multiple benchmarks covering classification, concept annotation, and captioning, showing consistent improvements over strong baselines. Ablation study: The contribution of individual components (e.g. critic, retrieval modules) is clearly analyzed and supports the design choices.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Limited methodological novelty: While the system integration is solid, many components (RAG, agent loops, tool orchestration) are adapted from existing paradigms, the novelty lies mainly in their combination for dermatology. Evaluation scope: Experiments are conducted on relatively small benchmark subsets (e.g.100 samples in some tasks), which may limit conclusions about real-world clinical robustness. Clinical validation: The paper lacks validation in real clinical settings or with clinician-in-the-loop evaluation, which is important for translation. Complexity and efficiency: The multi-tool pipeline may introduce significant computational overhead, but runtime or efficiency analysis is not discussed. Reproducibility details: Although code is mentioned, more implementation details (e.g. hyperparameters, tool orchestration specifics) would strengthen reproducibility.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
The paper would benefit from a clearer comparison with recent medical agent frameworks to better highlight novelty. Discussion on computational cost and latency would improve practical relevance. Future work could include prospective clinical validation or user studies with dermatologists.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper presents a well-engineered and practically relevant system that integrates multiple components into a coherent and interpretable framework for dermatological analysis. The empirical improvements across tasks and the inclusion of retrieval and self-reflection mechanisms are compelling strengths. However, the novelty is primarily at the system level, and limitations in evaluation scale and clinical validation reduce its overall impact. Overall, the work is above the acceptance threshold but would benefit from stronger validation and clearer differentiation from prior agent-based systems.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We sincerely thank all reviewers for their constructive feedback and positive evaluation. We are encouraged that the reviewers recognized DermAgent’s clinical motivation, evidence-grounded design, and consistent improvements over strong MLLM and medical-agent baselines. We address the key concerns below.
Q1.Case RAG leakage (R2). DermAgent prevents query-image leakage during Case RAG. Each input image is assigned an anonymized case/image ID, which is propagated to the retrieval module. Any retrieved candidate sharing the same ID is excluded before ranking and evidence aggregation; therefore, a test image cannot be used as supporting evidence for its own prediction. We will clarify this in the final manuscript.
Q2.Critic thresholds (R1-R3). The Critic thresholds are not diagnostic cutoffs, but conservative evidence-quality gates. They are used to trigger additional evidence refinement when PanDerm is over-confident or when retrieved cases are weakly matched. This helps the agent cross-check visual predictions against morphology, similar cases, and guideline-based differential diagnoses before final reasoning.
Q3.Dependence on visual representations, collective errors, and semantic validity of retrieved cases (R1). We thank the reviewer for this insightful comment. DermAgent is designed to avoid relying solely on representation-based visual priors. In Case RAG, each retrieved case contains not only a diagnostic label, but also clinical descriptions and hierarchical diagnostic categories, enabling the orchestrator to compare morphology and disease semantics rather than treating embedding similarity as a direct diagnostic shortcut. VLM-based and MAKE-derived morphological evidence is further cross-checked with guideline-supported differentials before final reasoning. Moreover, the Critic enforces this process: the Coverage Check prevents premature decisions dominated by visual-representation signals, while the Confidence Check mitigates over-confident predictions from tools such as PanDerm. Our ablation and qualitative analysis further supports this design, showing that each evidence source contributes to the final performance.
Q4.Novelty relative to recent medical agents (R3). DermAgent is purpose-built for dermatology rather than a generic medical-agent framework, which is essential considering the long-tailed taxonomy and morphological mimics. Unlike MDAgents [15], which studies LLM-collaboration patterns for general medical decision-making, DermAgent orchestrates dermatology-specialized vision tools within a unified Plan–Execute–Reflect loop. Unlike MedAgent-Pro’s general multimodal workflow [26], it grounds reasoning through dual retrieval with diagnosed skin cases and clinical guideline chunks. Unlike WSI-Agents [19], which targets pathology whole-slide images, DermAgent is tailored to clinical/dermoscopic skin images and introduces deterministic confidence, coverage, and conflict gates designed around dermatology-specific failure modes. Together, these components provide a traceable, evidence-grounded dermatology agent rather than a generic medical-agent framework.
Q5.Static benchmarks and clinical workflow (R1&R3). Our current evaluation follows established dermatology image-analysis benchmarks to ensure reproducibility and comparability. We agree that real clinical deployment requires interactive, multi-turn, and prospective validation. We will acknowledge this as an important direction for future work.
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper addresses an important and timely problem in dermatology, namely improving the reliability and grounding of multimodal large language models for clinical image understanding. Reviewers agreed that the work is well motivated and that the proposed framework presents a meaningful integration of visual perception, knowledge retrieval, tool orchestration, and critic-driven self-reflection. The explicit evidence chain and intermediate reasoning steps were viewed as particularly valuable for improving transparency and reducing hallucination, which is important in medical AI settings.
A major strength of the submission is its broad empirical evaluation across multiple dermatology tasks, including classification, concept annotation, and description generation, together with ablation studies that support the contribution of individual components. Reviewers found the framework practically relevant and appreciated its consistent improvements over strong baselines.
The main concerns relate to deployment feasibility and methodological clarity rather than to the overall validity of the approach. In particular, reviewers asked for better discussion of inference cost and efficiency, clearer justification or sensitivity analysis for the empirically chosen critic thresholds, and clarification regarding possible overlap between the retrieval database and evaluation data. Some reviewers also noted that the main novelty lies more in system integration than in fundamentally new algorithmic components, and that clinical validation remains limited to static benchmark settings.
Overall, the reviewers were positive about the paper’s significance, technical design, and empirical results. While some clarifications would strengthen the final version, the contribution is considered solid and relevant for the MICCAI community.
