List of Papers Browse by Subject Areas Author List
Abstract
Multimodal Large Language Models (MLLMs) are increasingly used for diagnostic support, particularly in settings where patients submit images for evaluation. In a domain such as dermatology, where diagnosis hinges on subtle morphological cues in photographs, generating plausible answers without meaningfully depending on the image presents a clinical safety risk. To analyze when images actually matter, we introduce DermDetail, a DermaVQA-based benchmark that systematically escalates the amount of prior-diagnostic information in the text prompt. We show that with progressive enrichment of textual context, image-token attention drops by nearly 2x. A length-matched control with no diagnostic content reverses this effect, confirming the drop is driven by semantic content rather than prompt length. Current MLLMs treat the image as a fallback, not a foundation. To mitigate this inherent textual bias, we introduce dermatologist-designed structured vignettes which encode key morphological and contextual features as structured text. We benchmark eight open- and closed-source MLLMs under matched modality conditions. By routing visual information through the language channel, we match or exceed full multimodal performance by up to 1.25x in Hierarchical F1 score (HF1). Furthermore, meaningful assessment requires evaluation beyond surface-form metrics that fail to capture clinically relevant near-misses or distinguish affirmed from negated diagnoses. We present ICDLens, an interpretable ICD-11-grounded evaluation toolkit with assertion-aware filtering and hierarchy-aware F1 scoring that credits diagnostically proximate errors while penalizing unrelated predictions. Together, this work diagnoses modality collapse, proposes structured mitigation, and introduces clinically faithful evaluation which advances safer deployment of MLLMs in healthcare.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/5540_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/LaraHossam/Better-Said-Than-Seen
Link to the Dataset(s)
DermaVQA dataset: https://osf.io/72rp3/
BibTex
@InProceedings{HasLar_Better_MICCAI2026,
author = { Hassan, Lara AND Mohamed, Alaa AND Mahmoud, Abdulrahman},
title = { { Better Said Than Seen: Exposing and Mitigating Modality Collapse with ICD-11-Grounded Evaluation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16895},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper investigates modality collapse in Multimodal Large Language Models (MLLMs) for teledermatology. The authors introduce DermDetail, a benchmark derived from DermaVQA that progressively enriches textual context to measure how image-token attention shifts as prior diagnostic information increases. They show that image attention drops when textual priors strengthen. To mitigate this, the authors propose dermatologist-designed structured vignettes that encode visual features as text, and introduce ICDLens, an evaluation toolkit.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The observation of modality collapse is clinically relevant. The paper provides a systematic demonstration how MLLMs treat visual and textual information in dermatology, which is an important. 2.The experimental design with seven prompt levels (P1-P7) is well-structured. The length-matched control (P7) effectively isolates the effect of semantic content from prompt length, strengthening the claim that the collapse is content-driven. 3.The ICDLens evaluation framework, which uses ICD-11 hierarchy-aware scoring with assertion-aware filtering, addresses a real gap in existing evaluation metrics for dermatological diagnosis.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Limited novelty in core contributions: The observation that MLLMs over-rely on textual priors is not new and has been noted in prior multimodal learning literatures. The proposed mitigation essentially converts visual information into text, which avoids the fundamental problem rather than solving it. The ICDLens toolkit is primarily an NLP pipeline with no image computing component.
2.Scope mismatch with MICCAI: The paper reads more as a technical report on LLM evaluation and NLP tooling rather than a medical image computing contribution. The core methodology does not involve any novel techniques typically expected at MICCAI.
3.The DermDetail benchmark is derived from only 228 cases, which is a relatively small sample. Generalizability to other datasets or clinical domains remains undemonstrated.
4.The ICDLens pipeline has a strong dependency on GPT-4o for the final entity selection step, introducing reproducibility concerns.
5.Structured vignettes are populated via GPT-4o from original images, raising a circularity concern: the vignette quality depends on another MLLM correctly interpreting the image in the first place.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(2) Reject — should be rejected, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper presents an interesting observation (modality collapse in MLLMs for dermatology) but falls short on novelty and scope for MICCAI. The main concerns are: (1) The core finding that MLLMs over-rely on text priors is not sufficiently novel. (2) The proposed mitigation (structured vignettes) removes the image entirely rather than improving image utilization, which is rather aligned with an NLP or LLM evaluation venue (e.g., ACL, EMNLP) than MICCAI. (3) ICDLens is an NLP pipeline with no image computing component. Overall, while the work is technically competent, it reads as a benchmarking/evaluation paper.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
- resource release: scoring and data
- methodology and analysis: identified the contribution of text vs image for accuracy
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
In “Better Said than Seen”, the authors use a ICD-11 grounded gold standard to evaluate the DermaVQA benchmark. They test multiple LLM’s and study the use of a expert-motivedated “vignette” based benchmarking where hierarchical questions are answered and used as additional structured input. Finally, the authors perform an analysis of question accuracy as related to text-dependence (vs image-dependence) using attention weights for quantification as well as release their resource for scoring and their structured gold standards. In general, the work is very interesting, systematic, and well written and would be a great contribution.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- adding domain expert or additional qualitative analysis would be nice
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(6) Strong Accept — must be accepted due to excellence
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Makes resource, methodology, and analysis contributions toward dermatology visual question answering. Evaluation is well done and interesting.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This paper studies modality collapse in medical MLLMs for teledermatology and argues that increasing textual prior can override visual evidence. It introduces three connected contributions: DermDetail, a prior-escalation benchmark derived from DermaVQA; structured vignette modality ablations, which test whether dermatologist-designed text surrogates can replace image input; and ICDLens, an ICD-11-grounded evaluation toolkit with assertion-aware filtering and hierarchy-aware F1 scoring for free-text dermatologic diagnoses. The paper shows that image-token attention drops substantially as diagnostic text priors strengthen, while structured vignette plus description inputs can match or exceed multimodal performance for several larger models.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.Timely and important problem. The paper targets a real safety concern for clinical MLLMs: producing plausible dermatologic diagnoses without meaningfully depending on the image. This is highly relevant for teledermatology, where morphology is central.
2.Clear conceptual framing. The three research questions are well motivated and coherent: when images matter, whether structured surrogates can mitigate the issue, and how to evaluate outputs more clinically faithfully.
3.Strong benchmark design for modality collapse. DermDetail is a neat controlled setup: fixed image, progressively enriched text priors, contradiction condition, and length-matched control. This makes the analysis more convincing than a simple prompt comparison. The prompt design shown in Figure 1 is particularly effective.
4.Useful empirical finding. Table 1 shows a substantial decrease in image attention with stronger textual priors in both MedGemma-4B-IT and Qwen3-VL-4B-Instruct, while HF1 improves, supporting the claim that accuracy gains come from text rather than image grounding.
5.Good controls. The contradictory-diagnosis condition and length-matched semantic control strengthen the core claim by showing that the effect is not only prompt length and does not fully reverse even under contradiction.
6.Interesting practical mitigation. The vignette experiments are thought-provoking. For several larger models, V+D matches or exceeds multimodal performance, suggesting a meaningful design direction for safer interfaces or structured data collection. Table 2 is informative here.
7.Evaluation contribution is valuable. ICDLens addresses a real gap in scoring clinical free-text outputs by using ICD-11 normalization, assertion awareness, and hierarchy-aware F1.This is a meaningful methodological contribution beyond the dermatology task itself. Figure 2 is clear and helpful.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Causal claims rest heavily on attention analysis. The core “modality collapse” argument is based mainly on attention allocation in two open-weight 4B models. Attention is an informative probe, but not a definitive causal explanation of reliance. The paper acknowledges this, but the strongest claims sometimes read more causally than the evidence warrants.
2.Limited interpretability analysis across model scales. The attention study is restricted to two smaller open models, while the modality ablation study includes eight models. This mismatch weakens the generalization of the collapse claim to the larger closed-source systems.
3.Mitigation may partly relocate rather than solve the problem. V+D appears strong because visual information is manually or automatically translated into language, which aligns with the strengths of text-dominant models. This is useful pragmatically, but it is less clear whether it addresses multimodal reasoning failure or simply bypasses it.
4.Potential confounding in vignette construction. The structured vignette and image description are generated using GPT-4o from the original query and image. This introduces another strong model into the pipeline and may inflate the apparent gains of V+D relative to raw multimodal input. More discussion of possible information leakage, verbosity advantages, or preprocessing asymmetry would help.
5.Evaluation stack is complex and partly LLM-dependent. ICDLens uses multiple learned/model-based components, including mention extraction, assertion refinement, reranking, and GPT-4o top-K selection. This is powerful, but raises reproducibility and stability questions. Ablations or error analysis for the coding pipeline would strengthen confidence.
6.Dataset scope is somewhat narrow. The prior-escalation benchmark uses a subset of 228 IIYI English cases, which is reasonable for analysis but limits breadth. It would help to clarify how robust the effect is across source domains, languages, and different dermatologic case mixes.
7.Clinical conclusions should be phrased carefully. The safety framing is compelling, but the experiments remain benchmark-based. The paper does not directly evaluate downstream clinician interaction, triage outcomes, or deployment workflows.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
This is a strong and timely paper. I have a few suggestions that could improve it further:
1.Please add more caution around interpreting attention as evidence of causal reliance. A complementary intervention-based analysis would strengthen the modality-collapse claim. 2.Please discuss the role of GPT-4o in creating vignettes/descriptions and in ICDLens candidate selection more explicitly, including possible advantages this introduces. 3.An ablation of ICDLens components would be valuable, for example: with/without assertion filtering, with/without reranking, and with/without hierarchy aware scoring. 4.It would help to include some manual error analysis of ICD-11 normalization failures. 5.The paper would benefit from clearer discussion of whether V+D is intended as a benchmark probe, a deployment strategy, or both. 6.If possible, include broader evidence that the prior escalation findings hold beyond the IIYI English subset.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I rate this paper as an Accept (5). The paper is novel, well motivated, and makes a meaningful contribution on three fronts: analyzing modality collapse in medical MLLMs, proposing a structured-text mitigation strategy, and introducing a clinically grounded evaluation toolkit. The benchmark design is thoughtful, the controls are good, and the empirical findings are both interesting and practically relevant. The ICD-11-grounded scoring framework is also likely to be useful beyond this specific paper.
The main limitations are that the strongest mechanistic claim relies on attention based evidence from only two open-weight models, and the evaluation/mitigation pipeline itself depends on additional LLM components. These are real concerns, but they do not outweigh the overall contribution. On balance, this is a solid and impactful paper that should be of broad interest to the MICCAI community.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
N/A
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
As acknowledged by reviewers, this manuscript presents a detailed dissection of text bias in medical VLMs in dermatology. While not the first to study text bias, this work investigates the phenomenon in the medical context and builds a principled evaluation protocol. As acknowledged by reviewers, the topic is timely and important (considering the high-stakes nature of medical decisions). The study is carefully designed and contextualized, and the empirical results are informative. Despite some limitations with dataset scale and reliance on a stronger VLM, the findings are still worthy of discussion for MICCAI. Overall, the manuscript’s merits outweigh its limited contribution in terms of methods for building new VLMs.
