List of Papers Browse by Subject Areas Author List
Abstract
Accurate emergency severity assessment fundamentally dictates patient survival and resource allocation. However, existing automated triage pipelines rely exclusively on tabular data and clinical text, neglecting the visual semantics of patient distress. A paramount barrier to anatomically informed reasoning is the absolute absence of visual modalities in public cohorts like the MIMIC-IV-ED dataset. To overcome this limitation, this study introduces MedTriage-LM, an anatomically grounded Multimodal Large Language Model (MLLM) that algorithmically injects visual priors without requiring real patient images. The architecture maps latent clinical representations onto a canonical human template via a weakly supervised Gaussian field generator, yielding continuous, interpretable Visual Phenotype Maps (VPMs). Synergizing these spatial representations with clinical embeddings via cross-attention shifts the paradigm from numerical risk scoring toward actionable three-class triage instruction prediction: Life-saving, High-Risk Assessment, and Resource Estimation. MedTriage-LM attains state-of-the-art performance among fine-tuned open-source models and approaches the capabilities of proprietary MLLMs, while additionally providing explicit, anatomically grounded interpretability. Crucially, explicit spatial grounding empowers an efficient 8B-parameter model to approach large proprietary multimodal foundation models, generating spatially grounded textual rationales that improve clinical trustworthiness.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1929_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/Leo1998-Lu/MedTriage-LM
Link to the Dataset(s)
N/A
BibTex
@InProceedings{LuZhi_MedTriageLM_MICCAI2026,
author = { Lu, Zhixiang AND Liu, Xiwei AND Wang, Jinfeng AND Zhou, Mian AND Nguyen, Anh AND Su, Jionglong AND Razzak, Imran AND Song, Sifan},
title = { { MedTriage-LM: Anatomically Grounded Visual Phenotype Synthesis for Interpretable ED Triage } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16878},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes MedTriage-LM, a multimodal triage framework that synthesizes Visual Phenotype Maps (VPMs) from tabular and textual clinical data to compensate for missing visual modalities in EHR datasets. The approach maps latent clinical representations onto a canonical anatomical template via Gaussian field synthesis and integrates these signals through a cross-attention architecture for triage prediction and rationale generation.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The concept of synthetic visual modality completion with anatomical grounding is novel and addresses a real limitation of datasets like MIMIC-IV. VPMs provide intuitive spatial representations that align with clinical reasoning. The model achieves competitive results against a wide range of baselines, including large multimodal models (Table 1). Includes multiple metrics (Accuracy, F1, AUROC, AUPRC) and ablation studies demonstrating the contribution of each component.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The VPM generation is rule-based and not grounded in real visual data, raising concerns about physiological realism and validity. The approach is closer to feature engineering (visual priors) than a fundamentally new multimodal learning paradigm. Near-parity with large models (e. g. , GPT-5.2) is not fully convincing due to inconsistent evaluation and differences in modality. No human evaluation or assessment of triage safety (e. g. , under/over-triage). Reliance on derived datasets (MIETIC) and weak labels may introduce bias and limit generalization. Use of BERTScore for rationale quality does not reflect true clinical reasoning or actionability.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
The paper would benefit from validation against real multimodal data, clinician evaluation, and analysis of whether VPMs correspond to actual physiological patterns. Stronger baselines and simpler alternatives (e. g. , learned embeddings without explicit maps) would help clarify the true contribution.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This work introduces a creative and potentially impactful idea, synthetic anatomical grounding for multimodal triage, which is novel and well motivated. However, the approach is largely heuristic, and the evaluation does not fully validate clinical realism or practical utility. Despite these limitations, the conceptual contribution and empirical results make it a promising direction for further exploration.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The paper introduces MedTriage-LM, a multimodal framework designed for visual modality completion in public emergency department datasets that lack real patient images. The core contribution is the synthesis of Visual Phenotype Maps (VPMs), which project clinical evidence from FT-Transformer (tabular) and ClinicalBERT (text) encoders onto a canonical human silhouette via a weakly supervised Gaussian field generator. By aligning these spatial priors with clinical embeddings through cross-attention, the model shifts triage objectives from numerical risk scoring to actionable three-class instruction prediction which are Life-saving, High-Risk Assessment, and Resource Estimation.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
A major strength of the paper is its problem formulation. Rather than focusing only on conventional numerical triage scoring, the model targets actionable triage instruction categories, which is more aligned with how decision support could be used in emergency settings. This gives the work stronger practical motivation than many earlier ED prediction pipelines focused mainly on risk estimation.
Another strength is the Visual Phenotype Map (VPM) as a structured anatomical representation. It provides a visualization of the clinical information extracted from the structured variables and complaint text.
The paper also has a strong empirical section. It compares against a broad set of baselines, including traditional tabular models, specialized triage systems, fine-tuned foundation models, and zero-shot multimodal LLMs. In addition, the ablation study is reasonably good, showing the effect of adding text, the spatial branch, auxiliary outcome loss, weak supervision, smoothness regularization, and generation loss.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
A major weakness of the paper is that its methodological claim of “visual modality completion” for ED triage is conceptually weaker than the paper claims. The Visual Phenotype Map is not derived from an independent visual modality, but is deterministically generated from the same structured variables and complaint text already provided to the model. For example, if the input text says “chest pain,” the method may show a body template with thoracic activation, but this does not mean recovery of genuinely missing visual evidence. Rather, the approach is better understood as an anatomically structured reparameterization of existing non-visual inputs. This may still be valuable as an inductive bias, but the paper overstates the contribution by presenting it as missing-modality completion.
Second weakness is that the method for training the anatomical map lacks proper validation because the authors do not report the accuracy or coverage of their rule-based labels. These labels are created by simply mapping specific complaint words to body regions, yet there is no data on how often these rules fail or how many patient cases they actually cover. Without this reporting, it is unclear if the anatomical mapper is a reliable clinical tool or just a limited set of pre-defined rules that may not work in a complex, real-world emergency department.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Although the paper addresses an important clinical problem and presents a reasonably strong empirical evaluation, I still have some things to talk about its central methodological claim.
My score is mainly influenced by concerns about how the paper frames its core contribution. The claimed “visual modality completion” is conceptually overstated, since the Visual Phenotype Map is not inferred from a truly missing visual source but is deterministically generated from the same structured variables and complaint text already provided to the model. In that sense, the method seems closer to an anatomically structured reparameterization of existing inputs than to the recovery of genuinely missing visual evidence. Explaining this distinction more clearly in the paper would strengthen the contribution.
I was also concerned by the limited validation of the rule-based anatomical supervision. The paper does not report the accuracy of the complaint-to-region mapping rules used to train the anatomical map. Without such analysis, it is difficult to assess whether this component is clinically meaningful or whether it mainly reflects a limited set of heuristics.
Overall, I found the paper interesting and promising, and the authors have also uploaded anonymized code, which is a positive sign for transparency and reproducibility. While I believe the paper has enough merit, clarifying the main claims and providing stronger evaluation of the rule-based labels would make the work considerably more convincing.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
After the rebuttal, I support acceptance. The authors clarified that the Visual Phenotype Map is shown as an anatomically grounded spatial reparameterization rather than recovery of real missing images, which addresses my main concern about the framing of “visual modality completion.” The ablation results also shows that the visual-map branch and weak anatomical supervision contribute non-trivially to performance. I still think the paper would benefit from stronger validation of the rule-based anatomical labels, including their coverage and failure modes. However, the proposed idea is clinically motivated, and supported by a reasonably broad empirical evaluation, they also provided anonymized open source code as well which will be published later on. I therefore consider the remaining limitations acceptable and maintain an accept recommendation.
Review #3
- Please describe the contribution of the paper
This paper proposed a framework to help diagnose emergency severity assessment using LLM (i.e., MedTriage-ML). The final goal of the proposed method is to provide not only a clinical description but also visual representations. Overall, the manuscript is well-structed, and the experimental results effectively demonstrate the framework performance.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
In particular, the proposed framework prevents the chronic limitations of applying LLMs, which are known as hallucinations. Especially, chief complaint mapping, deterministic algorithm, and spatial smoothness are applied to solve the limitations. Also, empirical results indicate that the proposed framework achieves efficiency using an open dataset (i.e., MIETIC).
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The experimental results show its efficiency using quantitative comparisons; however, it is important to consider how the AI-applied method can reproduce medical diagnoses correctly. Therefore, human evaluation of the proposed method is essential, as noted by the authors. However, as a conference paper, the manuscript is well described to show their recent research correctly. Therefore, I think it may be accepted in MICCAI as the current version.
Minor review: Fig 2 is too small to identify a detailed description.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
As described in the previous comment, the proposed framework prevents the chronic limitations of applying LLMs, which can be a threat to medical systems. Also, the experimental results well describe the efficiency.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors have provided a reasonable rebuttal that satisfactorily clarifies my concerns. I look forward to seeing the promised additional experiments and software tools implemented in their future work.
Author Feedback
Dear Reviewers, Area Chairs, and Program Chairs,
We sincerely appreciate your constructive feedback and recognition of MedTriage-LM’s motivation, interpretability, and empirical strength. Below, we address the primary concerns raised during the review process.
#1.On the claim of “visual modality completion” (R1, R2, Meta-Reviewer) We agree that the phrase “visual modality completion” may have led to a stronger interpretation than intended. We do not claim to recover real, hidden patient images. Instead, the Visual Phenotype Map (VPM) is deterministically synthesized from textual and tabular inputs to serve as an explicit spatial inductive bias. We will revise our terminology from “modality completion” to “anatomically grounded reparameterization” to accurately reflect this contribution and avoid overstatement.
#2.On rule-based supervision and whether VPMs are only heuristics (R1, R2) Weak anatomical labels are not used as diagnostic ground truths, but solely as weak supervision to constrain the spatial prior and ensure anatomical plausibility. The final triage prediction is learned end-to-end. As shown in our ablation study (Table 2), this branch is not merely cosmetic: introducing the visual map improves Macro F1 from 0.820 to 0.855, and adding weak anatomical supervision further raises it to 0.878.This confirms that structured anatomical grounding extracts highly discriminative signals useful for triage.
#3.On baselines and the role of VPMs versus model capacity (R1) To isolate our methodological contribution, our primary baseline is the identical fine-tuned Qwen3-VL-8B backbone, rather than GPT-5.2.As shown in Table 1, MedTriage-LM significantly outperforms this baseline across all metrics. Because the backbone capacity is identical, this controlled comparison demonstrates that our VPM-based cross-modal alignment is the primary driver of the performance leap. We will make this ablation more central in the manuscript.
#4.On clinical realism, rationale evaluation, and triage safety (R1, R3) We will clarify that this is a methodological study utilizing a public dataset, rather than a completed clinical trial. While our framework includes strict safeguards against hallucination (deterministic synthesis, spatial smoothness, conditioned rationales), we agree that BERTScore alone does not prove clinical actionability. In our revision, we will emphasize that rigorous under- and over-triage safety analyses and structured clinician assessments are essential validation steps before deployment.
#5.On dataset dependence and generalization (R1) We acknowledge that reliance on the MIETIC dataset limits immediate generalizability. We will explicitly narrow our present claims to triage-time variables within this specific cohort and reiterate the necessity of multi-institutional validation across diverse ED settings in future work.
In summary, MedTriage-LM does not reconstruct real images; it introduces a deterministic, anatomically grounded reparameterization that provides a strong spatial inductive bias. Our same-backbone ablations confirm its efficacy, while comprehensive clinical validations remain an important next step. We thank you again for the opportunity to strengthen our work.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Reviewer 1 and Reviewer 2 both found the idea interesting and potentially useful, but both questioned the strength of the central claim. In particular, Reviewer 1 noted that the Visual Phenotype Map is rule-based and not grounded in real visual data, and Reviewer 2 made the related point that the claimed visual modality completion is weaker than stated because the visual representation is generated from the same underlying inputs rather than from an independent modality. Reviewer 3 was more positive and gave an Accept, but still noted the need for stronger discussion of diagnostic correctness and human evaluation.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The main concern was overclaiming “visual modality completion,” but the authors explicitly acknowledged this and will reframe the contribution as anatomically grounded reparameterization. This resolves the central conceptual issue.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
Reviewers found the paper conceptually novel and clinically motivated, with useful anatomically grounded visual phenotype maps, broad empirical evaluation, ablations, and released anonymized code. Although validation of rule-based anatomical labels and human/clinical safety evaluation remain limited, the rebuttal clarified the intended framing and addressed the main concerns sufficiently. I therefore recommend acceptance.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
I think this one has a good rebuttal, and the scores are above the threshold. I recommend acceptance.