Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Video-based Clinical Gait Analysis often suffers from poor generalization as models overfit environmental biases instead of capturing pathological motion. To address this, we propose BioGait-VLM, a tri-modal Vision-Language-Biomechanics framework for interpretable clinical gait assessment. Unlike standard video encoders, our architecture incorporates a Temporal Evidence Distillation branch to capture rhythmic dynamics and a Biomechanical Tokenization branch that projects 3D skeleton sequences into language-aligned semantic tokens. This enables the model to explicitly reason about joint mechanics independent of visual shortcuts. To ensure rigorous benchmarking, we augment the public GAVD dataset with a high-fidelity Degenerative Cervical Myelopathy (DCM) cohort to form a unified 8-class taxonomy, establishing a strict subject-disjoint protocol to prevent data leakage. Under this setting, BioGait-VLM achieves state-of-the-art recognition accuracy. Furthermore, a blinded expert study confirms that biomechanical tokens significantly improve clinical plausibility and evidence grounding, offering a path toward transparent, privacy-preserving gait assessment.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/1613_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

N/A

Link to the Dataset(s)

GAVD dataset: https://github.com/Rahmyyy/GAVD

BibTex

@InProceedings{CheErd_BioGaitVLM_MICCAI2026,
        author = { Chen, Erdong AND Ji, Yuyang AND Greenberg, Jacob K. AND Steel, Benjamin AND Arkam, Faraz AND Lewis, Abigail AND Singh, Pranay AND Liu, Feng},
        title = { { BioGait-VLM: A Tri-Modal Vision–Language–Biomechanics Framework for Interpretable Clinical Gait Assessment } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16882},
        month = {September},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper addresses video-based clinical gait assessment, with a particular focus on improving generalization and interpretability. The authors argue that standard video models often overfit environmental cues and do not provide clinically meaningful biomechanical evidence. To address this, they propose BioGait-VLM, a tri-modal framework that combines visual features, temporal modeling, and biomechanics represented as language-aligned tokens. The paper also introduces a new DCM clinical gait dataset. Additionally, it evaluates not only classification performance but also interpretability through a blinded expert assessment.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The proposed architecture is interesting and well motivated. In particular, the combination of a temporal branch and a biomechanics-as-language branch is conceptually appealing, because it tries to disentangle motion evidence from static appearance and environment. The biomechanics tokenization idea is one of the most original aspects of the paper. 2.The paper establishes a more rigorous clinical gait benchmark by augmenting the public GAVD dataset with a newly collected DCM cohort and enforcing a subject-disjoint evaluation protocol.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The baseline comparison is not fully balanced or sufficiently comprehensive. The paper fine-tunes standard video encoders such as SlowFast and TSN, but evaluates the general-purpose LVLM baselines (Qwen3-VL and InternVL3.5) only in a zero-shot setting. In addition, the comparison does not include a stronger baseline that is closer to the core idea of the paper, such as a prior multimodal or skeleton-based method. 2.Although the overall performance is promising, the class-wise results remain quite uneven. For the full model, Myopathic class still have relatively low F1 scores (36.4), even though other categories perform much better. Therefore, the claims about strong generalization would be more convincing if phrased more carefully, as the method appears substantially stronger on some classes than on others. 3.The expert study is a valuable addition, but its design is limited, which makes the qualitative conclusions less robust than the presentation may suggest. The protocol states that the cases are divided into non-overlapping subsets and that each evaluator reviews 13 unique cases, which suggests that each case may have been rated by only one expert. If so, the reported Likert scores and comparative judgments should be interpreted with some caution.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This work addresses an important problem in clinical gait analysis, with a focus on generalization and interpretability. The proposed tri-modal framework is well motivated, and the biomechanics-as-language idea is novel and compelling. Additionally, the paper contributes a more rigorous benchmark by extending GAVD with a new DCM cohort under a subject-disjoint protocol. However, the empirical validation has some limitations. The baseline comparison is not fully balanced, the class-wise performance is uneven, and the expert study design appears limited. Overall, the paper is promising and relevant, but the evaluation could be stronger to fully support the claims.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    They authors created a novel tri-modal framework that projects 3D kinematics into text tokens, allowing the language model to reason about movement robustly and provide interpretable clinical assessments.

    Building a specialized branch to compress video frames into compact dynamic descriptors (motion queries M=32), ensuring the model captures the temporal evolution is a good idea.

    They created a new dataset for DCM, with expert annotations. New clinical datasets with expert annotations are always good to have.

    For me the main highlight of paper is blinded expert evaluation which is very rare and quite valuable in this field. It also significantly improves clinical correctness and grounds model predictions in real evidence.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Instead of using the InternVLM only for talking, authors use it as the central processor that aligns pixel-level visual features with high-level biomechanical text tokens. Which is good for internal understanding of model.

    The classification head driven by hidden states of LLM is a good approach. Their framework extracts the internal hidden states from llm after it has processed the gait data. These hidden states, which contain the model’s complex semantic reasoning (from interVL), are then concatenated with temporal features to form the final representation, which are used for final diagnosis.

    The blinded expert validation using 5 point Likert’s scale used in this papers is one of major strengths. Along with the leakage-aware subject-disjoint protocol for validation.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1: The authors likely skipped the specialized gait models because those models don’t typically generate “clinical rationales”. However, for the Diagnostic Classification task (the 8-class labels), a comparison with a specialized skeleton-only model (CT-GCN, Hyper-GCN etc) would have been the “gold standard” to prove that their tri-modal approach actually adds value. Without this, one could argue that a simple, lightweight GCN might achieve the same accuracy without the massive computational overhead of an LLM.

    2: the authors freeze the InternVL parameters and only train the temporal decoder and the classification head, based on this there is no mention of a language modeling loss used to refine the clinical reports during training. Moreover, the model relies entirely on the pre-trained knowledge of InternVL (even though it was not trained on clinical/mechanical data) to interpret the biomechanical tokens.

    3: The biomechanical branch which is the core of the paper’s interpretability, is entirely dependent on an off-the-shelf framework HSMR (CVPR’25) to extract skeleton parameters, did authors validated it’s in-wild performance (as authors claim the generalizability in abstract) before testing on their data as there must be a huge domain shift in the data HSMR was trained on and the GAVD and DCM data.

    4: Given the privacy-preserving nature of the Biomechanical Tokens, do the authors intend to release the code, de-identified skeleton parameters/sequences or the model’s pretrained weights to the research community (for reproducibility)? Just so results can be verified or model can be tested on GAVD public dataset.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Lack of comparison with SOTA models in GAIT domain. There is no mention of a language modeling loss used to refine the clinical reports during training. so how can model learn to generate reports showing what the experts look for in GAIT analysis report. No visual samples of collected data are provided.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #3

  • Please describe the contribution of the paper

    This paper introduces BioGait-VLM, a tri-modal framework for clinical gait assessment that fuses RGB video, a temporal evidence module, and “biomechanical tokenization” of 3D skeletons within a frozen large vision-language model (LVLM). The method adds a query-based Temporal Evidence Distillation (TED) branch to preserve rhythmic motion cues and converts per-frame joint kinematics into structured natural-language tokens that the LLM can “read,” aiming to improve both robustness and interpretability. The authors also curate a Degenerative Cervical Myelopathy (DCM) video cohort and combine it with GAVD to form an 8-class benchmark with strict subject-disjoint splits, reporting state-of-the-art performance and improved rationale quality in a small blinded expert study.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The tri-modal design is thoughtful: combining frozen LVLM visual tokens, a query-based temporal aggregation module, and explicit biomechanical signals is a promising way to mitigate shortcut learning and enhance interpretability.
    • The “biomechanical tokenization” idea, mapping numeric kinematic features to structured text aligned with an LLM’s semantic space, is novel and potentially impactful for clinical explainability and privacy (by enabling skeleton-centric reasoning).
    • The TED branch (latent motion queries with cross-attention) is a lightweight, architecture-agnostic way to inject temporal sensitivity beyond naive pooling in general-purpose LVLMs.
    • The authors re-partition the benchmark with patient-disjoint splits to reduce identity leakage, which is critical in this domain and directly addresses a known limitation of GAVD.
    • Ablations isolate the contribution of TED and biomechanical tokenization, supporting the claim that both are important.
    • The blinded expert assessment adds a valuable qualitative dimension, directly evaluating interpretability and evidence grounding.
    • The problem motivation and clinical relevance are clearly articulated; the system diagram and high-level pipeline are easy to follow.
    • Training protocol (frozen backbone, lightweight adapters) is sensible for limited clinical data and is reasonably described.
    • Interpretable, evidence-grounded gait assessment is a high-impact target for MICCAI; integrating biomechanics with LVLMs provides a credible path from black-box classifiers to clinically useful tools.
    • The addition of a curated DCM cohort fills a notable gap and enables evaluation on an underrepresented yet clinically important pathology.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • The approach hinges on reliable 3D pose/kinematics from monocular video (HSMR). The paper does not quantify failure modes, calibration assumptions, or uncertainty propagation from pose errors to tokenized text, which can be substantial in clinical settings.
    • The baseline comparison is weaker than it should be for the paper’s main claim. The strongest comparison should have been against a closely matched adapted VLM baseline using the same frozen InternVL backbone and training budget, but without the proposed TED and biomechanical design. The paper partly approximates this with “w/o both,” but that is still an internal ablation, not an external competitive baseline. The external VLM baselines are zero-shot Qwen3-VL and zero-shot InternVL3.5, which are much weaker comparisons because the proposed model is adapted to the task, while those are not. This makes the large performance gap less informative than it first appears.
    • Some class-wise results reveal substantial remaining weakness. Even in the full model, Style has an F1 of only 4.3, and Myopathic only 36.4.Parkinson’s improves meaningfully but is still just 35.3.This indicates the model still has serious confusion in several categories. Because the benchmark includes non-pathological style/exercise conditions together with neurological/pathological classes, these low scores are important: they suggest the model may not yet separate pathological and non-pathological deviations robustly enough for dependable clinical use.
    • The term “Temporal Evidence Distillation” is a misnomer here; the module is a query-based aggregator rather than a true distillation stage (no explicit teacher/student or evidence generation/selection). This could confuse readers and overstate novelty relative to existing temporal grounding/distillation ideas.
    • The privacy-preserving claim is only partly supported, as RGB is still used at inference; a skeleton-only variant and analysis are not reported.
    • Baselines are limited. Comparisons are only to SlowFast/TSN (fine-tuned) and two zero-shot LVLMs. Notably absent are skeleton-based classifiers (e.g., ST-GCN/PoseC3D/transformer-on-pose), video transformer baselines(TimeSformer/VideoMAE), and a numeric-skeleton fusion baseline that does not rely on text tokenization. This makes it hard to isolate whether gains are due to LVLM alignment versus simply adding skeleton information.
    • No analysis to rule out environmental confounding remains after re-splitting. DCM is collected in a standardized clinical environment; without person-cropping/background-masking or a skeleton-only evaluation, the very high DCM F1 (98.1%) could reflect setting-specific cues.
    • Statistical robustness is unclear: no confidence intervals or repeated trials are reported, and per-class sample sizes are not disclosed, yet per-class F1 varies drastically (including zeros for some classes in ablations/baselines).
    • The blinded study is small (4 raters, 52 sequences from 6 patients, DCM only), with no report of inter-rater reliability, randomization procedures, or power analysis. This limits generality to broader gait classes.
    • Important implementation details are missing: the exact 46D kinematic vector composition and joint definitions, coordinate system and normalization, smoothing/denoising, the tokenization template details, and how cross-modal fusion within the frozen LLM yields fvlm (which layers/tokens are pooled).
    • The fusion of features (V + Ebio) and placement of TED outputs in the LLM stack are not fully specified; reproducibility would benefit from more precise architectural specifics.
    • The paper does not adequately situate itself against: (i) skeleton-based gait analysis/classification methods; (ii) video transformers and recent masked/distilled video models (e.g., VideoMAE/MVD); and (iii) temporally grounded evidence generation/distillation paradigms in video-language (e.g., VITED), which are thematically close to the TED rationale.
    • The interpretability mechanism is plausible, but not validated for faithfulness. The paper shows that explanations become more concrete when biomechanics are injected as tokens. However, it does not test whether those tokenized joint descriptors are quantitatively accurate enough to support clinical reasoning. Since the 3D skeleton comes from an off-the-shelf estimator and then gets converted into text, there are at least two possible sources of distortion: pose-estimation error and discretization/templating into language. The paper evaluates clinician preference for the rationales, but not whether the cited biomechanical evidence is actually faithful to true motion measurements.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    This paper is strongest when it is concrete: the temporal module replaces weak temporal pooling, and the biomechanical branch injects pose-derived descriptors into the model’s reasoning pathway. That is the right story to emphasize. The paper is less convincing when it makes broader claims about robustness or transparency without corresponding analysis. I would encourage the authors to be more precise in interpreting their own tables. For example, the biomechanics branch clearly helps explain quality, but Table 1 does not show a clear DCM classification gain from that branch.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper presents a genuinely interesting and clinically relevant idea: augmenting a frozen LVLM with explicit temporal modeling and pose-derived biomechanical tokens for more interpretable gait assessment. The method is well motivated, the subject-disjoint evaluation protocol is a meaningful strength, and the results suggest that the proposed design improves both classification performance and the clinical grounding of generated rationales. In particular, the blinded expert study provides encouraging evidence that the biomechanics-aware variant produces outputs that are more concrete and clinically useful. However, the empirical validation is not yet strong enough for a higher score. The comparison set is not fully convincing, since key matched baselines are missing, including skeleton-only and direct visual-skeleton fusion approaches, and the external baselines are either relatively weak or not directly comparable to the proposed adaptation setting.

  • Reviewer confidence

    Not confident (1)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Author Feedback

N/A



Meta-Review

Meta-review #1

  • Your recommendation

    Provisional Accept

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The paper deals with the issue of clinical gait analysis and assessment. For this task, the authors introduce BioGait-VLM, a tri-modal framework that fuses RGB video, a temporal evidence module, and “biomechanical tokenization” of 3D skeletons within a frozen large vision-language model (LVLM). More importantly and innovatively, the authors have added a query-based Temporal Evidence Distillation (TED) branch to preserve rhythmic motion cues, converting per-frame joint kinematics into structured natural-language tokens that the LLM can “read,” aiming to improve both robustness and interpretability. To validate their approach, the authors also introduce a new DCM clinical gait dataset. Most importantly, the authors perform evaluation not only limited to classification performance but also interpretability through a blinded expert assessment, which is highly preferrable for clinical application settings.

    All reviewers unanimously have agreed that this paper is well motivated, has provided a decent technically sound framework, with a new dataset framing and reader study for the task of clinical gait assessment. The reviewers have unanimously agreed on a positive consideration for this work, however, certain clarifications to questions raised is also required for a fair justification. So, I suggest the authors to read through reviewers comments and answer judiciously. Specifically, below comments are encouraged to be clarified:

    1.Why skeleton-based models are not tried? (R#3,R#4): Comparison with a specialized skeleton-only model (CT-GCN, Hyper-GCN etc) would have been the “gold standard”. This makes it hard to isolate whether gains are due to LVLM alignment versus simply adding skeleton information. Additionally, the authors do not quantify failure modes, calibration assumptions, or uncertainty propagation from pose errors to tokenized text, which can be substantial in clinical settings. Moreover, the biomechanical interpretability branch is entirely dependent on an off-the-shelf framework HSMR (CVPR’25). Did the authors test the in-the-wild performance or issues when it doesn’t perform well? Additionally, failure modes, calibration assumptions, or uncertainty propagation from pose errors to tokenized text shoud have been discussed and explored.

    2.Lack of details for qualitative reader study (R#2, R#4): No report of inter-rater reliability, randomization procedures, or power analysis. The protocol also states that the cases are divided into non-overlapping subsets and that each evaluator reviews 13 unique cases, which suggests that each case may have been rated by only one expert. Please clarify.

    3.Dependency on InternVL parameters and baseline comparision (R#2, R#3, R#4): No mention of a language modeling loss used to refine the clinical reports during training as InternVL is frozen. Moreover, the model relies entirely on the pre-trained knowledge of InternVL. Additionally, please clarify why the authors fine-tune standard video encoders such as SlowFast and TSN, but evaluate the general-purpose LVLM baselines (Qwen3-VL and InternVL3.5) only in a zero-shot setting.

    4.Low class-wise performance (R#2,R#4): For the full model, Myopathic class still have relatively low F1 scores (36.4), even though other categories perform much better. Could the authors clarify and discuss why?

    There are also some reproducibility questions requiring clarification on architecture details and others, along with rationale for the interpretability study (R#4).



back to top