Abstract
Surgical workflow anticipation predicts future surgical phases and instrument usage to support proactive decision-making. Existing methods rely on visual observations and temporal architectures but lack implicit modeling of future surgical dynamics and procedural knowledge about how surgeries evolve.We propose FOGE, a future-oriented framework with knowledge guided embedding. First, we introduce a JEPAbased world-modeling objective that trains the temporal encoder to predict future visual representations, encouraging forward-looking temporal dynamics without additional inference cost. We further design a hierarchical temporal interaction module to enrich frame features through local context exchange. Second, we propose a VLM semantic prior module that distills procedural reasoning from a large vision-language model through a two-pass reasoning-to-prior pipeline. Finally, we design a masked-query transformer decoder where world and task queries share a single attention layer while remaining structurally decoupled via attention masking. Experiments on the Cholec80 benchmark demonstrate that FOGE achieves state-of-the-art performance, and ablation studies validate the contribution of each component.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/CREATE_004.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
Open Review Page: https://openreview.net/forum?id=xw5qJQeNpW
BibTex
@InProceedings{ZhaShu_Predictive_MICCAISAT2026,
author = { Zhang, Shuo AND Wu, Jinlin AND Yi, Dong AND Yang, Zixuan AND Lei, Zhen AND Liu, Hongbin},
title = { { Predictive Representation Learning with Prior Knowledge for Surgical Workflow Forecasting } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17275},
month = {pending},
page = {pending}
}