List of Papers Browse by Subject Areas Author List
Abstract
Emotion recognition using physiological signals is pivotal for mental health assessment, and multimodal fusion is a key strategy for enhancing performance. Electroencephalography (EEG) and eye movement (EYE) signals are widely used as they provide complementary affective information. However, EEG’s complex acquisition limits its real-world feasibility, while the more accessible EYE modality exhibits inferior performance when used alone. To address this, we propose CoSim, a framework integrates Conditional Prompting with Similarity-Guided Knowledge Augmentation (SKA). Our approach aims to enhance the inference-time performance of the EYE modality by transferring knowledge from the information-rich EEG modality, which is available exclusively for training. During training, a shared representation space is learned from paired EEG-EYE data, augmented by samples from similar subjects to mitigate data scarcity and individual variability. During testing, when EEG is unavailable, a Conditional Prompting Generator (CPG) generates a discriminative representation from the EYE input to compensate for the missing modality. The framework is built on a Fourier-Enhanced Transformer (FET) backbone to capture critical time-frequency characteristics. Extensive experiments demonstrate that our model achieves state-of-the-art performance. For instance, on the SEED dataset, our EYE-only model reaches 98.52% accuracy, achieving comparable performance to full-modality and validating its potential for practical deployment. The code is available at https://github.com/AnXiaoL/CoSim.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/4596_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/AnXiaoL/CoSim
Link to the Dataset(s)
SEED dataset: https://bcmi.sjtu.edu.cn/home/seed/seed.html
SEED-IV dataset: https://bcmi.sjtu.edu.cn/home/seed/seed-iv.html
BibTex
@InProceedings{AnXia_CoSim_MICCAI2026,
author = { An, Xiaoling AND He, Siyuan AND Liu, Zewen AND Zhou, Chaofei AND Qin, Jing AND Zhang, Daoqiang AND Hao, Xiaoke},
title = { { CoSim: Unleashing Eye Movements for EEG-Free Emotion Recognition via Conditional Prompting and Similarity-Guided Augmentation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16896},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The authors present an approach to classify emotional state based on EEG and EYE data. The authors introduce a conditional prompting generator that generates pseudo EEG representation conditioned on the EYE input when EEG data is absent. Authors also introduce a technique called similarity-guided knowledge augmentation that utilizes data from physiologically similar subjects.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The concept of the conditional prompting generator is interesting and one of the potential strengths of the contribution if validated properly. Likewise the proposed similarity guided augmentation is an interesting concept that could translate to other use cases beyond the one considered.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
My main issue is the limited sample size in the SEED datasets that were used to validate the approaches. Authors have described how they select similar subjects and also how the CPG is trained to generate EEG data but it is unclear how this would translate to a real world setting with more than 15 or 20 subjects. In other words, it could be that the subjects in the SEED study are very similar and that the model simply learns to predict the same average patterns. The accuracy achieved is certainly good but to truly judge the significance of these results it would be helpful to analyze the differences in feature distributions in the train and test sets. How different are they really? Can the models really generalize to subjects which have a different EYE - EEG response?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
More clarification is needed on the dataset and subjects included (see above) to understand the significance of the results. I am happy to raise my vote if this is convincingly addressed in the rebuttal.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
After reviewing the comments of other reviewers and the rebuttal. I recommend acceptance. Authors have convincingly addressed my and others’ concerns.
Review #2
- Please describe the contribution of the paper
This paper proposes CoSim, a missing-modality emotion recognition framework that aims to perform EEG-free inference using only eye-movement (EYE) signals at test time, while leveraging paired EEG-EYE data during training. Methodologically, the framework combines three components: (1) a Conditional Prompting Generator (CPG) that synthesizes pseudo-EEG features conditioned on EYE features, (2) a Similarity-Guided Knowledge Augmentation (SKA) strategy that augments a target subject’s training data with samples from top-K similar subjects, and (3) a Fourier-Enhanced Transformer (FET) backbone intended to better model time-frequency structure in physiological signals. The full training pipeline contains a universal multimodal pretraining stage, followed by a personalized adaptive finetuning stage with random modality dropout, and finally an EYE-only test stage where pseudo-EEG features are generated and fused with EYE features for classification. The paper reports strong accuracy on SEED and SEED-IV under a subject-dependent setting.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
(1) The paper addresses a relevant and practically motivated problem: reducing dependence on EEG at test time while still exploiting EEG during training. This is an important direction because EEG acquisition is cumbersome, whereas eye tracking is substantially more accessible. The motivation is clearly stated, and the test-time setting is meaningful.
(2) The paper presents a reasonably coherent system design rather than a single isolated trick. The combination of multimodal pretraining, conditional pseudo-feature generation, subject-aware data augmentation, and a specialized transformer backbone is well organized and easy to follow in Fig. 1 and the method section. In particular, the distinction between pretraining, personalized finetuning, and EYE-only inference is clearly described.
(3) Under the chosen protocol, the method outperforms several prior cross-modal baselines and also compares favorably to some EEG-only and full-modality baselines. The ablation study also suggests that CPG, SKA, and FET each contribute positively to performance.
(4) The feature visualization and hyperparameter sensitivity analysis for SKA support the intuition behind the framework and make the paper more complete experimentally.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
（1) The authors claim the approach aims to “enhance the inference-time performance” of the EYE modality in the abstract. However, the experimental section primarily reports static accuracy on standard datasets. To truly demonstrate “enhanced inference-time performance” in a way that justifies the complexity of CPG, the authors should provide a more detailed analysis of the computational overhead and latency incurred by pseudo-feature generation during inference relative to standard EYE-only models. (2) The proposed CPG module functions primarily as a statistical feature translator. However, it fails to leverage explicit neurophysiological priors governing the EEG-EYE relationship (e.g., the coupling between saccadic eye movements and specific neural oscillations). The authors should clarify how the FET backbone or CPG specifically aligns with known physiological mechanisms rather than relying solely on adversarial alignment. (3) The novelty of SKA is not fully convincing, and its effectiveness remains insufficiently validated. The idea of selecting top-K similar subjects based on prototype similarity and then performing weighted sampling is intuitive. Still, it appears closer to a standard neighbor-based augmentation strategy than to a genuinely new formulation. More importantly, the paper does not compare SKA with simpler alternatives, such as class-balanced augmentation or EYE-only nearest-neighbor matching. Since SKA is intended to reduce overfitting, stronger evidence is needed to show that the similarity criterion itself matters.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper addresses a relevant and practical problem: enabling EEG-free inference while still leveraging EEG during training. The framework is generally coherent, clearly presented, and empirically strong under the subject-dependent setting. The ablation, feature visualization, and sensitivity analysis also improve the experimental completeness.
My main concerns are threefold. First, although the paper emphasizes inference-time improvement, it does not quantify the additional test-time cost or latency introduced by pseudo-EEG generation. Second, CPG is effective empirically, but its physiological grounding remains weak, as the connection to known EEG-EYE mechanisms is not clearly established. Third, the novelty and validation of SKA are not fully convincing. The method resembles a neighbor-based augmentation strategy, and the paper lacks comparisons with simpler alternatives such as class-balanced augmentation or EYE-only nearest-neighbor matching. Overall, my score reflects a meaningful problem and strong results, but also concerns about practical justification, physiological interpretability, and the validation of key components.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
I appreciate the authors’ effort in providing a detailed rebuttal and temporary experimental logs. While some minor presentation and formatting issues have been resolved, the core concerns regarding the methodology’s intrinsic novelty, the justification for physiological coupling, and experimental validation remain insufficiently addressed. Therefore, I maintain my opinion based on the following critical points:
1.Disconnect Between Neurophysiological Priors and Network Architecture: The authors provide a sophisticated neuroscientific narrative in their rebuttal to justify the FET and CPG modules. However, there remains a fundamental gap between these high-level biological concepts and the actual mathematical formulation of the model. Simply using frequency-domain partitioning (Eqs. 7, 8) to separate periodic and aperiodic components is a standard signal processing technique; it does not inherently mean the network is “explicitly driven” by specific EEG-EYE coupling mechanisms. Without integrating structural constraints that explicitly reflect how saccades modulate neural oscillations, the model remains fundamentally a data-driven statistical translator operating under an adversarial alignment framework, making the “biological plausibility” claim over-engineered and less convincing.
2.The authors clarify that during inference, CPG adds a 2.14 ms latency, which is acceptable for real-time applications. However, the parameter size balloons drastically from 97.4k to 1.5M (a nearly 15-fold increase). For an “EYE-only” deployment scenario—which usually implies resource-constrained edge devices (like wearable eye-trackers)—such a massive parameter explosion severely undercuts the practicality of the approach. Trading a massive footprint for accuracy via pseudo-feature generation undermines the original motivation of lightweight EYE-modality inference, and the complexity overhead is not fully justified compared to optimizing a native, robust EYE-only model.
In summary, while the performance metrics on the SEED dataset are high, the paper leans heavily on complex deep learning modules justified by post-hoc neuroscientific narratives, without true structural integration of those priors. Given the incremental nature of the SKA module and the substantial parameter overhead, the paper, in its current form, does not meet the high bar of methodological rigor and practical innovation required for acceptance.
Review #3
- Please describe the contribution of the paper
Traditional emotion recognition model use Electroencephalography (EEG) and Eye Movement (EYE) signals. To reduce cost and deployment complexity, the authors propose a framework that uses only EYE at inference. The system integrates Conditional Prompting with Similarity-Guided Knowledge Augmentation (SKA) to generate pseudo-EEG representations conditioned on the EYE input, then use a machine learning model built on a Fourier-Enhanced Transformer (FET) backbone for classification. The reported results are promising and suggest potential for more practical, scalable, and cost-effective deployment of emotion recognition systems.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1) The authors developed Conditional Prompting Generator (CPG) that generates pseudo-EEG representations conditioned on the EYE input.
2) They also Similarity-Guided Knowledge Augmentation (SKA) aimed to reduce CPG overfitting.
3) They developed and ML model using Fourier-Enhanced Transformer (FET) Backbone, achieving higher accuracy than the state-of-the-art in the literature.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
Although the paper is excellently written and is publishable in its current form. There are few correction that would further improve it.
1) In section 2.1 the authors forgot to define the lambda components of their equations.
2) The authors should soften the claim in Section 2.3: “To mitigate this, we propose SKA strategy that regularizes training by leveraging data from similar subjects.” to “To mitigate this, we propose SKA strategy that aims to regularize training by leveraging data from similar subjects.”, because the earlier sentence presents the effect as established, whereas this is not necessarily guaranteed. Although augmenting a target subject’s training set with data from similar subjects may improve generalization, it may also introduce bias or encourage the model to fit subject-specific patterns, depending on how similarity is defined and how the augmentation is performed.
3) In the third sentence of Section 2.4, the authors should revise “Inspired by FAN [1], …” to “Inspired by Fourier Analysis Networks (FAN) [1], …” since this is the first occurrence of the acronym.
4) The first sentence in the third paragraph of Section 2.4 should be revised from “By applying this structure to the Query, Key, and Value projections, …” to “By applying this structure to the Query (Q), Key (K), and Value (V) projections, …”. This would make the connection to Equation (10) clearer for the reader.
5) In the Dataset subsection of Section 3.1, the authors should cite the original sources of the SEED and SEED-IV datasets. This is important both to properly acknowledge the dataset creators and to help readers easily locate the data.
6) The claim (in the second sentence of Section 3.3) that removing SKA makes the model “prone to overfitting” is not sufficiently supported by the reported mean accuracy +/- standard deviation alone. A performance drop only indicates that SKA contributes positively under the evaluated setting, but it does not by itself establish overfitting as the cause. To support such a claim, the authors should provide additional evidence, such as training vs. validation/test performance, loss curves, or explicit generalization-gap analysis. Otherwise, the statement should be softened to indicate reduced robustness or degraded generalization rather than overfitting (as the authors did with CPG in the first sentence).
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(6) Strong Accept — must be accepted due to excellence
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is grammatically well-written. It also presented the framework in sufficient detail, as allowed by the conference’s page limitation. To the best of my knowledge, I could not find any paper that developed a similar framework
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We thank the Meta Reviewer (Meta) and reviewers for constructive feedback and acknowledging our novelty. Due to the double-blind policy, the code will be open-sourced upon acceptance. Responses to core concerns follow: [Meta, R1] Generalization & Train/Test Shift: SEED is a standard benchmark; its original paper [Zheng & Lu, IEEE TAMD ‘15] confirms significant cross-subject variability. The t-SNE plots in Fig. 3a/b show CPG-generated features avoid mode collapse and don’t degrade to a static mean, capturing dynamic physiological changes. The train/test sets also use distinct video stimuli. Due to signal non-stationarity and stimulus disparity, a significant distribution shift inherently exists. Maintaining high accuracy here proves our model extracts domain-invariant features and universal physiological laws, demonstrating robust generalization. Broad-scenario generalization will be verified via large-scale multi-center datasets in future work. [Meta, R2] Inference Cost: During inference, CPG only executes a lightweight generator, adding marginal constant-time (O(1)) overhead compared to GANs. Profiling logs on SEED show pseudo-feature generation adds just 2.14 ms latency (total inference per sample is 3.22 ms). We trade 2.14 ms latency and increased parameters (from 97.4k to 1.5M) for an accuracy leap from 84.90% to 98.52% (Tab. 2b). This boosts performance at an acceptable delay, fully satisfying real-time deployment needs. A detailed time analysis will be added. [Meta, R2] Physiological Interpretation: Neuroscience confirms EYE-EEG coupling via: (1) periodic rhythmic synchronization (e.g., saccade-EEG phase-locking [Staudigl et al., PLoS Biol. ‘17]); (2) aperiodic state covariation (e.g., pupil-arousal mapping to 1/f EEG background [Joshi et al., Neuron ‘16; Donoghue et al., Nat. Neurosci. ‘20]). Leveraging this prior, CPG (Eqs. 1, 2) converts eye-movement cues into conditional prompts, providing initial cross-modal guidance. Since neural rhythms and aperiodic baselines are highly temporally entangled, FET (Eqs. 7, 8) mathematically enforces signal decoupling via the frequency domain. Features are strictly partitioned into periodic subspaces (saccades mapped to neural oscillations) and aperiodic subspaces (pupil-linked arousal baselines). This structural constraint avoids blind data-driven mapping, explicitly ensuring biological plausibility. Thus, our architectural design is driven by these neurophysiological priors. [Meta, R2] SKA Mechanism & Baselines: We carefully considered this during design. Our goal is to extract data with deep physiological similarity, not mere volume augmentation. Conventional class-balanced augmentation introduces heterogeneous noise, while eye-only nearest-neighbor search suffers from signal sparsity. Thus, SKA uses information-rich joint EEG-EYE features to build a similarity matrix during training, overcoming these issues and outperforming both baselines. Fig. 3c further validates this: as neighbor count k increases, including low-similarity subjects causes a noticeable accuracy drop. This confirms that blind augmentation introduces interference, making SKA’s precise matching optimal. [Meta, R2, R3] “Overfitting” Clarification: We agree a simple accuracy drop doesn’t rigorously prove overfitting. However, an analysis of the training curves (from the training and test loss logs) reveals SKA reduces the generalization gap by 60.7% on average. This mechanistically confirms that SKA substantially improves generalization ability, indicating the baseline drop involves overfitting factors. To ensure academic rigor, we will use more precise terminology. For example, following R3’s suggestion, we will replace ‘overfitting’ with ‘degraded generalization’ and ‘reduced robustness’. [R3] We will adopt all suggestions: (1) clearly define λ in Sec. 2.1; (2) tone down the assertion in Sec. 2.3 to “aims to regularize”; (3) provide full names for FAN and Q/K/V at first use; (4) add original dataset citations.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper addresses a practical problem of EEG-free emotion recognition by using paired EEG-EYE data during training and only EYE signals at inference. The proposed CoSim framework combines a Conditional Prompting Generator, Similarity-Guided Knowledge Augmentation, and a Fourier-Enhanced Transformer, and reviewers generally acknowledge the clear motivation, coherent design, and promising empirical results.
I recommend inviting a rebuttal. The authors should mainly address the following points: clarify whether the method can generalize beyond the small SEED/SEED-IV subject pools and provide analysis of train/test feature distribution differences; report the inference-time overhead and latency introduced by pseudo-EEG generation; better justify the physiological basis of the EEG-EYE relationship modeled by CPG and FET; strengthen the validation of SKA by comparing with simpler alternatives such as class-balanced augmentation or EYE-only nearest-neighbor matching; and soften unsupported claims about overfitting unless training/validation curves or generalization-gap analyses are provided.
Overall, the paper has clear merit, but the rebuttal should focus on dataset generalization, practical inference cost, physiological interpretation, and stronger evidence for the proposed SKA mechanism.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
This paper addresses a practical and meaningful problem by enabling EYE-only emotion recognition at inference time while using paired EEG-EYE data during training. The proposed CoSim framework is clearly organized, combining conditional pseudo-EEG generation, similarity-guided subject augmentation, and a Fourier-enhanced backbone, and the reported results, ablation study, visualization, and sensitivity analysis provide reasonable support for its effectiveness. After rebuttal, two reviewers support acceptance, including one reviewer who changed from weak reject to accept, while one reviewer maintains concerns about the strength of the physiological interpretation, the novelty of SKA, and the increased parameter size. I consider these concerns valid but not fatal; they mainly require clearer positioning and limitation discussion rather than rejection. Therefore, I recommend acceptance, with the expectation that the authors revise the camera-ready version to soften claims about explicit neurophysiological grounding, clearly report inference latency and parameter size, clarify the role and novelty of SKA, cite the original SEED and SEED-IV datasets, and avoid unsupported statements about overfitting.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation.
I tend to agree more with R2.The paper has some merit in terms of the proposed framework and strong performance. The rebuttal, however, does not adequately address some of the major concerns regarding insufficient novelty and validation. With substantially high computational overhead at test time and lacking generalization beyond the specific data distribution (SEED), it is difficult to justify an acceptance.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
There were several points raised by the reviewers: The generalisation of the method to other datasets, practical deployment of the model, a justification for the method tied to the physiology,
The others attempted to answer these concerns. The authors response related to the prior underpinning does seem overstated based on what the experimental design can show versus the thinking. However the experiments and approach overall seem interesting and novel even if some of the interpretation of what the method is doing is likely overstated. I would suggest the authors think carefully about what they can actually demonstrate and keep those points in while removing or softening language for elements they believe are encoding specific types of information but have no evidence for.
The inference speed, and small sample size were both address well in the rebuttal.