List of Papers Browse by Subject Areas Author List
Abstract
Electroencephalography (EEG) motor imagery (MI) classification is a fundamental, yet challenging task due to the variation of signals between individuals i.e., inter-subject variability. Previous approaches try to mitigate this using task-specific (TS) EEG signals from the target subject in training. However, recording TS EEG signals requires time and limits its applicability in various fields. In contrast, resting state (RS) EEG signals are a viable alternative due to ease of acquisition with rich subject information. In this paper, we propose a novel subject-adaptive transfer learning strategy that utilizes RS EEG signals to adapt models on unseen subject data. Specifically, we disentangle extracted features into task- and subject-dependent features and use them to calibrate RS EEG signals for obtaining task information while preserving subject characteristics. The calibrated signals are then used to adapt the model to the target subject, enabling the model to simulate processing TS EEG signals of the target subject. The proposed method achieves state-of-the-art accuracy on three public benchmarks, demonstrating the effectiveness of our method in cross-subject EEG MI classification. Our findings highlight the potential of leveraging RS EEG signals to advance practical brain-computer interface systems. The code is available at https://github.com/SionAn/MICCAI2024-ResTL.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0192_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0192_supp.pdf
Link to the Code Repository
https://github.com/SionAn/MICCAI2024-ResTL
Link to the Dataset(s)
N/A
BibTex
@InProceedings{An_SubjectAdaptive_MICCAI2024,
author = { An, Sion and Kang, Myeongkyun and Kim, Soopil and Chikontwe, Philip and Shen, Li and Park, Sang Hyun},
title = { { Subject-Adaptive Transfer Learning Using Resting State EEG Signals for Cross-Subject EEG Motor Imagery Classification } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15011},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper proposes a subject-adaptive transfer learning strategy that utilizes resting state (RS) EEG signals to adapt models on unseen subject data. The proposed method achieves state-of-the-art accuracy on three public datasets.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This method represents the first attempt to leverage resting-state (RS) EEG signals for model adaptation in EEG motor imagery (MI) classification. The approach calibrates RS EEG signals using the classifier to capture task-dependent features while retaining subject-specific characteristics through the incorporation of feature disentanglement and inverse image synthesis methods from the pretrained classifier. This significantly reduces the need for collecting task-specific (TS) EEG signals from the target subject, as only RS EEG data is required for calibration.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Some details of the experiment may be missing:
- On the BCI IV 2a dataset, most of the results in Table 1 can correspond to Table 1 in [1] (such as EEGNet, CRAM, etc.), both of which are four classification experiments. However, MIN2Net has the same results as Table 3 in [1], which records the results under binary classification. Please confirm the specific tasks performed for each different classification network (binary classification task or quaternary classification task).
- The methods DeepDream (published in 2015) and DeepInversion (published in 2020) compared in the experiment in Table 2 have been published for some time. Is there a more novel method in this field to compare and better demonstrate the effectiveness of the proposed method in this paper. [1] Kwak, Y., Kong, K., Song, W.J., Kim, S.E.: Subject-invariant deep neural networks based on baseline correction for eeg motor imagery bci. IEEE Journal of Biomedical and Health Informatics 27(4), 1801–1812 (2023).
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
It would be easy to reproduce if the authors make their code public.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- In the case of using the RS EEG method, Table 1 lists the results of CRAM-BCM. If the results of CRAM-ResTL can be added for comparison, it will be better to prove the effectiveness of the proposed method in this paper.
- Ablation studies may be needed to further explore the effect of every module.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is novel and the method can be tested. The dataset used is public, allowing for benchmarking.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This paper proposes a novel approach to EEG motor imagery classification using resting state EEG signals.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Innovative subject-adaptive transfer learning framework using resting state EEG signals to handle inter-subject variability in EEG motor imagery classification, creatively overcoming the limitations of needing task-specific signals.
- Extensive testing was conducted across three EEG datasets, demonstrating improved performance over existing methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The method involves complex processes like feature disentanglement and signal calibration, which may not be easily replicable without substantial computational resources, potentially limiting wider adoption.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
The methodology is novel, but the authors have committed to publishing the code upon acceptance.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
well-written well-motivated well-novel method
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The methodology is novel, but it seems impractical for actual application. Additionally, the reproducibility of the code is difficult to verify as it has not been made available immediately.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The paper presents a novel MI subject-adaptive transfer learning strategy using resting state EEG signals. The extracted features were disentangled into task- and subject-dependent features to calibrate RS EEG signals, which were used to adapt the model to simulate processing TS EEG signals of the target subject. The method achieved state-of-the-art accuracy on three MI dataset.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This work proposed the use of RS EEG signals for model adaptation in EEG MI classification, significantly reducing the workload of collecting TS EEG signals from target subjects. By incorporating feature disentanglement and inversely image synthesizing method from the pretrained classifier, RS EEG signals were calibrated to contain task-dependent features while retaining subject characteristics. The accuracy of the MI classification task is significantly improved by the subject-adaptive transfer Learning strategy.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The RS EEG more reflects the basic state of the brain and the connectivity of the neural network,and there are significant differences in physiological mechanism and expression form between RS EEG and the TS EEG. What are the clear advantages of this strategy based calibrated RS EEG signal over other data enhancement methods that does not rely on physiological signals?
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Do you have any additional comments regarding the paper’s reproducibility?
no
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The paper is well-written and easy to follow. If possible, the author can discuss the stability and consistency of the characteristic information of the subjects in the resting state and the task state.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Technical novelty and result achieved.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
We sincerely appreciate the reviewers for their constructive comments, which have helped us refine our work further. Based on comments, we response to them here and will revise some parts of the manuscript.
(R1) Advantages over other data enhancement methods that does not rely on physiological signals: As shown in Table 2 (first and third rows), the classification performance significantly degraded when the resting state EEG signals were not used. This indicates the subject-specific features are crucial for classifying EEG signals from unseen subjects. However, Existing EEG data enhancement methods [12, 22] typically augment the training set using random noise, which lacks specific subject information. Thus, the methods perform poorly in classifying EEG signals from new unseen subjects due to the absence of subject-specific data. In contrast, ResTL preserves subject characteristics from RS EEG signals, enabling the model to individual subjects more effectively. (R1) Stability and consistency of the characteristics in resting and task states: Figure 2 shows the distribution of the task and subject features from the same session of the same subject via tSNE visualization. This visualization demonstrates the task- and subject-dependent features remain stable and consistent, ensuring reliable classification. However, EEG signals from the same subject can exhibit slight variations over time i.e., inter-session variability [a].” [a] Liu, W., Guo, C., & Gao, C. (2024). A cross-session motor imagery classification method based on Riemannian geometry and deep domain adaptation. Expert Systems with Applications, 237, 121612. (R3) Some details of the experiment: We conducted a 4-class classification on the BCI IV-2a dataset and a 2-class classification on BCI IV-2b and OpenBMI datasets. Upon careful review, we identified the mistake of MIN2Net on the BCI IV-2a dataset. We reproduced the results and updated Table 1 to reflect a performance of 53.58 +- 6.96%. (R3) Recent calibration methods: To the best of our knowledge, there is no recent work we further compare with ResTL. Some generative models, such as GAN, might be adapted to compare with ResTL by using RS EEG signals as input instead of random noise. However, these models would require independent training and a novel method to preserve subject characteristics from RS EEG signals. (R3) CRAM-ResTL: We reproduced CRAM using the official implementation but encountered unstable training issues, which resulted in poor performance. (R3) Ablation study for each module: The task encoder is essential to classify EEG signal and calibrate resting state (RS) EEG signal. We conducted an ablation study focusing solely on subject-dependent features. As shown in Table 2, omitting these features resulted in degrading classification accuracy. (R4) Limitations on real-world application and wider adoption: We respectfully disagree with the limitations raised by the reviewers. ResTL is actually more suitable for real-world applications and diverse tasks compared to existing methods based on task-state (TS) EEG signals, such as domain adaptation and transfer learning, since it only requires resting-state (RS) EEG signals for model adaptation. Collecting TS EEG signals is time-consuming and demands more effort from subjects as the number of classes increases, whereas ResTL circumvents these issues. Moreover, once the calibration and adaptation processes are performed for a target subject, only light computational resources are needed. Therefore, we believe that ResTL is highly appropriate for real-world applications, facilitating its use with new subjects. Furthermore, ResTL is a model-agnostic method for EEG classification, capable of integrating existing EEG encoders with task- and subject-specific encoders. Additionally, ResTL has the potential to classify other signals characterized by significant inter-subject variability. These features enhance ResTL’s applicability across a wide range of tasks.
Meta-Review
Meta-review not available, early accepted paper.