Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Foundation models offer a promising paradigm for Electroencephalography (EEG) analysis, leveraging generalizable representations from vast unlabeled datasets. Yet, Transformer-based architectures face a critical bottleneck: global attention mechanisms couple the attention memory state to the signal duration, causing memory overflow during continuous monitoring. To address this, we introduce S-CEReBrO (Streaming CEReBrO), an evolution of the CEReBrO architecture designed for continuous monitoring. Our novel Windowed Alternating Attention mechanism factorizes attention computation into fixed-size spatiotemporal windows, guaranteeing constant KV cache memory as only the active window requires resident attention maps. Empirical scaling analysis confirms that windowed alternating attention can process signals 100× longer than full self-attention and 3×longer than low-rank linear attention. Compared to low-rank linear attention on long contexts, windowed alternating attention requires 55% of the memory while increasing inference throughput by 2.1×. Pre-trained on >25,000 hours of recordings from >12,000 subjects, S-CEReBrO achieves state-of-the-art performance on 7 of 11 downstream tasks, with up to 60% fewer parameters. This work represents a significant step toward the realization of efficient, generalizable, and continuous EEG monitoring. An accompanying code repository is available.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/6624_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/pulp-bio/biofoundation

Link to the Dataset(s)

CHB-MIT: https://physionet.org/content/chbmit/1.0.0/ Neonate: https://zenodo.org/records/1280684 PhysioNet-MI: https://www.physionet.org/content/eegmmidb/1.0.0/ STEW: https://ieee-dataport.org/open-access/stew-simultaneous-task-eeg-workload-dataset ISRUC: https://sleeptight.isr.uc.pt/?page_id=48 Mumtaz: https://figshare.com/articles/dataset/EEG_Data_New/4 TUEG: https://isip.piconepress.com/projects/tuh_eeg/ SEED: https://bcmi.sjtu.edu.cn/home/seed/

BibTex

@InProceedings{BucGle_SCEReBrO_MICCAI2026,
        author = { Bucagu, Glenn Anta AND Ingolfsson, Thorir Mar AND Li, Yawei AND Benini, Luca},
        title = { { S-CEReBrO: Breaking the Memory Barrier in Continuous EEG Monitoring } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16894},
        month = {September},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper addresses a fundamental limitation of Transformer-based EEG foundation models: their inability to scale to long-duration, continuous monitoring due to memory constraints. In standard architectures, the key–value (KV) cache grows with the length of the input signal, making long-term deployment impractical, especially in real-world settings such as wearable or bedside monitoring. To overcome this, the authors propose S-CEReBrO, a streaming EEG foundation model built around a novel attention mechanism called Windowed Alternating Attention (WAA). The core idea is to enforce strict locality in both time and space by restricting attention to fixed-size windows, while alternating between temporal and spatial attention layers. This design ensures that only a small, bounded subset of tokens needs to be retained in memory at any time, resulting in a constant KV cache memory footprint regardless of signal duration. Importantly, the model is not simply a theoretical construct but is designed explicitly for streaming scenarios. By incorporating a FIFO-style buffer that evicts stale tokens, S-CEReBrO maintains a continuous and stateful representation of the signal without requiring full-history storage. This allows the model to process EEG recordings of several hours in duration, far exceeding the limits of conventional self-attention models. Beyond the architectural contribution, the paper also presents a large-scale pretrained EEG foundation model trained on over 25,000 hours of data. Through extensive evaluation on 11 downstream tasks, the authors show that their approach not only improves efficiency but also achieves state-of-the-art performance on the majority of benchmarks, often with fewer parameters than competing methods. Taken together, the work represents a meaningful step toward practical, real-time EEG foundation models.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    One of the most compelling strengths of the paper is its clear focus on a real and pressing systems-level problem: the incompatibility of Transformer architectures with continuous, long-duration EEG monitoring. Rather than proposing incremental performance improvements on short benchmarks, the authors tackle a constraint that directly limits real-world deployment. This grounding in practical constraints significantly strengthens the contribution. The proposed Windowed Alternating Attention mechanism is a well-motivated and carefully designed solution to this problem. While locality-based attention has been explored in other domains, the specific formulation here, which combines temporal and spatial windowing with alternating layers, dilation, and shifting, demonstrates a thoughtful adaptation to the unique structure of EEG data. The mechanism balances efficiency and expressivity by maintaining local attention while still enabling indirect long-range interactions through layer-wise propagation. The additional design choice of enforcing constant KV memory through explicit token eviction further distinguishes the approach from many prior efficient attention methods. Another notable strength is the integration of algorithmic innovation with system-level design. The use of streaming via FIFO buffering is not merely an implementation detail but a core part of the model’s functionality. This allows S-CEReBrO to operate continuously without artificial segmentation, preserving temporal coherence in a way that many prior approaches do not. The accompanying scaling experiments provide convincing evidence of the practical benefits, showing substantial improvements in both memory usage and inference speed, as well as the ability to handle significantly longer signals. The experimental evaluation is broad and comprehensive, covering 11 downstream tasks that span clinical, cognitive, and BCI applications. This diversity strengthens the claim that the learned representations are generalizable. Moreover, the model achieves state-of-the-art performance on 7 of these tasks despite having a relatively small parameter count. This highlights an important insight: that carefully designed inductive biases and efficient attention mechanisms can outperform larger models relying on global attention. The paper also benefits from large-scale pretraining on a diverse EEG corpus, which is essential for foundation model performance. The inclusion of heterogeneous datasets with varying channel configurations and recording conditions supports robustness and generalization, and the use of masked autoencoding aligns with successful strategies in related domains. Finally, the work offers an implicit but important conceptual contribution: it challenges the assumption that global attention is necessary for high-quality EEG modeling. By demonstrating strong performance with localized attention, the paper suggests that exploiting the inherent spatiotemporal structure of EEG signals may be more effective than relying on fully global interactions.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    Despite its strengths, the paper’s novelty is somewhat limited when viewed in the broader context of efficient Transformer research. Many of the core ideas underlying the proposed method, such as windowed attention, locality constraints, and separating spatial and temporal attention—have been explored in prior work. For example, sliding window attention has been used in models like Longformer, and alternating spatial-temporal attention is already present in the original CEReBrO architecture. As a result, the primary contribution lies more in the combination and adaptation of existing ideas rather than the introduction of a fundamentally new mechanism. While this integration is well executed, the paper could more clearly articulate what aspects are truly novel and why they are uniquely suited to EEG. Another limitation is the lack of comparison with a broader set of long-context attention methods. The evaluation focuses on a subset of EEG foundation models and baselines, but omits several influential approaches from the efficient attention literature, such as Performer or Longformer-style architectures. Without these comparisons, it is difficult to determine whether the observed gains are specific to the proposed design or simply reflect the general advantages of locality and reduced memory usage. The paper would also benefit from more extensive ablation studies. Key components of the model, such as window sizes, dilation factors, and the alternating structure, are introduced without a detailed analysis of their individual contributions. This makes it challenging to assess which elements are essential and whether simpler variants could achieve similar performance. In particular, understanding the trade-offs between window size and performance would be valuable, especially given the central role of locality in the model. A related concern is the limited investigation of long-range dependency modeling. While the authors argue that long-range interactions can be captured indirectly through the alternating attention structure, this claim is not rigorously validated. There is little analysis of scenarios where global context might be important, nor any controlled experiments demonstrating the model’s ability to capture dependencies over long time horizons. This leaves open the question of whether the efficiency gains come at the cost of missing important temporal relationships. From an empirical standpoint, although the model achieves state-of-the-art results on several tasks, the performance improvements are not uniformly strong. In some cases, the gains over existing methods are modest, and on at least one task the model underperforms simpler baselines. This suggests that the benefits of the proposed approach may be task-dependent, and the paper could provide a more nuanced discussion of where and why the model excels. Finally, there are some reproducibility concerns. Certain references are anonymized or incomplete, and the paper provides limited details about training procedures, hyperparameter tuning, and implementation specifics. The absence of code or a clear statement about availability further complicates the ability of others to validate and build upon the work.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper addresses a critical memory bottleneck in Transformer-based EEG foundation models for long-duration monitoring. While the engineering solution is robust, it has several limiting factors:

    • Limited conceptual novelty: The “Windowed Alternating Attention” (WAA) mechanism primarily combines and adapts existing concepts, such as sliding window attention and alternating spatial-temporal layers, rather than introducing a fundamentally new primitive.
    • Empirical gaps: The evaluation omits comparisons with broader efficient attention architectures like Performer or Longformer, making it difficult to isolate the specific benefits of the WAA design.
    • Unverified dependencies: The authors claim WAA captures long-range context through layer-wise propagation, but the paper lacks rigorous validation or stress tests to prove that essential temporal relationships are preserved.
    • Insufficient analysis: The manuscript lacks detailed ablation studies for key hyperparameters, such as window size and dilation, leaving the individual contributions of these components unclear.
    • Reproducibility concerns: The absence of public code and the use of anonymized or incomplete references hinder the community’s ability to validate and build upon the work. Summary: Despite its practical utility for 8-hour monitoring, the paper relies more on system design than conceptual innovation. Without proof that it captures global context better than simpler alternatives.
  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Reject

  • [Post rebuttal] Please justify your final decision from above.

    Although S-CEReBrO successfully solves the memory crisis via a constant O(1) footprint, its stateless sliding window buffer introduces a severe physiological limitation: it is structurally blind to long-term neurological trends. By permanently evicting tokens once they slide out of the active window, the architecture sacrifices the ability to directly link or track slow-evolving clinical conditions (such as gradual shifts in coma recovery, sleep cycle transitions, or long pre-seizure-pre-ictal patterns) that span across hours rather than seconds. A comparative evaluation against stateful models like Mamba or optimized RNNs is therefore critical. Because stateful models compress historical sequence data into a continuous, fixed-size hidden state vector, they prove that an efficient O(1) memory footprint can be achieved without establishing hard window boundaries that completely destroy long-term historical context.

    In Section 3.1, the authors mathematically define tokenization using a window length L and stride S, but they omit the actual numerical value of the stride (S) throughout the text and in Table 2.Because S directly dictates the number of temporal patches (N_p), it fundamentally impacts the model’s reported linear computational complexity of O(CN_p(w_t+w_s)). To ensure reproducibility of the scaling and throughput analysis, the authors must explicitly state the stride size and briefly clarify its significance regarding token overlap and efficiency during continuous streaming.



Review #2

  • Please describe the contribution of the paper

    This paper extends the foundational EEG model CEReBrO (provided in the supplementary material) for streaming applications. The proposed technique is well described and opens new applications needing long and continuous EEG monitoring.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Novel windowing (temporal and spatial/cross-channel) technique for handling streaming data in a trasformer-like architecture for multi-channel signals (such as EEG).

    Demonstrates that faster inference and virtually infinitely long temporal streams (8h) can be processed with significantly reduced resource usage (linear scaling with sequence length) and constant KV cache.

    The sCEReBrO foundational model was trained on a large set of datasets (25k hours, 12k subjects) with a diverse set of EEG montages, showing its adaptability to the various devices used in practice.

    Excellent comparison on a large number of datasets (11) with currently leading techniques on both classification and regression.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The paper is not improving on the performance of the CEReBrO model and the performance is roughly the same.

    In general, the paper clarity is excellent, however, the particularities of adapting the foundational model to the diverse datasets used in evaluation could be better documented (or code provided) to improve reproducibility.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This is an excellent EEG processing technique achieving new SOTA results, and useful especially whenever continuous monitoring is needed.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    This is an interesting technique to address very long contexts in temporal sequences and the authors sufficiently addressed concerns during the rebuttal.



Review #3

  • Please describe the contribution of the paper

    This paper proposes S-CEReBrO, a foundation model for continuous EEG monitoring. The core innovation is the Windowed Alternating Attention mechanism, which restricts temporal and spatial attention within fixed-size local windows, achieving O(CT) linear computational complexity and O(1) constant KV cache memory (relative to signal length 𝑇 T ). The receptive field is expanded without increasing resident memory through dilation and shift factors. Pretrained on >25,000 hours of EEG data from >12,000 subjects, it achieves SOTA in 7 out of 11 downstream tasks with only 2.4M parameters. Supplementary materials provide the complete paper of its predecessor, CEReBrO, including MCU deployment and quantization analysis.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.Precise Problem Formulation with Practical Significance: The linear growth of KV cache with signal length in continuous EEG monitoring is a real engineering bottleneck. This paper is the first work to reduce the resident memory of EEG foundation models to O(1). The comparison in Table 1 clearly demonstrates this unique advantage. 2.Elegant Design of Windowed Alternating Attention: The combination of alternating attention (existing work) with windowed locality (from LLM domain), adapted to EEG’s 2D spatiotemporal structure via dilation/shift factors (rather than simply transplanting 1D text window attention), is a clear and targeted design. Replacing fixed segmentation with FIFO streaming maintains temporal continuity, which is crucial for scenarios like seizure detection. 3.Extremely Comprehensive Experiments: 11 downstream tasks cover clinical (seizure detection, sleep staging, abnormal classification), BCI (motor imagery, emotion recognition), and cognitive monitoring (workload, alertness), spanning 128-1000Hz sampling rates, 6-64 channels, and 4-30s durations. Reporting mean ± standard deviation of 5 random seeds for all results demonstrates excellent statistical rigor. 4.Compelling Scaling Analysis: Fig. 2 clearly illustrates the advantages of windowed alternating attention in memory and inference latency—it can process 30,000s (8h) signals, which is 60× longer than full self-attention and 2× longer than linear attention. It also uses 2.8× less memory and is 3.7× faster in inference. 5.Outstanding Parameter Efficiency: With only 2.4M parameters, it is the smallest among all foundation model baselines (BIOT 3.2M, CBraMod 4.0M, LaBraM 5.8M), yet performs better on most tasks. This is not only an academic contribution but also has significant value for practical deployment. 6.Thoughtful Ablation Design: The supplementary CEReBrO paper includes attention mechanism comparison (Table V), channel embedding strategy comparison (Table VI), pretraining effects (Table VII), and quantization analysis (Fig. 3), clearly attributing contributions to each component. The direct comparison of S-CEReBrO vs CEReBrO in Table 5 proves the additional gains of the windowed mechanism’s local inductive bias in cognitive monitoring tasks. 7.Complete Empirical Evidence for MCU Deployment: The deployment validation on GAP9 with 350ms latency and 10mJ energy consumption verifies a complete closed loop from theory to practice, which is first-of-its-kind in the EEG foundation model domain.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.Insufficient Discussion on Losing Long-Range Dependencies: The paper claims “global attention is often redundant for EEG analysis,” yet S-CEReBrO did not reach SOTA on 4 tasks (SHU-MI, TUAB, ISRUC, Mental Arithmetic). Notably, the gap is significant on Mental Arithmetic (68.97% vs best 75.66%). The paper does not analyze whether these tasks require global temporal dependencies or the limitations of the windowed mechanism in these specific scenarios. 2.Lack of Systematic Analysis on Window Size Selection: wt=5 and ws=7 are fixed hyperparameters, but the paper lacks sensitivity analysis for different window sizes. The optimal window size may vary across tasks and sampling rates. The rationale for choosing dilation factors {1,2, 4} and shift factors {0,1, -1, 2, -2} is also not explained. 3.Insufficient Analysis of Data Distribution Differences: Pretraining is dominated by TUEG clinical data, but downstream tasks include BCI and emotion recognition, which are significantly different. The paper does not analyze how the domain gap affects the performance of different tasks.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper proposes an elegant solution to the practical bottleneck of continuous EEG monitoring (KV cache memory growing linearly with signal length). Windowed Alternating Attention achieves O(1) resident memory, making it the first foundation model in this field to reach this goal. Experiments cover 11 tasks with rigorous statistics (5 seeds mean ± std), achieving SOTA in 7/11 tasks with only 2.4M parameters. Scaling analysis and MCU deployment validate the practical value of the method. The main shortcomings are the lack of window size sensitivity analysis, but the overall contribution is solid, experiments are comprehensive, and writing is clear, meeting the acceptance criteria.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    S-CEReBrO represents an exemplary MICCAI submission that targets a massive roadblock in current clinical workflows: the memory wall of using Transformer foundation models for continuous patient monitoring. The paper is recommended for a clear Accept based on the following:

    • High Clinical and Practical Impact: Transitioning from an memory overhead to an constant KV cache footprint is a practical breakthrough. It transitions continuous ICU or ambulatory EEG tracking from a theoretical concept into something that can run on standard edge medical hardware without crashing.
    • Robust and Honest Rebuttal: The authors handled critiques with objective mathematical definitions, corrected minor reviewer reading errors politely, and provided solid domain reasoning as to why low-sample datasets behave differently under heavy-parameter foundation models.
    • Broad Generalization: Proving stable performance across a gamut of varying window sizes (from 4-second to 10-second segments) across multiple distinct open-source clinical datasets removes concerns regarding architectural overfitting.

    Recommendation for Final Publication: The authors must ensure that the structural definitions of their bounded space-time neighborhoods and the mathematical constraints yielding the cache are prominently detailed in Section 3.Additionally, the contextual limitation regarding sample-starved datasets (like Arithmetic) should be added to the final discussion to guide future clinical fine-tuning tracks.



Author Feedback

We thank the reviewers for their feedback. We itemise our responses. 1) Distinction from Longformer, Performer, & FIFO. EEG decoding is bidirectional: tasks generally require past+future context around each prediction. WAA’s core property is that the active-token neighborhood is bounded by design: every query at (c,t) attends only to N_t and N_s (Eq. 3,5) independently of T. The KV cache only holds tokens inside these neighborhoods, yielding O(1) memory. FIFO implements this: as new tokens stream in, those outside every future query’s neighborhood are evicted. Information from evicted tokens is preserved through depth, because WSA mixes channels at each timestep before eviction, and alternating WSA/WTA layers propagate features forward (Sec. 3.2-3). Competing methods lack both space-time prior and O(1) KV cache memory under bidirectional inference: Longformer’s 1D windows break EEG structure, and the bidirectional form needs an O(T) KV cache. Performer uses content-agnostic random features with no spatial/temporal prior, and its bidirectional kernel sum materializes the entire sequence (O(T)). WAA is the only mechanism combining O(1) KV memory with an EEG-aware spatiotemporal prior under bidirectional decoding (Table 1). 2) Long-range dependencies & benchmark scope. Current EEG datasets operate on second-scale segments; minute/hour-scale labeled benchmarks are not publicly established. Our paper shows that locality can improve performance: S-CEReBrO leads at long and short segments alike: CHB-MIT (10s), SEED-VIG (8s), Neonate (5s), STEW (4s) (Table 4). The scaling claim (Fig. 2) is a capability result: WAA is the only mechanism running at hour-scale, so no baseline comparison is currently possible. Establishing such benchmarks is important future work our architecture enables. 3) Non-SOTA tasks & window/dilation rationale. A single configuration across tasks is a core goal; per-task tuning would contradict the foundation model premise. w_s=7 covers standard 10/20 montage neighborhoods and w_t=5 is the smallest window whose cascaded receptive field over the 3 WTA layers covers the 30s pre-training context; shifts {0,±1,±2} tile contiguous receptive fields across dilations {1,2,4} without gaps (Sec. 3.2-3.3, Table 2). Of the 4 non-leading tasks, TUAB (−0.06 AUROC vs CBraMod), ISRUC (−0.70 F1 vs CBraMod) and Mumtaz (−0.78 AUROC vs BIOT) are within one std. Mental Arithmetic shows the largest gap (6.69 AUROC), but its 1,707 samples are an order of magnitude smaller than other benchmarks and the leader is a small supervised model, indicating sample efficiency rather than receptive field is the bottleneck. We respectfully note that R3 lists SHU-MI as non-leading, but S-CEReBrO leads SHU-MI (66.48 vs BIOT 66.16). 4) Reproducibility. All datasets are public and cited. Anonymized reference [2] is our prior work under review at a different venue. We detail the architecture and hyperparameters in our manuscript and will release code upon publication. 5) Difference from CEReBrO. CEReBrO uses global alternating attention with O(T) resident memory (Table 1), making 8-hour streaming infeasible. S-CEReBrO introduces bounded space-time neighborhoods, achieving O(1) KV cache and 6× longer signal capacity (Sec 3.4,Fig. 2). This is an architectural change, not tuning. S-CEReBrO’s gains over CEReBrO are notable (Table 5): on STEW +3.14% Balanced Acc, +3.10% Weighted F1, +0.0213 Cohen κ; on SEED-VIG, -0.0114 RMSE and +0.0209 Pearson. 6) Pre-training/downstream domain gap. We analyze how the gap affects each task in Table 4.Grouping tasks by distance from TUEG: clinical tasks close to TUEG (TUAB, Mumtaz, CHB-MIT) yield leads or within-std results; BCI tasks (PhysioNet-MI, SHU-MI) yield leads or within-std results; affective/state tasks (SEED-V, SEED-VIG, STEW) all lead; the pediatric task (Neonate) leads. S-CEReBrO retains performance on distant tasks. This robustness comes from the channel embeddings (Sec. 3.1), which handle different montages.



Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The paper presents promising results and can help with more clarifications on how the FIFO mechanism works in the proposed structure, the choice of window sizes and further discussion on why the performance gains are limited on some of the tasks.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The authors have addressed the concerns. In the final revisions It is recommended to add structural definitions of the bounded space-time neighborhoods and the mathematical constraints yielding the cache are added in Section 3, also add discussion on the limitation of the sample starved dataset.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    Two reviewers independently assigned the high score (5) and R2 maintained this after reading the rebuttal. The rebuttal convincingly distinguished WAA from Longformer (O(T) KV cache under bidirectional decoding, incompatible with EEG spatial structure) and Performer (content-agnostic, O(T) materialization), addressing the sole dissenter’s main concern about novelty. The O(1) KV cache achievement -the O(1) memory result is practically important for the field-, SOTA on 7/11 tasks with “only” 2.4M parameters, large-scale pretraining on 25k hours of diverse EEG, and MCU deployment validation form a comprehensive and practically significant package.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The reviewers raised concerns regarding the novelty of the proposed attention mechanism, the ability of the model to capture long-range dependencies, the rationale behind the selected windowing parameters, and the limited comparison with other efficient attention architectures. The rebuttal addressed these concerns satisfactorily by clarifying the FIFO streaming mechanism, the bounded space-time neighborhood design, the relationship to existing efficient attention methods, and the rationale for the selected window and dilation configurations.

    The paper addresses an important practical limitation of current EEG foundation models by enabling continuous long-duration monitoring with constant KV-cache memory requirements. The proposed architecture demonstrates strong scalability characteristics while maintaining competitive or state-of-the-art performance across a broad set of downstream tasks. Although questions regarding extremely long-range temporal dependencies remain valid limitations, the current experimental evidence supports the effectiveness of the proposed design for realistic EEG monitoring scenarios.



back to top