Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Non-contact pulse wave reconstruction hinges on the precise recovery of waveform morphology, including the dicrotic notch. Conventional RGB-based methods, which extract physiological signals from recorded facial videos, are constrained by the integral imaging mechanism of standard cameras, where the exposure process induces a smoothing effect that attenuates subtle vascular pulsation details. Conversely, neuromorphic event cameras, while offering exceptional sensitivity to intensity fluctuations, are inherently susceptible to noise and artifacts induced by minor motion. To exploit the synergy between frame-based integration and event-based differential sensing, we propose a novel multimodal network named Fusion-E2Pulse. This framework utilizes filtered RGB signals as structural priors to suppress motion artifacts, while leveraging the high-sensitivity of event streams to recover fine-grained morphological details. Experimental results demonstrate that Fusion-E2Pulse achieves compelling performance, effectively balancing noise suppression and morphological fidelity; under the subject-independent split (w/ ICA prior), it achieves a mean absolute error of 0.73 bpm for heart rate estimation, a waveform correlation of up to 0.919, and a diastolic phase duration error of 22.18 ms, supporting its ability to preserve fine-grained physiological waveform morphology.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/4165_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/fengqianiqan/Fusion_E2Pulse.git

Link to the Dataset(s)

https://doi.org/10.5281/zenodo.18765701

BibTex

@InProceedings{FenQia_FusionE2Pulse_MICCAI2026,
        author = { Feng, Qian AND Guo, Hao AND Niu, Yan AND Xu, Zhenhuan AND Li, Yidi},
        title = { { Fusion-E2Pulse: A Multimodal Event-RGB Fusion Network for Non-contact Pulse Wave Reconstruction } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16896},
        month = {September},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper
    • Propose a novel event-RGB multimodal fusion paradigm for non-contact pulse wave reconstruction, which complementary solves the inherent limitations of single-modal methods (RGB over-smoothing, event noise vulnerability).
    • Design an attentional fusion bottleneck (Squeeze-and-Excitation + Self-Attention) to adaptively calibrate the weights of dual-modal features, avoiding early fusion interference.
    • Construct a time-frequency adversarial supervision framework integrating WGAN-GP, Soft-DTW morphological loss, and STFT spectral loss to guarantee periodic consistency and waveform fidelity.
    • Achieve new SOTA results on the EMPD dataset, enabling precise recovery of pathological biomarkers (e. g. , dicrotic notch) with low heart rate error and high waveform correlation.
  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Well-motivated fusion logic The work accurately exploits the physical complementarity between event cameras (high sensitivity to subtle pulsations) and RGB cameras (stable structural periodicity). The design directly targets the core bottlenecks of single-modal non-contact pulse monitoring, with clear clinical and technical value.
    • Sound network architecture The dual-stream encoder avoids premature modal interference; the attentional fusion bottleneck adaptively balances noise suppression and detail preservation, which is well-suited for physiological signal characteristics.
    • Innovative time-frequency adversarial supervision Different from conventional time-domain GANs, the frequency-domain discriminator focuses on physiological periodicity, and the composite loss effectively optimizes both heart rate accuracy and waveform morphology. Ablation studies verify the necessity of each loss component.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • Unresolved cross-site timing bias RGB is captured from the face while event streams are from the wrist, introducing Pulse Transit Time (PTT) error. The paper does not model or calibrate this systematic bias, which affects morphological accuracy.
    • Insufficient generalization validation Experiments are only conducted on the EMPD dataset. No cross-dataset testing, no evaluation under challenging conditions (large motion, varying illumination, different camera hardware), so the clinical robustness is unproven.
    • Lack of real-time deployment analysis Inference speed, computational complexity (FLOPs/Params), and edge-device feasibility are not reported, which is critical for clinical real-time monitoring applications.
    • Missing comparison with SOTA baselines The paper only conducts comparisons with traditional rPPG algorithms (CHROM, ICA, POS) and a simple event-only network, but does not compare with any advanced deep learning-based methods for rPPG or pulse wave reconstruction (e. g. , PhysNet, RhythmFormer, PulseGAN).
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    Expand related work and discuss recent SOTA works on event camera-based physiological measurement and rPPG waveform reconstruction.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The fusion paradigm is well-motivated and technically sound; the time-frequency adversarial supervision is innovative; experiments are comprehensive. This is a solid, well-executed work with clear novelty and practical value. It meets the acceptance criteria of MICCAI. Minor revisions are recommended to address the weaknesses above.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper proposes Fusion-E2Pulse, a multimodal framework for non-contact pulse wave reconstruction that combines RGB video and event-camera signals to recover physiologically meaningful waveform morphology. The method is motivated by the complementary sensing properties of the two modalities: RGB signals provide smoother and more stable low-frequency structural information, while event streams are more sensitive to subtle intensity fluctuations and therefore potentially preserve fine-grained pulse morphology, but are also more vulnerable to motion-induced noise. To exploit this complementarity, the paper introduces a two-stage pipeline consisting of multimodal data preparation and adversarial pulse reconstruction. The reconstruction model uses dual encoders for RGB and event inputs, an attentional fusion bottleneck with SE and self-attention modules, a decoder with skip connections, and a discriminator operating in the frequency domain. Training combines adversarial, morphological, and spectral objectives. Experiments on the EMPD dataset evaluate both heart-rate estimation and waveform morphology under time-split and record-split protocols, with comparisons against event-only and RGB-derived baselines. The main contribution is a multimodal fusion framework aimed not only at HR estimation but also at reconstruction of fine-grained pulse waveform landmarks such as the dicrotic notch.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Well-motivated multimodal formulation: The paper addresses a meaningful limitation of standard camera-based pulse reconstruction by arguing that RGB sensing smooths subtle pulsatile dynamics, while event cameras can capture finer changes but are more fragile to noise and motion. The proposed fusion of RGB structural priors with event micro-dynamics is intuitive and clinically relevant for waveform reconstruction, especially when the goal goes beyond scalar HR estimation toward morphology recovery.

    Clear architecture and pipeline design: The overall method is easy to follow. Figure 1 on page 3 clearly presents the two-stage design, including preprocessing for both modalities, dual encoders, attentional fusion, decoder, and the frequency-domain discriminator. The roles of the main modules are well separated, and the description of event preprocessing and RGB signal extraction is sufficiently concrete to understand how the final inputs are constructed.

    Good emphasis on waveform morphology rather than HR alone: A notable strength is that the paper does not reduce the task to heart-rate estimation only. In addition to MAE, RMSE, MAPE, and HR correlation, the authors evaluate waveform correlation, SNR, systolic and diastolic phase duration, and pulse width. This is appropriate for a pulse reconstruction paper whose stated aim is to preserve subtle landmarks such as the dicrotic notch. The waveform examples in Figure 3 and the morphology metrics in Tables 1 and 2 strengthen this aspect of the work.

    Useful ablation over losses and fusion variants: Table 2 provides a reasonably informative ablation over adversarial, spectral, and morphological losses, and also compares different RGB priors, namely CHROM, ICA, and POS. This helps support two central claims: first, that adversarial training alone is insufficient, and second, that multimodal fusion improves over event-only reconstruction.

    Strong empirical performance within the presented setup: Within the EMPD benchmark and the baselines considered, Fusion-E2Pulse appears consistently strong. The best fusion variant achieves lower HR error than the event-only model and far better results than the RGB-only priors, while also improving waveform-level measures. The record-split evaluation is especially useful because it attempts to test generalization across samples rather than only within-record continuity.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The main limitation is that the experimental comparison is too restricted for the scope of the claims. The paper compares against event-only and classical RGB-derived signals such as CHROM, ICA, and POS, but it does not compare against stronger learned rPPG or waveform-reconstruction approaches such as PhysNet, RhythmFormer, PulseGAN, or other recent deep models discussed in the paper itself. Since the manuscript claims state-of-the-art performance, the absence of direct comparison to modern learning-based baselines makes that claim difficult to assess.

    The RGB branch is not learning directly from raw video in the same way as many modern RGB-based baselines. Instead, it uses pre-extracted signals from CHROM, ICA, and POS as structural priors. This is a reasonable design choice, but it also means the comparison against RGB-only methods is somewhat asymmetric. The fusion system benefits from a learned reconstruction model on top of both modalities, whereas the RGB-only baselines are classical signal-processing outputs rather than comparably expressive learned models. This weakens the strength of the empirical conclusion that the proposed approach is broadly superior.

    All experiments are conducted on EMPD, which is valuable and multimodal, but still represents a single benchmark with a specific acquisition setup. Given that the method depends on coordinated RGB, event, and contact-PPG data, external validation is understandably difficult, but the lack of evaluation on another dataset or under clearly distinct conditions makes it hard to judge robustness across devices, skin tones, motion patterns, or illumination regimes.

    The paper combines several familiar ingredients: dual-stream encoding, attention-based fusion, adversarial learning, Soft-DTW, STFT-based loss, and classical rPPG priors. The overall combination is sensible and appears effective, but the architectural novelty at the module level is limited. The main contribution is better characterized as a careful multimodal integration of existing ideas for a less-explored sensing setup than as a fundamentally new modeling framework. Important implementation and protocol details could be clearer. The paper gives the main optimizer settings and loss weights, but several details remain underspecified for full assessment and reproducibility. For example, the exact event aggregation strategy, ROI localization robustness, preprocessing sensitivity, sequence lengths used during training, and details of how the discriminator input is normalized or parameterized are not fully elaborated. Since the method relies heavily on preprocessing and modality synchronization, more detail would improve confidence in reproducibility.

    The paper is motivated by cardiovascular screening and waveform morphology, but the experiments stop at signal-level reconstruction metrics. There is no downstream validation showing that the reconstructed waveforms improve clinically meaningful tasks such as disease stratification, arterial stiffness assessment, or reliable dicrotic-notch localization under realistic deployment conditions. This does not invalidate the work, but it does mean the clinical relevance remains indirect.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    This is a well-motivated and technically coherent paper on an interesting sensing setup. I particularly appreciate that the work focuses on waveform morphology rather than only HR estimation, and that the evaluation includes both temporal and record-level split protocols. The main way to strengthen the paper would be to broaden the baseline suite. In particular, the manuscript would benefit from comparison against stronger learned RGB-based and pulse-reconstruction methods, not only classical CHROM, ICA, and POS priors. It would also help to clarify the fairness of comparison, since the proposed system uses a learned fusion and reconstruction model while the RGB-only counterparts are classical extracted signals rather than end-to-end learned models. Finally, a more detailed discussion of robustness to motion, illumination, and sensor variation would strengthen the practical impact of the work.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    My recommendation is based on the fact that the paper presents a meaningful and well-executed multimodal approach to a relevant problem, namely high-fidelity non-contact pulse reconstruction from complementary event and RGB sensing. The motivation is sound, the system design is clear, and the experiments show consistent gains over the baselines that are included. I also view the emphasis on waveform morphology as a positive aspect, since this better matches the medical value of pulse-wave analysis than HR estimation alone.

    At the same time, I do not consider the current empirical evidence fully sufficient for a stronger acceptance recommendation. The baseline suite is too narrow, especially given the state-of-the-art claim, and the novelty lies more in thoughtful multimodal integration than in a clearly new modeling principle. The reliance on a single dataset also limits how strongly one can assess robustness and generalizability. Overall, I think the paper is solid and potentially useful, but it would be notably stronger with broader learned baselines and clearer positioning relative to recent deep rPPG and waveform-generation methods.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #3

  • Please describe the contribution of the paper

    This work proposes Fusion-E2Pulse, a multimodal deep learning approach that fuses signals from an RGB and an event camera for remote photoplethysmography (rPPG). The approach consists of two major stages conducted in series. First, multimodal data preparation, which involves extracting pulsatile waveforms from the RGB and event cameras independently using classical techniques. Second, adversarial pulse reconstruction, which amounts to generating a denoised pulsatile signal by combining the two extracted signals using a deep fusion network-style architecture. The proposed method is evaluated on the EMPD dataset, demonstrating minor improvements over the event-only baseline.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Novelty: the proposed Fusion-E2Pulse architecture is novel, as the literature contains no other works that specifically fuse RGB and event camera information for rPPG. This is only possible because of the EMPD dataset, which this work utilizes, and whose architecture the proposed method is largely based upon [1].
    • Strong evaluation protocol: the work uses a comprehensive suite of evaluation metrics for both HR estimation and pulsatile signal morphology and provides useful baseline comparisons, enabling rigorous comparison with related work.
    • Thorough ablation study: the authors conduct a strong ablation study (sec 3.3) that helps understand the impact of the different loss components in the final proposed training objective. References: [1] Feng, Q., Li, P., Gao, R., Jiale, X., Gong, R., Li, Y.: Empd: An event-based multimodal physiological dataset for remote pulse wave detection (part 1 of 9) (Feb 2026). https://doi.org/10.5281/zenodo.18765701
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • Incremental novelty: while the proposed method is novel, it is largely identical to the architecture proposed by the original work that developed the EMPD dataset [1]. In particular, the proposed event preprocessing is identical to the event aggregation module of the original work, and the adversarial pulse reconstruction stage largely resembles the waveform reconstruction module of the original work.
    • Incremental/insignificant results: the experimental results suggest that the proposed fusion method offers minor gains in performance in comparison to the already very strong event-only baseline, which resembles the method proposed in the original EMPD dataset work [1]. Table 1: HR MAE is improved by 0.1-0.2bpm and SNR is improved by <0.5db, which are clinically insignificant. Figure 3: Bland-Altman analysis reveals that fusion methods do shrink 95% limits of agreement by ~1-1.5bpm References: [1] Feng, Q., Li, P., Gao, R., Jiale, X., Gong, R., Li, Y.: Empd: An event-based multimodal physiological dataset for remote pulse wave detection (part 1 of 9) (Feb 2026). https://doi.org/10.5281/zenodo.18765701
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    Could there be data leakage in the record split experiments? In particular, can the authors please clarify how many subjects in the test set were not seen in the training or validation set? I believe a stronger evaluation protocol would be to partition the dataset at the subject level, i.e., assign each of the 83 subjects to the train, validation, or test set.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper’s methodology is largely similar to the very strong event-only baseline proposed in the original EMPD work, with minor modifications to incorporate RGB signal fusion. Nevertheless, the experimental results suggest that introducing this RGB fusion only improves rPPG performance incrementally (HR MAE by ~1 bpm and SNR by ~0.5 dB). Therefore, my initial score for this work is a “weak reject”.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Author Feedback

We thank the Meta-Reviewer (MR) and all reviewers for their constructive feedback. We are encouraged that MR recognized our motivation and emphasis on waveform morphology. Below we address the “decisive concerns.” 1.Clarification on Data Partitioning and Robustness (MR, R3) To address the MR’s concern regarding potential data leakage and to clarify the inaccuracies in the assessment of our framework’s robustness, we provide a detailed statistical verification of the model’s performance under a protocol where the training, validation, and test sets are strictly non-overlapping at the subject level. By presenting the metrics under this strictly subject-disjoint partition (58 training, 8 validation, 17 testing), we clarify that the core findings are not artifacts of the record-level split. In this subject-independent view, the architecture maintains a Mean Absolute Error (MAE) of 0.73 bpm, a Pearson correlation coefficient ($r_w$) of 0.919, and a Systolic Peak Duration (SPD) error of 17.56 ms. This clarification verifies that the proposed fusion mechanism’s performance, as reported in the manuscript, stems from its genuine generalization capability across unseen individuals rather than data overlap. 2.Comparison with Stronger Learned Baselines (MR, R1, R2) We acknowledge the broad use of “SOTA” and will adopt more precise terms like “advanced” in the final version to better reflect our method’s standing within the specialized multimodal Event-RGB fusion paradigm. The baselines (CHROM, POS, ICA) were selected as physiological structural priors for our fusion framework rather than for direct competition with all frame-based deep learning methods. Given the unique sensing advantages of asynchronous event streams over standard cameras, we focus on the marginal gains in morphological fidelity provided by the fusion mechanism. 3.Novelty vs. EMPD Dataset Paper (MR, R3) We clarify the distinction between this submission and the foundational EMPD work. The EMPD paper focuses on the dataset release and a baseline for HR estimation. This work (Fusion-E2Pulse) addresses the much more challenging task of high-fidelity waveform reconstruction. The technical novelty lies in: 1) The Attentional Fusion Bottleneck, which adaptively calibrates dual-modal weights to suppress event noise while preserving RGB structural priors. 2) The time-frequency adversarial framework using Soft-DTW, which is to enable the recovery of subtle biomarkers like the dicrotic notch. As shown in our new ablation results, the full model outperforms single-modal methods in waveform correlation. 4.Single Dataset and Incremental Gains (MR, R3) Due to unique hardware requirements, EMPD is currently the only available benchmark for this modality. However, our rigorous subject-independent evaluation mitigates this limitation. Regarding “incremental gains” (R3), we acknowledge the observation regarding the modest numerical improvement in HR MAE. However, we emphasize that Fusion-E2Pulse is designed for waveform reconstruction rather than simple HR tracking. While single-modal methods can estimate pulse frequency, they often fail to recover fine-grained morphological landmarks. Our fusion paradigm achieves higher waveform correlation, which is essential for advanced pulse wave analysis. Finally, we clarify that the camera-ready version will explicitly acknowledge current limitations, including the single-dataset dependency and the necessity for further clinical validation, to ensure a balanced and rigorous presentation of our findings.



Meta-Review

Meta-review #1

  • Your recommendation

    Provisional Accept

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This is a technically coherent and potentially useful methodological paper on multimodal non-contact pulse wave reconstruction, but the current evidence leaves the submission in a genuinely borderline position. The paper’s main strengths are the clear motivation for combining RGB structural priors with event-camera dynamics, an architecture that is consistent with that motivation, and an evaluation that appropriately considers waveform morphology rather than reducing the task to heart-rate estimation alone. The ablation study is also helpful in showing the contribution of the loss components and the effect of different RGB priors. Within the presented setup, the method consistently improves over the included single-modal baselines. The decisive concerns relate to the scope and interpretation of the empirical evidence. After checking the manuscript, the baseline suite is narrower than the broad state-of-the-art claim would require, since the comparisons do not include stronger learned rPPG or waveform-reconstruction baselines. In addition, all experiments are conducted on a single dataset. Most importantly, the manuscript defines the record split at the record level but later interprets it as evidence of robustness to unseen individuals; this point needs clarification because the subject-disjointness of that split is not established in the paper as written. One review also appears to overstate the weakness of the empirical gains and to attach unsupported significance language to them; accordingly, I assign limited weight to those aspects of that criticism in the overall assessment.



back to top