Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

High-quality paired medical image-mask data are scarce due to privacy constraints and the cost of expert annotation. While joint generative modeling offers a scalable solution for paired image-mask augmentation, existing methods often couple structure and appearance, leading to degraded region-of-interest (ROI) geometry, boundary inconsistency, and synthetic-to-real distribution shift. We propose Dual-Stream Diffusion (DSD), a joint generative framework that models paired data with two decoupled denoising streams: a structure stream for mask prediction and an appearance stream for image synthesis, thereby mitigating optimization conflict. To further preserve image-mask coherence, DSD introduces two structure-aware modules: Time-Gated Fusion provides progressive structural guidance during denoising, while Structure-Aware Heads enforce boundary-aligned prediction with improved image fidelity. We evaluate DSD on three public datasets (Kvasir-SEG, ISIC 2016, and 3D-IRCADb-01) and compare against competitive generation baselines. Across all datasets, DSD consistently improves both generation quality and image-mask consistency, demonstrating robust paired synthesis for data augmentation in medical image segmentation.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/4375_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{LiYu_DualStream_MICCAI2026,
        author = { Li, Yu AND Liu, Jiaqing AND Jain, Rahul Kumar AND Zhang, Chujie AND Chen, Yen-Wei},
        title = { { Dual-Stream Diffusion with Structure-Aware Fusion for Joint Synthesis of Coherent Medical Image-Mask Pairs } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16890},
        month = {September},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper proposes a joint generative framework that models coherent image-mask pairs with decoupled structure and appearance streams.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The overall presentation is clear, aside from a few minor details, and the experimental evaluation is sufficiently extensive.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The proposed method models the interaction from structure to appearance in both the time-gated fusion module and the structure-aware heads. It would be interesting to see how the network performs when considering the opposite direction of interaction, or a bidirectional interaction scheme. The authors may comment on this design choice. 2.Regarding the evaluation protocol, particularly for pair consistency, it is somewhat unclear how Dice and IoU directly reflect pair consistency. 3.There are a few issues in the presentation.

    • There appear to be typos, including T_{A}i in Section 2.2
    • The description stating that “l denotes the low-frequency branch” does not seem to match Figure 1.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall, this paper presents a well-motivated generative framework for modeling coherent image-mask pairs, and the empirical evaluation is fairly comprehensive. Although several aspects of the method and evaluation would benefit from further clarification, these issues appear to be relatively minor and do not substantially undermine the overall contribution.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper proposes Dual-Stream Diffusion (DSD), a joint diffusion framework for generating coherent medical image-mask pairs. The main idea is to decouple structure and appearance into two denoising streams: a structure stream for mask-related geometry and an appearance stream for image synthesis. To improve coherence between generated images and masks, the paper introduces two additional components: (1) Time-Gated Fusion, which injects structure tokens into the appearance stream through cross-attention modulated by a time-dependent gate, and (2) Structure-Aware Heads, which separately predict masks and coarse/high-frequency image components. The method is evaluated on three public datasets spanning endoscopy, dermoscopy, and CT slices, and the paper reports improvements over several recent generation baselines in image fidelity, image-mask consistency, and full-replacement downstream segmentation experiments.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.Well-motivated problem formulation. Generating coherent image-mask pairs is an important and practically relevant problem for segmentation-oriented medical data augmentation. The paper identifies a real challenge in prior work: jointly modeling geometry and appearance in a single stream can lead to boundary misalignment and inconsistent image-mask pairs. This is a meaningful motivation for proposing a more structured generative design. 

    2.Plausible and reasonably clear methodological contribution. The dual-stream decomposition is intuitive: separating structure and appearance is a sensible design choice for paired image-mask generation. The proposed Time-Gated Fusion is also a reasonable mechanism, since stronger structural control early in denoising and relaxed control later is consistent with how one might expect coarse geometry and fine texture to emerge over the diffusion trajectory. The Structure-Aware Heads further align with the paper’s goal of preserving mask-image coherence, especially around ROI boundaries. 

    3.Broader-than-average empirical evaluation. The evaluation is fairly broad, the method is tested on three public datasets from distinct modalities/domains: Kvasir-SEG, ISIC 2016, and 3D-IRCADb-01 (via 2D slices). The paper reports not only fidelity (KID), but also image-mask consistency via a frozen segmentation evaluator and a full-replacement experiment where synthetic data alone are used to train a segmenter tested on real data. This is a stronger evaluation design than simply reporting a generative metric.  4.Consistent quantitative gains over baselines. In Table 1, the proposed method achieves the best non-real-data results across all three datasets for Dice, IoU, and KID. In Table 2, DSD is best among synthetic-data methods on ISIC 2016 and 3D-IRCADb-01, and remains competitive on Kvasir-SEG. These results suggest that the method is not only improving appearance fidelity, but also producing usable labels for downstream segmentation.  5.Ablation is present and directionally supportive. The paper includes an ablation of major components—dual stream, gating, cross-attention, structure-aware heads, and the high-frequency head. While the ablation could be stronger, it does provide some evidence that the full configuration is responsible for the best performance. 

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The innovation is moderate, not particularly strong.

    This paper integrates several common concepts: diffusion-based pairwise generation, semantic/structural decoupling from appearance, feature fusion based on cross-attention mechanisms, and multi-branch prediction heads. The paper itself cites several related pairwise generation methods and diffusion-based medical segmentation/generation work, including DiffAtlas, MedSegFactory, and JiT-style denoising. The contribution of this paper lies more in the clever combination and improvement of existing methods’ architectures than in entirely new formulas.

    2.The baseline comparisons may not be thorough enough.

    This method is compared with SatSynth, DiffGen, DiffAtlas, MSF, and JiT. However, since DSD is specifically designed for image-mask pairwise generation, the comparison results would be more convincing if the paper could more clearly distinguish which improvements come from: dual-stream factorization, which from time-gated structural guidance, which from structure-aware output heads, and which from direct target prediction/JiT-style denoising.

    3.The ablation experiments could be more complete.

    Table 3 only reports KID. Given that the primary scientific motivation of this paper is to improve image-mask consistency, I would expect ablation experiments to also include the Dice/IoU consistency metric.

    4.The 3D aspect is weakly argued.

    One of the three datasets is 3D-IRCADb-01, but the method was not evaluated as a true 3D model on this dataset; instead, the paper extracted 2D axial slices and copied CT images to the three channels. This is acceptable for a broader evaluation, but it weakens the impression that the paper explores volume pairing generation. This is not explicitly emphasized in the title and framing, but readers may still infer a greater cross-dimensional generality than actually presented.

    5.Clinical relevance is indirect.

    While the application area is medical imaging, the paper primarily focuses on synthetic data generation and segmentation enhancement. There are no clinical reader studies, no task-specific clinical endpoints, and no validation of the clinical realism of the generated images beyond KID and segmentation-based surrogate metrics. (However, for a methodological paper, I think this is acceptable.)

    6.Some technical details remain unclear, affecting the full reproducibility of the experiments.

    The paper provides some key settings, such as architecture size, optimizer family, time-step scheduling, and solver steps. However, some details remain ambiguous, including training duration, precise computational budget, selection/adjustment of loss weights outside the default values ​​λm = λx = 1.0, preprocessing differences between different datasets, and whether the baselines compared are reproduced or taken from the original report. The method is more reproducible than many submitted papers, but still lacks detail.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I view this paper favorably because it addresses this important problem through an approach that is both intuitive and well-validated. The dual-stream architecture, time-gated fusion, and structure-aware head collectively form a comprehensive framework for improving image-mask consistency in paired medical synthesis. Unlike many papers in this field, this study evaluates the method across three datasets and demonstrates its performance across multiple dimensions: fidelity, consistency, qualitative examples, t-SNE visualizations, and full-replacement downstream segmentation. Quantitative results consistently outperform previously reported baseline methods. 

    My primary concerns lie in the degree of novelty and the depth of analysis. While the method represents a meaningful architectural optimization, it does not constitute a major conceptual leap compared to previous diffusion-based paired synthesis methods. Furthermore, the scope of the ablation experiments is narrow; only the KID metric is reported, and a comprehensive analysis of individual components using the most important task-specific metrics in the paper is lacking. Finally, given that the research motivation is data augmentation, I would have preferred to see explicit experiments using real data combined with synthetic data augmentation, in addition to the full-replacement setting. 

    Overall, I consider this paper to be above average; it is a credible and useful contribution and is likely to be accepted.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #3

  • Please describe the contribution of the paper

    Contributions: 1.The paper proposes a Dual-Stream Diffusion (DSD) framework for joint medical image-mask synthesis, with separate streams for structure and appearance modeling. 2.It introduces a Time-Gated Fusion module and Structure-Aware Heads in this framework. 3.Experiments on three public datasets suggest that the proposed method improves generation quality, image-mask consistency, and the usefulness of synthetic pairs for downstream segmentation.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The paper addresses a relevant and well-motivated problem. The focus on paired medical image-mask synthesis is meaningful, and the discussion of structure-appearance coupling, boundary inconsistency, and synthetic-to-real shift is well aligned with practical downstream segmentation needs. 2.The method is reasonably well organized and internally coherent. The dual-stream design, time-gated fusion, and structure-aware heads follow a clear line of thought, rather than appearing as a collection of loosely connected modules. 3.The evaluation protocol is stronger than reporting image-generation metrics alone. In addition to KID, the paper includes pair-consistency evaluation and full-replacement segmentation experiments, which makes the empirical study more practically relevant.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.While the paper is thoughtfully designed, the overall methodological novelty feels somewhat incremental. The main contribution seems to lie more in a task-specific combination of existing design patterns than in a distinctly new generative formulation. 2.The ablation study is not fully aligned with the paper’s main claims. The method is motivated primarily by improved image-mask coherence and downstream usefulness, yet Table 3 reports only KID, without showing how each component affects pair consistency or replacement segmentation performance. 3.The paper does not provide enough discussion of fairness in comparison or model efficiency. Since the proposed model uses a dual-stream ViT-M/16 design, it would be helpful to clarify parameter/computation differences versus the baselines, rather than stating only that all methods use the same preprocessing. 4.Some design choices are presented more as reasonable heuristics than as thoroughly validated decisions. For example, the fixed time-gating exponent, the equal loss weights, and the joint training across three datasets are not analyzed in much depth. 5.It would strengthen the empirical study to include comparisons with some non-diffusion baselines as well, such as SAM-based or SAM3-based pipelines could provide a useful additional reference beyond diffusion-oriented methods. 6.In Fig. 3, the t-SNE overlap on ISIC 2016 appears less convincing than on the other two datasets. Could the authors discuss the possible reasons for this behavior, such as larger appearance variability, class-specific texture diversity, or a residual domain gap in the generated samples?

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Please see weaknesses

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Author Feedback

We sincerely thank the reviewers and Chairs for their insightful and constructive feedback on our manuscript and work. We address the main concerns below.

Q: Ablation and pair consistency (Meta, R1, R2, R3). In Tables 1 and 2, Dice and IoU are used as indirect pair-consistency measures: a pretrained segmentation evaluator predicts a mask from each synthetic image and compares it with the paired synthetic mask. Higher Dice and IoU indicate that structure is recoverable and aligned with the paired mask, while mismatches reduce the scores. Tables 1 and 2 assess overall model consistency and downstream utility, whereas Table 3 isolates the effects of individual components on distributional fidelity. This setting makes the role of Table 3 clearer as an analysis of distributional fidelity. Q: Novelty and prior work (Meta, R2, R3). We appreciate the reviewers for raising this important point. Our novelty and contribution lie in integrating structure–appearance decoupling, cross-attention, and dual-stream prediction into a diffusion framework specifically designed to improve image–mask coherence in joint synthesis. Compared with MedSegFactory, DSD differs in both optimization space and interaction design: it performs pixel-space joint synthesis with decoupled image and mask denoising, uses one-way time-gated structure-to-appearance fusion, and employs structure-aware heads to separately predict mask structure, coarse appearance, and high-frequency details for better boundary-aligned generation. Q: Fairness, attribution, and augmentation (Meta, R2, R3). Our baselines were selected for their task relevance to paired image–mask synthesis and segmentation-oriented synthetic data generation, including SatSynth, DiffGen, DiffAtlas, and MedSegFactory. JiT was included as an adapted reference to provide an additional comparison under a joint image–mask synthesis setting. For attribution, Table 3 isolates the contribution of key components, including dual streams, gating, cross-attention, structure-aware heads, and the high-frequency head, while Tables 1 and 2 evaluate overall consistency and downstream utility. We agree that real+synthetic augmentation is important; our full-replacement setting is intended as a stricter test, as the segmentation model is trained only on synthetic pairs and evaluated on real data. Q: Design choices and scope (Meta, R1, R2, R3). The single-direction structure-to-appearance interaction was designed for our task: the structure stream serves as the geometric and label anchor, while the appearance stream is generated conditioned on it. Time-gating follows the same principle, enforcing stronger structural guidance at early denoising steps and allowing more flexible appearance refinement later. We use equal loss weights to avoid dataset-specific tuning. We use 3D-IRCADb-01 only as a paired CT benchmark through axial 2D slices, not as evidence of true 3D generation. Accordingly, we position the clinical relevance of this work as indirect, mainly through segmentation-oriented data augmentation rather than direct clinical deployment. Q: Minor clarifications (R1, R3). The lower overlap on ISIC 2016 mainly reflects its greater appearance variability in color, texture, and illumination, making its distribution harder to model than the more structure-constrained polyp and CT datasets. We appreciate the reviewers for pointing these out and will correct the typo in Sec. 2.2 and the notation inconsistency in Fig. 1.



Meta-Review

Meta-review #1

  • Your recommendation

    Provisional Accept

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The reviewers are overall positive about DSD, agreeing that it presents a plausible and reasonably clear methodological contribution to paired medical image–mask diffusion synthesis, with a well-motivated design and consistent empirical gains across endoscopy, dermoscopy, and CT modalities over diverse strong baselines. One reviewer raises a more measured concern that the novelty is incremental, though this is balanced against the breadth and consistency of the empirical case and the clarity of the design noted across the panel.

    For the camera-ready, authors should expand the ablation in Tab 3 — which currently reports only KID despite the paper’s central claim being image-mask coherence, and clarify the contribution relative to existing dual-stream image-mask diffusion designs, including the dual-stream architecture in MedSegFactory (Mao et al., ICCV 2025). Additional points raised by the reviewers — including compute-fairness against a single-stream baseline, design symmetry of the structure→appearance interaction, and the “real + synthetic” augmentation setting — should also be addressed.



back to top