List of Papers Browse by Subject Areas Author List
Abstract
Counterfactual generation for chest X-rays (CXR) aims to simulate plausible pathological changes while preserving patient-specific anatomy. However, diffusion-based editing methods often suffer from structural drift, where stable anatomical semantics propagate globally through attention and distort non-target regions, and unstable pathology expression, since subtle and localized lesions induce weak and noisy conditioning signals. We present an inference-time attention regulation framework for reliable counterfactual CXR synthesis. An anatomy-aware attention regularization module gates self-attention with organ masks, confining structural interactions to anatomical ROIs and reducing unintended distortions. A pathology-guided module enhances pathology-token cross-attention within target anatomical regions during early denoising and performs lightweight latent corrections driven by an attention-concentration energy, enabling controllable lesion localization and extent. Extensive evaluations on CXR datasets show improved anatomical consistency and more precise, controllable pathological edits compared with standard diffusion editing, supporting localized counterfactual analysis and data augmentation for downstream tasks.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/5497_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{ZhaZic_MaskGuided_MICCAI2026,
author = { Zhang, Zichun AND Nie, Weizhi AND Guo, Honglin AND Su, Yuting},
title = { { Mask-Guided Attention Regulation for Anatomically Consistent Counterfactual CXR Synthesis } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16890},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes an inference-time attention regulation framework for counterfactual CXR synthesis, aiming to improve anatomical consistency and localization of pathological edits in diffusion-based image editing. Instead of modifying the model architecture or training procedure, the method introduces a set of test-time interventions during diffusion sampling, including anatomy-aware self-attention gating based on organ masks, pathology-guided cross-attention reweighting, and a lightweight latent correction step
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.Well-motivated problem formulation The paper addresses a known issue in diffusion-based medical image editing, namely structural drift during generation, and clearly articulates the challenges of maintaining anatomical consistency while introducing localized pathological changes.
2.Training-free and flexible design A key strength of the method is that it operates entirely at inference time without requiring retraining or additional learnable modules. This makes the approach flexible and potentially easier to apply across datasets or domains compared to methods that rely on task-specific training.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Unclear applicability to high-resolution medical imaging Chest X-rays are typically high-resolution images (e. g. , 1024×1024 or higher), which are important for detecting subtle pathological patterns. However, the paper does not clearly specify the resolution at which the method operates or evaluate its performance at clinically relevant resolutions. If the method is limited to lower resolutions, it may only capture coarse changes, significantly limiting its clinical usefulness.
2.Limited clinical relevance of the constrained editing setup The method constrains generation within lung regions using segmentation masks. While this may improve quantitative metrics by restricting modifications to a smaller area, most clinically relevant findings in CXR are already concentrated in the lung region. Therefore, preserving or improving non-target regions (outside the lungs) may have limited clinical value, and the reported gains may not fully reflect meaningful improvements for diagnosis.
3.Questionable clinical utility of counterfactual CXR generation The clinical motivation for counterfactual generation in CXR is not sufficiently clarified. While counterfactual analysis is often valuable in complex diseases with multiple interacting factors or unclear mechanisms (e. g. , neurodegenerative disorders), its role in CXR—where many findings are more directly observable—remains unclear. The paper does not convincingly demonstrate what specific clinical questions or workflows this approach would support.
4.Lack of clear clinical application scenario Although the paper claims potential benefits such as data augmentation and interpretability, it does not provide concrete clinical use cases or evaluations (e. g. , reader studies or downstream clinical tasks) to support these claims. As a result, the practical impact of the generated counterfactual images remains uncertain.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Please ref. to the weakness
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Reject
- [Post rebuttal] Please justify your final decision from above.
The authors confirmed all of my concerns instead of addressing my concerns….
Review #2
- Please describe the contribution of the paper
The proposed method utilizes two attention control mechanisms to ensure structural consistency and precise lesion editing in chest X-rays. First, Anatomy-aware Attention Regularization prevents structural drift that may occur during lesion correction by suppressing self-attention regions using organ masks. Simultaneously, Pathology-guided Attention Regulation amplifies cross-attention of pathology tokens within the target region and performs attention energy-based latent spatial correction to precisely control the location and extent of minute lesions. Through these complementary approaches, it generates counterfactual images that reflect accurate pathological changes while preserving the patient’s intrinsic structure.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.Operating solely at inference time, which provides computational efficiency.
2.The lung masks utilized in this study are automatically segmented based on high image contrast, facilitating the acquisition of large-scale data and ensuring high accuracy without the need for manual annotation by medical specialists. Furthermore, the characteristics of the lungs, which possess stability in position and shape, contribute to the model learning consistent spatial reference points.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.Table 1 shows only marginal performance gains over prior work. The proposed method fails to outperform ‘ProgEmu’ on both LPIPS and FID (bold formatting appears to be incorrectly applied in the table).
2.The paper cites InstructX2X[18] in the introduction ‘Structural instability’. Given that InstructX2X adopts a similar mask-and-prompt-based editing strategy, it is recommend to be included as a baseline in the SOTA comparison of Table 1. 3.In Figure 2, cardiomegaly represents pathological enlargement of the heart, and thus the pathology token’s attention should be concentrated on the cardiac region. However, since Omega is derived from the lung segmentation mask (M_anat), the cardiac region either falls entirely outside the mask or is only indirectly captured at the boundary where the lung and heart overlap. As a result, the cross-attention amplification (Eq. 6) cannot accurately target the cardiac region, and the self-attention gating (Eq. 5) may even suppress information flow toward the heart. It would be worth clarifying which specific design choice accounts for the visually plausible ‘mild cardiomegaly’ result shown in Figure 2. 4.The qualitative results for ‘mild pleural effusion’ and ‘mild cardiomegaly’ in Figure 2, much like Table 1, make it difficult to clearly identify the advantages of the proposed method over prior work. Given that attention modulation is the central methodological contribution targeting localized pathology, it is recommended to include attention map visualizations. (Specifically to verify whether attention is indeed concentrated on the target region and whether leakage into non-editing regions is effectively suppressed.)
5.The first contribution stated in the Introduction claims “~reduce repeated training and maintenance costs under cross-device and cross-domain shifts, improving generality and controllability”. It is unclear which aspect of the proposed method substantiates this claim of improved generalization, and no supporting experiments across different devices or domains appear to be provided.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
1.The results in Table 1 raise concerns about whether the proposed method offers meaningful gains over SOTA, and whether the core contributions are being adequately captured by the evaluation metrics.
2.M_anat is limited to lung segmentation masks, effectively constraining the spatial prior Omega to only three conditions (left, right, and bilateral lung). More fundamentally, these conditions are fixed prior to inference through simple text keyword parsing, meaning Omega is assigned independently of where the diffusion model actually responds to the given instruction. This implicitly assumes that the model’s editing intent aligns with the predefined organ mask, an assumption that does not always hold. As mentioned earlier, cardiomegaly is a representative counterexample, and based on the dataset used, pleural effusion and others could be further considered.
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Mentioned previously in Potential concern
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
Most major concerns have been addressed, but the presentation of the paper was not optimal, and further clarification is needed.
Review #3
- Please describe the contribution of the paper
The paper proposes an inference-time attention regulation framework for counterfactual chest X-ray synthesis. The main idea is to improve anatomical consistency and localized pathology editing by combining anatomy-aware self-attention gating with pathology-guided cross-attention regulation and a lightweight latent correction step. A practical aspect of the work is that it aims to improve controllability at inference time rather than relying on additional control branches or repeated retraining.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper addresses an important problem in medical counterfactual generation. In chest X-rays, preserving anatomy while editing pathology is genuinely hard, and the paper focuses on exactly this issue. The motivation is clear and relevant.
- The experimental results are reasonably solid. In Table 1, the method gets the best confidence and CLIP-I and competitive FID/LPIPS relative to the compared baselines, and the qualitative results suggest more localized and stable edits in non-target regions.
- The method is designed as an inference-time regulation approach, which makes it more appealing than methods that require more domain-specific retraining or additional control modules.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- The paper focuses only on chest X-rays and does not really test how broadly the method generalizes beyond this setting. Since a lot of the motivation is about general inference-time controllability, the empirical scope feels somewhat limited.
- The paper would be stronger with expert evaluation or a more task-specific analysis of whether the generated pathology is actually anatomically and clinically meaningful.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I am leaning weak accept because the paper is clear, practically motivated, and technically sound enough. The idea of regulating attention at inference time for anatomically consistent counterfactual CXR generation is useful, and the empirical results are overall positive. My main reservations are that the novelty is somewhat incremental and the evaluation is still limited from a medical validation perspective, but I think the paper is good enough to make a worthwhile contribution.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
My concerns regarding the scope were addressed.
Author Feedback
We appreciate reviewers’ comments and grouped the responses below.
Reply@R1/R3-Clinical utility: We do not position counterfactual CXR generation as replacing radiological diagnosis, and will clarify this scope in the final version. Its value is to create controlled “what-if” pairs that preserve patient anatomy while selectively changing a target pathology. These pairs support shortcut-bias inspection and audit whether diagnostic AI relies on the intended lesion rather than background, acquisition style, or unrelated anatomy, and can support augmentation for underrepresented findings. Thus, preserving non-target regions is central to counterfactual validity. We agree expert studies would further validate clinical usefulness, while this work focuses on anatomy-consistent and pathology-controllable generation.
Reply@R1/R2/R3-Resolution and generality: Our experiments use 512×512 images, as stated in the implementation details. This is a practical setting for latent-diffusion editing and enables fair comparison with diffusion baselines under feasible memory cost. We agree this should not be interpreted as diagnostic-resolution clinical deployment, especially for subtle findings, and will make this limitation clearer in the final version. Also, our “generality” claim refers to regulating attention at inference time without retraining new control branches for each setting, not to an empirically proven cross-device or domain performance gain. The submitted evidence mainly supports inference-time controllability in CXR; high-resolution and broader cross-domain validation remain important future directions.
Reply@R2-ROI prior and cardiomegaly: We clarify that M_anat/Ω is not lung-only. Sec. 2.3 states that organ masks include “left/right lung and heart,” and M_ROI is selected or combined from them using anatomical cues in the text. Thus, Ω is not restricted to left/right/bilateral lung: lung or pleura findings use lung ROIs, while cardiomegaly uses the heart/cardiac-silhouette ROI. Ω is not designed to automatically discover lesion locations from the model response; it is an interpretable anatomical prior for constraining where a specified counterfactual edit should occur. For shape-changing findings such as cardiomegaly, the ROI is a soft prior and can include the cardiac boundary/margin rather than a rigid original mask, allowing plausible enlargement of the cardiac silhouette. The model response is still involved through pathology-token cross-attention: Eq. 7 measures how token attention aligns with Ω, and Eq. 9 corrects the latent trajectory according to this concentration rather than blindly forcing pixels to change. If the selected ROI mismatches the target pathology, regulation may be less effective; this is a limitation of ROI-based regulation rather than a contradiction of the method.
Reply@R2-Metrics and baselines: We thank R2 for catching a clerical bolding error in Table 1: ProgEmu is better on LPIPS and FID, and the boldface should be corrected; this does not affect the reported values. Our claim is a trade-off among pathology alignment, structural preservation, realism, and inference-time flexibility: our method achieves the best Conf and CLIP-I while remaining competitive on LPIPS/FID without training extra control branches. For counterfactual editing, lower LPIPS/FID may reflect conservative editing or realism, but not necessarily successful target-pathology realization. The ablation also supports this: removing pathology cross-attention reduces Conf/CLIP-I, while removing anatomy regularization reduces SSIM. Attention-map visualization would clarify the mechanism and, if space permits, can be included in the final version. Our baselines cover major paradigms including inpainting, progressive editing, instruction-based generation, and multimodal autoregressive counterfactual generation; InstructX2X trains a local editing model, whereas our method performs training-free inference-time attention regulation.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper received mixed scores 3, 3, 4.Concerns include the design and comparison in experiments, as well as method generalization. AC invites the authors for a rebuttal to clear the concerns raised by all reviewers.
R2 raised an ethics concern – which should be considered as part of the general weaknesses instead of an ethics issue. Authors shall address those during rebuttal as well.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
This paper received 2 accept and 1 reject. 2 reviewers leaning towards acceptance are satisfied with the authors’ rebuttal and state their concerns have been resolved. Reviewer 1 remains towards rejection and stated their concerns are not resolved. I read through the paper, all reviews, and the authors’ rebuttal. I think this paper proposed an interesting method and also provided sufficient experimental support. In the rebuttal, the authors clarified their paper’s position and scope, which I do not agree with R1 that this is a weakness. As a MIC paper, its contribution is sufficient for publishing at MICCAI. Therefore, I am leaning towards acceptance.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
This paper was reviewed by three experts in the field. The initial recommendations were one Weak Accept and two Weak Rejects. After the rebuttal, two reviewers recommended acceptance, while one reviewer maintained a Reject recommendation. While the paper has some limitations regarding clinical utility and application, it also demonstrates several strengths, including well-motivated problem formulation and training-free attention regulation design. Therefore, the paper is recommended for acceptance to MICCAI 2026.The authors are encouraged to incorporate the necessary changes discussed during the rebuttal process into the final camera-ready version.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The authors addressed some of the reviewers’ major concerns. However, the paper presentation requires improvement in organization and clarity.