List of Papers Browse by Subject Areas Author List
Abstract
The task of single-source domain generalization (SDG) in medical image segmentation is crucial due to frequent domain shifts in clinical image datasets. To address the challenge of poor generalization across different domains, we introduce a Plug-and-Play module for data augmentation called MoreStyle. MoreStylediversifies image styles by relaxing low-frequency constraints in Fourier space, guiding the image reconstruction network. With the help of adversarial learning, MoreStylefurther expands the style range and pinpoints the most intricate style combinations within latent features. To handle significant style variations, we introduce an uncertainty-weighted loss. This loss emphasizes hard-to-classify pixels resulting only from style shifts while mitigating true hard-to-classify pixels in both MoreStyle-generated and original images. Extensive experiments on two widely used benchmarks demonstrate that the proposed MoreStyle effectively helps to achieve good domain generalization ability, and has the potential to further boost the performance of some state-of-the-art SDG methods.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0782_paper.pdf
SharedIt Link: https://rdcu.be/dZxdT
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72111-3_41
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0782_supp.pdf
Link to the Code Repository
https://github.com/zhaohaoyu376/morestyle
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Zha_MoreStyle_MICCAI2024,
author = { Zhao, Haoyu and Dong, Wenhui and Yu, Rui and Zhao, Zhou and Du, Bo and Xu, Yongchao},
title = { { MoreStyle: Relax Low-frequency Constraint of Fourier-based Image Reconstruction in Generalizable Medical Image Segmentation } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15008},
month = {October},
page = {434 -- 444}
}
Reviews
Review #1
- Please describe the contribution of the paper
the author introduces a Plug-and-Play module for data augmentation called MoreStyle. This MoreStyle combines an auxiliary reconstruction decoder with an adversarial noise encoder which is to generate perturbations for the reconstruction decoder. This author also proposes a novel loss in Fourier space.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The evaluation of this paper is strong:
- two different datasets to better support the advantages of the proposed method
- figure 1 shows the visualization of the widespread of the tSNE space is easy to show the potential of the generalization ability of the proposed method.
- table 1 involves a clearly comparison of multiple different backbone and the added proposed MoreStyle.
- This paper clearly explains the deficiencies of related work in data augmentation.
- The evaluation of this paper is strong:
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The specific novelty of this paper is not clearly demonstrated. The author claims that they propose a novel loss “customized uncertainty-weighted loss”, however , the modified fourier spectrum diversity somehow follows the work [11] and the novelty is relatively low.
- THe pipeline of the proposed method is vague, and there lacks an explanation of why utilize Adversarial Style Augmentation (ASA).
- In the experiment, the figure 3 only shows the qualitative segmentation results compared with some SOTA methods, and there is no other qualitative figures to show the data augmentation results, which is the main contribution of this paper.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
The author utilizes two public datasets, so the data is clear.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The motivation is very clear in this paper, but the main problem is that the overall pipeline from the input to the final output in the method section is vague. there also lacks a clear demonstration of how each module in the pipeline is connected and related. The pipeline figure is somehow complex and a more straightforward pipeline would be better. The novelty of the proposed loss should be more clarified regarding the data augmentation task.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- The description in the method section is not very organized and the thus the novelty might be less expressed and somehow be low.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Reject — should be rejected, independent of rebuttal (2)
- [Post rebuttal] Please justify your decision
Although the rebuttal clarify the novelty of the proposed method regarding adding the added loss function, the author does not calrify the whole method in a high-level overview, so it is still vague to me to understand the whole pipeline of each module and the whole network training process.
Also, the rebuttal format is somehow disorganized. The author answers questions to different authors together and not directly to the questions makes this rebuttal hard to read.
Review #2
- Please describe the contribution of the paper
The author discusses the development of a novel Plug-and-Play module called MoreStyle, which aims to enhance the diversity of training data for medical image segmentation. It achieves this by relaxing low-frequency constraints in Fourier space and utilizing adversarial style augmentation to generate images of diverse styles. The author also proposes an uncertainty-weighted loss to better utilize these diversely style-augmented images.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Based on the methodology and the ablative experiments, the paper seems to be well-structured and technically sound.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Writing of the paper can be improved. Introduction part is not coherent. More secifically, the passages before introducing proposed modules, there are at least twelve studies cited in consecutive listed portions, however none of their contributions has been discussed. Several phrases seems to be very generic an unclear, for example : ‘‘A straightforward approach to enhance domain robustness is data augmentation [3,20,19,26], which expands the range of the data and constrains the decision boundaries. Many Fourier-based data augmentation methods [22,8,25,27] emerge’’
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The overall methodology is convincing, although using fourier based frequency interventions and weighted entropy losses are widely used in domian adaptation methods.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The plug and play module can be useful in medical image applications and it also has potential to be further explored.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper advances the domain generalization ability of models used in medical image segmentation by introducing novel data augmentation modules and loss functions. It reports improved results compared to the current state of the art, and includes an ablation study that elucidates the rationale behind the chosen methodologies.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The results. The plug-and-play module.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The article claims that MoreStyle converges faster than the advanced MoreStyle method, yet it provides no experimental evidence to support this assertion. Additionally, the paper does not specify the epoch parameter settings used in the comparative experiments.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
Clear description will help reproduce the work.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
(1) While the paper includes experimental verification on two datasets, this analysis could be extended to additional medical imaging datasets, particularly those with significant style variations. Such an extension would provide a more comprehensive assessment of the MoreStyle module’s performance and robustness. Additionally, it would be beneficial to explore the effects of varying parameter settings on model performance.
(2) The paper could enhance its theoretical analysis of the MoreStyle module’s mechanisms, especially detailing how Fourier space operations and adversarial learning synergistically contribute to improved domain generalization effects.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The author compared many state-of-the-art methods and achieved impressive results.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Accept — should be accepted, independent of rebuttal (5)
- [Post rebuttal] Please justify your decision
Most of the concerns have been addressed by the author.
Author Feedback
We thank the reviewers for the valuable comments, which help to improve the quality of this paper. We are encouraged that the reviewers found our motivation and idea to be novel (R3, R4, R5), and that our comparisons against varying backbone showed significant improvements (R5). We respond to your concerns as follows. Detail of Fourier space operations and adversarial learning (R4, R5): Low-frequency components capture style variations across domains due to their high energy distributions, while high-frequency components focus on object structures and identity[3]. Therefore, it is reasonable for the reconstruction decoder to focus less on low-frequency components to produce images of diverse styles. To further enhance this capability, we utilize adversarial learning to generate data samples of more diverse styles that fool the segmentation network, significantly improving segmentation robustness[4], as shown in Table 3. Novelty clarification (R5): CCSDG[1] employs the same data augmentation method as FDA[2]. Fourier-based data augmentation methods, including FDA[2] and FACT[3], generate style-augmented images by just exchanging the low-frequency components of the amplitude spectrum. MoreStyle utilizes adversarial training and L_{FSD} to encourage the reconstruction decoder to generate images with more diverse low-frequency components, thereby also enhancing the stylistic diversity of the images. As illustrated in Fig 1 (b) and Fig 2 in the supplementary material, MoreStyle is widely spread across the tSNE space, further demonstrating its capability to generate images of more varied styles. We also compare the data augmentation of MoreStyle with FDA[2] and FACT[3] through ablation experiments, demonstrating superior performance. The results are presented in Table 4. Faster convergence speed (R4): MaxStyle[4] requires 1500 epochs for network convergence. In contrast, MoreStyle achieves good performance in just 100 epochs, showcasing a much faster convergence rate, as detailed in Section 3.2. MoreStyle adds 3.76M parameters over the baseline but does not increase inference time, maintaining efficiency while significantly enhancing performance, as shown in Table 3. Discussion about the contributions of related work(R3): Due to page limitations, our discussion on the contributions of related work is limited. We will conduct a fuller discussion about the contributions of related work in the camera-ready paper. Additional dataset (R4): We conduct comparative experiments on two public datasets. We are willing to test the WIA-LD2ND on more datasets. Due to the rebuttal policy prohibiting new experimental results, we will provide the results on GitHub in the future. Data augmentation results (R5): As shown in Fig. 1 and Fig. 2 in supplementary material, which show the data augmentation results, they demonstrate that MoreStyle generate images with more diverse styles. The effects of varying parameter (R4): We conducted a series of ablation studies on varying parameters as shown in Table 1 in supplementary material. [1] Hu, Shishuai and Liao, Zehui and Xia, Yong: Devil is in channels: Contrastive single domain generalization for medical image segmentation. MICCAI, 2023 [2] Yang, Yanchao and Soatto, Stefano: FDA: Fourier domain adaptation for semantic segmentation. CVPR, 2020 [3] Xu, Qinwei and Zhang, Ruipeng and Zhang, Ya and Wang, Yanfeng and Tian, Qi: A fourier-based framework for domain generalization. CVPR, 2021 [4] Chen, Chen and Li, Zeju and Ouyang, Cheng and Sinclair, Matthew and Bai, Wenjia and Rueckert, Daniel: MaxStyle: Adversarial style composition for robust medical image segmentation. MICCAI, 2022
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
One reviewer downgraded the paper from WR to R after the rebuttal, and the main remaining concern with the paper is clarification of the overall process. The AC considered the paper, the rebuttal, and the post-rebuttal comments and felt that the major concerns by the reviewers have been generally addressed. The AC did not find sufficient evidence to overturn the majority of the reviewers’ recommendations and decided to recommend acceptance of the paper. The authors are expected to improve the final version as clarified in the rebuttal, and improve the presentation of the whole pipeline to make it clear.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
One reviewer downgraded the paper from WR to R after the rebuttal, and the main remaining concern with the paper is clarification of the overall process. The AC considered the paper, the rebuttal, and the post-rebuttal comments and felt that the major concerns by the reviewers have been generally addressed. The AC did not find sufficient evidence to overturn the majority of the reviewers’ recommendations and decided to recommend acceptance of the paper. The authors are expected to improve the final version as clarified in the rebuttal, and improve the presentation of the whole pipeline to make it clear.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
N/A