List of Papers Browse by Subject Areas Author List
Abstract
In medical image segmentation, personalized cross-silo federated learning (FL) is becoming popular for utilizing varied data across healthcare settings to overcome data scarcity and privacy concerns. However, existing methods often suffer from client drift, leading to inconsistent performance and delayed training. We propose a new framework, Personalized Federated Learning via Feature Enhancement (pFLFE), designed to mitigate these challenges. pFLFE consists of two main stages: feature enhancement and supervised learning. The first stage improves differentiation between foreground and background features, and the second uses these enhanced features for learning from segmentation masks. We also design an alternative training approach that requires fewer communication rounds without compromising segmentation quality, even with limited communication resources. Through experiments on three medical segmentation tasks, we demonstrate that pFLFE outperforms the state-of-the-art methods.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0563_paper.pdf
SharedIt Link: https://rdcu.be/dV55h
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72117-5_56
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0563_supp.zip
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Xie_pFLFE_MICCAI2024,
author = { Xie, Luyuan and Lin, Manqing and Liu, Siyuan and Xu, ChenMing and Luan, Tianyu and Li, Cong and Fang, Yuejian and Shen, Qingni and Wu, Zhonghai},
title = { { pFLFE: Cross-silo Personalized Federated Learning via Feature Enhancement on Medical Image Segmentation } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15010},
month = {October},
page = {599 -- 610}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper presents pFLFE, a novel framework for personalized cross-silo federated learning in medical image segmentation. It addresses the client drift issue in existing methods by introducing a two-stage process: feature enhancement and supervised learning. The feature enhancement stage improves differentiation between foreground and background features, while the supervised learning stage uses these enhanced features for learning from segmentation masks. The framework also proposes an alternative training approach that requires fewer communication rounds without compromising segmentation quality. Experiments on three medical segmentation tasks demonstrate that pFLFE outperforms state-of-the-art methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1, The introduction of a two-stage process for feature enhancement and supervised learning is novel, improving the segmentation accuracy and training stability. 2, pFLFE effectively addresses the client drift problem, which is a significant challenge in federated learning, leading to more consistent performance. 3, The alternative training approach requiring fewer communication rounds is a significant advancement, especially in scenarios with limited communication resources.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1, The two-stage process, while effective, may introduce complexity in implementation and require more computational resources. 2, There is a possibility of overfitting due to the personalized nature of the framework, which may limit its generalizability. 3, The evaluation is limited to three medical segmentation tasks, and further testing is needed to assess the framework’s performance across a broader range of applications.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Do you have any additional comments regarding the paper’s reproducibility?
No
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
1, The paper could explore additional methods to further enhance the feature representation in the local feature enhancement stage, potentially incorporating more advanced techniques or algorithms to improve the differentiation between foreground and background features. 2, While the paper presents a fast-converging framework (FC-pFLFE) that requires fewer communication rounds, it may benefit from exploring more strategies to reduce communication overhead, such as compression techniques or more efficient aggregation protocols. 3, The paper might extend its experimentation to include more diverse medical segmentation tasks or datasets, which would help in validating the generalizability and robustness of the proposed pFLFE framework across a wider range of medical imaging applications. This could also involve testing the framework’s performance on multi-modal medical images.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The two-stage process for feature enhancement and supervised learning improves the segmentation accuracy and training stability. pFLFE also addresses the client drift problem, which is a significant challenge in federated learning.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This paper proposes a new framework, Personalized Federated Learning via Feature Enhancement (pFLFE), designed to mitigate the client drifts.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.This work has comprehensive and thorough experiments to support their method’s effectiveness. 2.Using representation learning to improve the local feature learning to address the client drifts is reasonable. 3.The writing is good enough to tell the technical details.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1.The presentation could be improved.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
1.The left part of fig. 2 has four arrows with different colors. You could simplify them to make the figure clearer. 2.Quantitative analysis of the communication and computation costs is needed. It seems the representation learning may cost more training resources. 3.Why only show the learning curves of three compared methods?
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Writing, experiments, and novelty
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
Cross-silo FL is gaining popularity in medical image segmentation due to its ability to utilize varied data across different healthcare settings without compromising data privacy. However, client drift remains a significant issue.
The authors propose a two-stage framework that includes feature enhancement and supervised learning. The feature enhancement stage aims to improve the differentiation between foreground and background features, while the supervised learning stage uses these enhanced features to learn from segmentation masks.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Fig 1. of the paper has a good motivation in learning invariant representation
- The proposed local feature enhancement method seems well resolved the issue.
- Comprehensive experiments show a good performance of the method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Since the proposed method incorporates data augmentation, it could lead to an unfair comparison over baseline methods that do not use augmentation. It is suggested that augmentation techniques be integrated into the baseline methods, particularly for FedRep and FedProx. Additionally, as the paper presents a PFL approach where only the encoder is trained globally, for a more fair comparison, it is recommended that FedProx, which also seeks to learn invariant features, should be modified to train only the encoder globally.
- Given that the model needs additional training for local feature enhancement, the efficiency of the model should be carefully considered. It is important to evaluate the cost with this extra training step to understand its impact on the overall efficiency of the model.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
- Since the proposed method incorporates data augmentation, it could lead to an unfair comparison over baseline methods that do not use augmentation. It is suggested that augmentation techniques be integrated into the baseline methods, particularly for FedRep and FedProx. Additionally, as the paper presents a PFL approach where only the encoder is trained globally, for a more fair comparison, it is recommended that FedProx, which also seeks to learn invariant features, should be modified to train only the encoder globally.
- Given that the model needs additional training for local feature enhancement, the efficiency of the model should be carefully considered. It is important to evaluate the cost with this extra training step to understand its impact on the overall efficiency of the model.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Since the proposed method incorporates data augmentation, it could lead to an unfair comparison over baseline methods that do not use augmentation. It is suggested that augmentation techniques be integrated into the baseline methods, particularly for FedRep and FedProx. Additionally, as the paper presents a PFL approach where only the encoder is trained globally, for a more fair comparison, it is recommended that FedProx, which also seeks to learn invariant features, should be modified to train only the encoder globally.
- Given that the model needs additional training for local feature enhancement, the efficiency of the model should be carefully considered. It is important to evaluate the cost with this extra training step to understand its impact on the overall efficiency of the model.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
Thank you to the three reviewers for their constructive feedback. We will revise the final version of the paper according to your requirements.
We thank Reviewer R3, R4, and R6 for the in-depth discussion, constructive suggestions, and endorsements for our contributions: (1) a personalized framework named pFLFE for medical image segmentation, (2) an alternative fast-converging framework, and (3) extensive experiments on 3 segmentation tasks. We are encouraged that our framework is “novel” (R3, R4), “effective” (R4, R6), and “reasonable” (R4), the experiments are
comprehensive" (R3, R4, R6), and the paper iswell-written” (R3, R4, R6). We will release the code upon paper acceptance.[R3Q1, R4Q2,R6Q2]. Two-stage training process efficiency. A training phase in pFLFE includes two communication rounds, whereas other methods include only one communication round per training phase. To ensure fairness, we evaluated our method over 50 training phases (ours, 100 communication rounds) against other methods over 100 training phases (theirs, 100 communication rounds) in experiments. Besides, in our two-stage training, the supervised learning stage uses Unet for fully supervised training which is similar to other methods. In the local enhancement stage, we replace the decoder with a smaller linear layer, which will reduce the computational resources and complexity. The results in Fig. 4 show that we achieve better performance with fewer communication rounds, which means our approach converges faster and more efficiently in training.
[R3Q2]. Generalizability concerns of the personalized framework. Similar to the previous methods, in our design, the encoder of the network is shared during inference, with the personalized part being only the decoder. Although in the contrastive learning stage, we perform feature enhancement for each client’s data, the goal of this stage is to train the encoder, whose parameters will ultimately be aggregated and shared among all clients during inference. Therefore, our design does not introduce additional generalizability concerns compared to previous methods. We demonstrate the generalization in three different segmentation tasks, as shown in Tab. 7, Sup Tab. 1, and Sup Tab. 2. The experimental results indicate that our method improves generalization compared to previous methods.
We appreciate your constructive ideas on the extensiveness of our experiments, additional representation learning methods, communication rounds reduction, and multi-modal design. Your suggestions have greatly inspired our future work.
[R4Q3]. Why only show the learning curves of the three compared methods? FedAvg, FedRep, and LG-FedAvg are the most widely used frameworks in this area. Comparison with those 3 frameworks would help us better understand the effectiveness of our method.
We thank R4 for the constructive and insightful comments on the training resource analysis and experiment details!
[R6Q1]. Data augmentation. The data augmentation we use is served as a standard approach in contrastive learning. As contrastive learning is a part of our newly designed framework, comparison with previous methods using data augmentation would not bring unfairness issues. Moreover, for the supervised learning part, our approach does not involve any form of data augmentation, which is the same as previous approaches. This would further ensure fairness in comparison.
Meta-Review
Meta-review not available, early accepted paper.