Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

In the evolving application of medical artificial intelligence, federated learning stands out for its capacity to protect the privacy of training data, enabling collaborative model development without sharing local data from healthcare entities. However, the heterogeneity of data and systems across institutions presents significant challenges, undermining the efficiency of federated learning and the exchange of information between clients. To address these issues, we introduce a novel approach, MH-pFLGB, which employs a global bypass strategy to mitigate the reliance on public datasets and navigate the complexities of non-IID data distributions. Our method enhances traditional federated learning by integrating a global bypass model, which would share the information among the client, but also serves as part of the network to enhance the performance on each client. Additionally, \model provides a feature fusion module to better combine the local and global features. We validate MH-pFLGB’s effectiveness and adaptability through extensive testing on different medical tasks, demonstrating superior performance compared to existing state-of-the-art methods.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0040_paper.pdf

SharedIt Link: https://rdcu.be/dV55b

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72117-5_50

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0040_supp.pdf

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Xie_MHpFLGB_MICCAI2024,
        author = { Xie, Luyuan and Lin, Manqing and Xu, ChenMing and Luan, Tianyu and Zeng, Zhipeng and Qian, Wenjun and Li, Cong and Fang, Yuejian and Shen, Qingni and Wu, Zhonghai},
        title = { { MH-pFLGB: Model Heterogeneous personalized Federated Learning via Global Bypass for Medical Image Analysis } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15010},
        month = {October},
        page = {534 -- 545}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper introduces an approach for personalized federated learning to address heterogeneous models. The authors utilize a global bypass mechanism that eliminates the need for medical datasets. They use a 3-step training which are local model training, global bypass model training and global aggregation. They have evaluated their approach on 3 non-IID medical tasks including image classification and segmentation, claiming to surpass current state-of-the-art methods.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The global bypass model seems effective in improving the FL model performance, with thorough experimental details.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    • The definition of a “global bypass model” should be clearly explained. It seems to be a NN model that required data-driven training, but you also refer to it as a way to share information between clients (??) Specifically, it is claimed that “the global bypass model to transfer information among different clients” –> is such sharing secure? and is this violation of data privacy in FL? (In the intro, it is explained that in the server, global bypass are aggregated to share information. – I believe these are places that lack clarity.) What information is shared (the learned result from the global bypass model, I assume)?

    • Data security/privacy/fairness issues of the proposed global bypass model framework is not addressed or discussed. I see this a major downside.

    • The paper lacks comprehensive comparison (as well as literature survey) that also compute sharable, perturbed, or perhaps encoded features, that can also be used to improve the sharing of information across clients. What are the pros and cons of your proposed method compared with these methods, in terms of practicability, performance, privacy preserving, etc.

    • Paper contains grammatical errors and some typos.

    • Clients with data or resource limitations may contribute less overall, potentially leading to less effective model convergence in global bypass model training (Eq 2.).

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    Public sharing of your implementation code can improve the reproducibility of your method.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Please see weakness pointed above.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall, I think this paper has several issues and unclarities on the global bypass model, which otherwise making the framework reduced to a general FL method.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • [Post rebuttal] Please justify your decision

    I appreciate the authors for providing a rebuttal. Your rebuttal clarifies some of my questions, but I still have doubt on your answers to the questions asked in the initial review.

    Regarding security: your solution of sharing information from parameters of GBM can still be used for adversarial training, which are designed to attack black box models, right?

    Regarding fairness: your solution of evenly average (not weighted average) of parameters can still cause fairness issues for non-iid data among clients, right? as clients possess very few data of one kind can dominate when performing evenly average, right?

    I believe it is still hard to justify the proposed Global Bypass Model can work well under the claim that “no public dataset is required”, when facing real-world challenges, especially for medical image analysis where the task could be highly personalized.

    For these reasons, I stand with my rating of weak reject, but I could be wrong.



Review #2

  • Please describe the contribution of the paper

    The paper proposes a novel personalized federated learning approach by adopting a global bypass mechanism and weighted feature fusion module. These proposed modules alleviate the necessity for public datasets in global training and enhance the integration of features from both global and local models.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    This paper designs a bypass mechanism and feature weighted fusion module for personalized federated learning. The comprehensive experiments also show the efficacy and versatility of proposed framework to different medical tasks like classification and segmentation. The performance is good compared with many SOTA methods.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The personalized federated approach proposed by the authors is simple and lightweight. Some details of the experiment setting are missing. More ablation studies may be needed to explain the setting of hyperparameters during local and global model training.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    It would be helpful to include more details about the experiment setting under the scenario of different label distribution.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    (1) The weights λ for global and local model in equation (1) and (2) are important parameters during the training stage. How did authors decide the values? (2) Extensive experiment section but conclusions are very short. (3) The experiment setting about the classification under different label distribution task should be detailed. (4) About the experiment results shown in Table 2, authors use bold numbers to emphasize the best results. Some of the columns in Table 2 are not marked with the highest results.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The authors discussed about the challenges of personalized federated learning and proposed a novel method of personalized federated learning. Extensive experiments on different tasks are conducted to show the efficacy of the proposed method. However the conclusion and ablation study are relatively simple.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Accept — should be accepted, independent of rebuttal (5)

  • [Post rebuttal] Please justify your decision

    All my concerns have been addressed. The paper should be accepted.



Review #3

  • Please describe the contribution of the paper

    This paper proposes a federated learning method for medical images with model heterogeneity. A global bypass strategy is used to mitigate the reliance on public datasets with different distributions. Different kinds of applications are used to evaluate the performance of the proposed method.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1. Federated learning is an interesting and important problem.

    2. The proposed method seems reasonable.

    3. The method achieves good performance on several tasks compared with compared methods.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1. Fig. 1 is difficult to follow.

    2. It seems that the proposed method is a general approach without deeply exploiting the specificity of medical image analysis.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    1. Fig. 1 is not easy to understand. In particular, the four steps are not clearly presented.

    2. Although the proposed method performs well for federated learning, the specificity of medical images is not well captured in the model. In other words, it seems possible to apply the method to other non-medical applications. What kind of insight regarding medical image analysis is considered in the submission?

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Please refer to the comments regarding the presentation and design of the method.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • [Post rebuttal] Please justify your decision

    I appreciate the authors for their response. I will keep my score.



Author Feedback

We thank Reviewer R1, R3, and R4 for the in-depth discussion, constructive suggestions, and endorsements for our contributions: (1) the novel Model Heterogeneous FL framework, (2) the lightweight global bypass model for information sharing, and (3) our comprehensive experiments. We are encouraged that our framework is “novel” (R3, R4), “effective” (R1, R3, R4), “reasonable” (R4), and clear and detailed description" (R1, R3, R4), the experiments arecomprehensive” (R1, R3, R4), and the paper is ``well-written” (R1, R3, R4). We will release the code upon paper acceptance.

[R1Q1]. Global Bypass Model definition, information sharing, and security. We only share the parameters instead of any subject-specific features of our Global Bypass Model (GBM). The structure of GBM varies in different tasks, but for the same task, the GBM model structure is identical across different clients, and the model’s functionality is consistent with the local model. Therefore, on the central server, we can aggregate the uploaded GBM model parameters. Since the information is shared only through the parameters of GBM, data privacy can be well secured.

[R1Q2, R1Q5]. Data security/privacy/fairness; Client’s contribution with limited data. As our framework only shares the parameters of the global bypass model among various clients, security/privacy would not be a problem with our method. We evenly average the parameters of each client in aggregation, so our framework is fair to each client no matter the data scale difference among the clients.

[R1Q3]. Comparison with feature-sharing methods. As our framework shares information via model parameters without the needs of sharing subject-specific features, we have fewer privacy concerns compared to feature-sharing methods. Besides, our approach outperformed previous works shown in Tab.1-3. In terms of practicability, our methods only share a lightweight bypass model, which would greatly save computational resource compared to directly sharing features in the dataset.

We appreciate R1 for the insightful questions about data security, fairness, and performance comparison!

[R3Q1]. The weights λ, conclusions are very short, and Table optimization. We tried a few λs and the one we are using is well-balanced and has the best overall performance.

[R3Q3]. Experiment setting details. Our experimental setup is as follows: We first sort the data based on labels, and we will divide the data into 16 equal parts. Each of the 8 clients will randomly select two equal parts without repetition. This can ensure that each client has a different category and that the data distribution of each client meets Non-IID.

We thank R3 for the constructive comments on the experiment details and conclusions improvements!

[R4Q1]. Details for Fig 1(a). Fig 1(a) includes three steps: a. Local model training. b. Global bypass model training. c. Upload, aggregation, and download. Local model training: Train the local model with the global bypass model frozen. Global bypass model training: Fine-tune the global bypass model with the local model frozen. Upload, aggregation, and download: Upload the global model to the server, aggregate the parameters, and then download and distribute the aggregated model for the next training round.

[R4Q2]. Framework specificity of medical images. There are 2 reasons why our design is specifically suitable for medical scenarios. A) Compared to previous methods, our framework eliminates the need for a public dataset, which is highly medical-friendly considering the difficulty of medical dataset collection and publication. B) Our framework does not share patient-specific features, but only the network parameters, which raise fewer concerns about data privacy. This is also very useful in medical scenarios.

We thank R3 for the insightful comments on the specificity of our design in medical scenarios!



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    This work proposes a new heterogeneous FL method with a Global Bypass Model. The review comments are mixed. The rebuttal addresses the clarification questions on the methods and justifies the results; however, the reviewer still has concerns about the security and fairness of the proposed method. The authors should carefully revise these statements in the final version.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    This work proposes a new heterogeneous FL method with a Global Bypass Model. The review comments are mixed. The rebuttal addresses the clarification questions on the methods and justifies the results; however, the reviewer still has concerns about the security and fairness of the proposed method. The authors should carefully revise these statements in the final version.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    N/A

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    N/A



back to top