Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Medical image segmentation is crucial in the field of medical imaging, assisting healthcare professionals in analyzing images and improving diagnostic performance. Recent advancements in Transformer-based networks, which utilize self-attention mechanism, have proven their effectiveness in various medical problems, including medical imaging. However, existing self-attention mechanism in Transformers only captures pairwise correlations among image patches, neglecting non-pairwise correlations that are essential for performance enhancement. On the other hand, recently, graph-based networks have emerged to capture both pairwise and non-pairwise correlations effectively. Inspired by recent Hypergraph Neural Network (HGNN), we propose a novel hypergraph-based network for medical image segmentation. Our contribution lies in formulating novel and efficient HGNN methods for constructing Hyperedges. To effectively aggregate multiple patches with similar attributes at both feature and local levels, we introduce an improved adaptive technique leveraging the K-Nearest Neighbors (KNN) algorithm to enhance the hypergraph construction process. Additionally, we generalize the concept of Convolutional Neural Networks (CNNs) to hypergraphs. Our method achieves state-of-the-art results on two publicly available segmentation datasets, and visualization results further validate its effectiveness. Our code is released on Github: https://github.com/11yxk/AHGNN.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2689_paper.pdf

SharedIt Link: https://rdcu.be/dV5J7

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72114-4_3

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/2689_supp.pdf

Link to the Code Repository

https://github.com/11yxk/AHGNN

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Cha_ANovel_MICCAI2024,
        author = { Chai, Shurong and Jain, Rahul K. and Mo, Shaocong and Liu, Jiaqing and Yang, Yulin and Li, Yinhao and Tateyama, Tomoko and Lin, Lanfen and Chen, Yen-Wei},
        title = { { A Novel Adaptive Hypergraph Neural Network for Enhancing Medical Image Segmentation } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15009},
        month = {October},
        page = {23 -- 33}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper proposes a medical image segmentation framework using the concept of hypergraphs. It introduces a novel node degree-based hyperedge construction strategy, where the node degree represents its importance, and neighbors are adaptively assigned based on the node degrees. The experimental results demonstrate state-of-the-art performance on two publicly available segmentation datasets, and the visualization results further validate the effectiveness of the proposed method.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. This paper capture the correlations between image patches by leveraging adaptive techniques to enhance the hypergraph construction process using K-NN algorithms and relating the degree of any node to its importance.
    2. This paper is well-structured.
    3. The paper provides visualization results to further validate the effectiveness of the proposed method.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. This paper validates the proposed method on two datasets, but only achieves state-of-the-art results on one dataset. This suggests that the generalization capability of the method may be a potential issue.
    2. The novelty is limited. Compared to other GNN-based segmentation methods, the only difference is replacing GNN with HGNN.
    3. The paper should conduct a more comprehensive comparison with current state-of-the-art techniques, including different configurations and variants of HGNN models.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    1. The author should provide a comparison of the parameter counts for different models.
    2. The author showed in the experiments that the vanilla transformer performed worse than the hypergraph, but there are now many transformers that incorporate graph priors, such as GraphFormer[1]. The author should also provide comparative results against these models (by replacing the AHGNN module with methods like GraphFormer).
    3. The proposed method did not achieve the best results on the ACDC dataset, which raises concerns about the generalization capability of the algorithm.

    [1]Ying C, Cai T, Luo S, et al. Do transformers really perform badly for graph representation?[J]. Advances in neural information processing systems, 2021, 34: 28877-28888.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper proposed a HGNN based segmentation network, while achieve sota results on only one dataset. Besides, the novelty is limited.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper
    1. This paper presents a segmentation framework using the concept of hypergraph for medical image segmentation.
    2. This paper introduces a novel strategy for constructing hyperedges utilizing the degree of nodes. The degree of nodes represents their importance, thus, neighbors are assigned to nodes adaptively based on their degree.
    3. This paper generalizes the concept of CNN to the hypergraph domain, enabling the creation of hyperedges to learn local features in a manner similar to the sliding window receptive field.
  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The idea of the proposed adaptive hypergraph modeling strategy is interesting, and the visualization results and the supplementary indicated its effectiveness.
    2. The article is well structured, the experimental details are well developed and high results are achieved in the experimental part.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. Fig. 1 shows the comparison between CNN, Transformer, Graph, Hypergraph. However, the difference between the adaptive hypergraph (this work) and the conventional hypergraph is not be described.
    2. \epsilon-ball is also widely used for hypergraph modeling, in which each hyperedge contains different numbers of nodes. This paper lacks of a comparison of this strategy.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    The idea of the proposed adaptive hypergraph modeling strategy is interesting, and the visualization results and the supplementary indicated its effectiveness. However, the motivation of this paper is to group pixels that have similar characteristics. It should be noted that the \epsilon-ball, which is also widely used in hypergraph modeling, includes hyperedges containing varying numbers of nodes. Compared to the proposed two-step adaptive hypergraph modeling strategy, the \epsilon-ball approach requires only one step. This paper lacks a comparison with this strategy. The authors should provide more discussion.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Existing hypergraph modeling methods are divided into feature-based methods (KNN, \epsilon, cluster, etc.) and knowledge-based methods (network, attributes, etc.). KNN generates hyperedges with a fixed number of nodes (dependent on K), whereas other methods produce hyperedges with varying node counts. The authors fail to introduce other modeling methods. Although the experimental results and visualizations look promising, further discussion and additional experiments are required.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    The authors introduce a novel medical image segmentation framework based on combination of CNN and hypergraph neural netwrok. It builds upon traditional graph-based methods by leveraging hypergraph neural networks (HGNNs). The model employs a U-Net-like architecture combined with an adaptive hypergraph construction mechanism. This mechanism uses the K-Nearest Neighbors (KNN) algorithm to dynamically create hyperedges that capture intricate dependencies beyond simple pairwise interactions. The effectiveness of this method is validated on two public datasets, achieving state-of-the-art results with improved segmentation accuracy compared to existing approaches.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1-The application of hypergraphs to capture non-pairwise correlations provide a more comprehensive analysis of image data, potentially improving segmentation accuracy significantly. 2-The adaptive mechanism for hypergraph construction based on node degree allows the model to better represent the structural and semantic complexities of medical images. 3-The generalization of CNN operations to hypergraphs, mimicking the receptive field through adaptive hyperedges, is a novel approach that could enhance feature extraction capabilities in medical image segmentation tasks.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    . Authors haven’t discussed about number of patches, and its effect. How many patches are made using one image and what will be effect if we increase or decrease the number of patches per image?

    1. While the hypergraph approach is promising, its complexity might pose challenges in terms of computational efficiency and scalability, particularly when applied to larger datasets or higher resolution images. What is authors response to scalability and computational issues? 3.The paper could benefit from more detailed explanations of certain technical aspects, such as the specific adjustments made for the node degree in hypergraph construction and the implications of these adjustments on the model’s performance.
    2. The validation of the model on additional datasets with varying characteristics could help establish its robustness and applicability to a broader range of medical imaging tasks.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    Overall methodology is well explained but in AHGNN module authors haven’t mentioned the number so layers for HGC.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    The paper omits comparisons with key baseline models, nnU-Net [1] and nnFormer [2]. These models achieve higher Dice scores (ACDC: 92.06, Synapse: 86.99) than the authors report (ACDC: 92.02, Synapse: 84.06). An explanation for this discrepancy is needed.

    1. The fusion module in Figure 2 lacks color coding according to the legend. Please address this oversight.
    2. The paper mentions specific values for K neighbors (1), kernel size (3), and stride (1) but fails to explain the rationale behind these choices.
    3. The well-structured paper presents a clear methodology, new knowledge, and substantial results. However, including discussions on potential limitations (e.g., accuracy saturation) and deeper insights into the model’s behaviour across diverse segmentation scenarios would strengthen the paper’s impact. [1] Isensee, Fabian, et al. “nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation.” Nature methods 18.2 (2021): 203-211. [2] Zhou, Hong-Yu, et al. “nnformer: Volumetric medical image segmentation via a 3d transformer.” IEEE Transactions on Image Processing (2023).
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Authors proposal of hypergraph based on node degree is new knowledge to field along with other learning of local features.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Accept — should be accepted, independent of rebuttal (5)

  • [Post rebuttal] Please justify your decision

    Authors provide enough evidence to counter the critiques for my side. I am satisfied with their explanations.



Author Feedback

We are grateful to the reviewers for their valuable comments to improve our paper. The following are our responses to the reviewers’ comments. If paper is accepted, we will add these changes in the final version. R#: Reviewers | comments order

1.Method (A) Novelty: replacing GNN with HGNN (R#3: 3-2) Our contribution lies in formulating novel and efficient HGNN methods for constructing Hyperedges. Conceptually, we did not replace GNN with HGNN. Instead, we introduced a novel approach based on node degree using KNN, which has not been utilized previously and offers an improved scheme to construct hypergraphs. Our results validate the effectiveness of the method. (B) Patch number and its effect (R#4: 3-1) We construct hyperedge by considering one pixel as one node in feature space. A node is considered as a patch. We will conduct further experiments with different numbers of pixels per patch. (C) Technical details & parameter adjustments (R#4: 3-3) We will emphasize node degree construction and its implications in hypergraph construction, along with providing visualization results. We have also provided experiments using tuning parameter (alpha) to adjust effect of node’s degree. We observe that a low value of alpha is needed to avoid redundant information.

2.Experiments (A) Additional dataset to compare (R#3: 3-1, 6-3; R#4: 3-4) We validated our method using two widely used benchmark public datasets. Our method surpasses SOTA methods and achieves competitive result. For one dataset and compared to only one method, we observe a marginal and almost negligible drop in accuracy. Additionally, we have conducted experiments on another dataset (MoNuSeg), achieving SOTA results. The results validate the effectiveness of our method, highlighting the novel Hyperedge construction mechanism. (B) Detailed comparison with SOTA and other baseline methods (R#3: 3-3, 6-2) We will add additional comparisons with methods such as HGNN+ and Graphormer. (C) Comparison: nnU-Net and nnFormer (R#4: 7-1) Accuracy in nnU-Net and nnFormer is tested using 3D volumes, while our method is evaluated on 2D slice images. So, the evaluation methods are different. In the final paper, we will compare the proposed method with these methods using the same evaluation method. (D) Analysis: existing hypergraph modeling methods (\epsilon ball) (R#1: 3-2, 6-1) Our method models feature and local level hyperedge construction, merging these hyperedges to form a robust representation. This procedure operates simultaneously. While the \epsilon-ball approach uses a fixed-size radius to include nodes, resulting in redundant information. In contrast, our hyperedge construction method is adaptive based on the degree of nodes, efficiently capturing information. We will add a visualization comparison of hyperedge construction using these methods. (E) Computation complexity (R#3: 6-1, R#4: 3-2) Hypergraph construction process does not increase parameter count and maintains low computational cost by using low-resolution feature space. We calculated parameter count and our method requires fewer parameters. Further, increasing number of pixels in a patch for higher resolution and using a sparse matrix for hypergraph modeling (as in PyTorch Geometric library), may further reduce computation cost. Our method is efficient and computational time is comparable. We will include the parameter counts in the relevant table. Table-1(c): Transformer: 55.55M, Graph: 69.75M, Ours: 43.08M Table 2: ScaleFormer:113.81M, MAXFormer:88.93M, Ours: 43.08M (F) Model scalability (R#4: 3-2) Our method can be easily integrated into other encoder-decoder frameworks such as TransUnet and ResUNet. (G) Hyperparameter analysis: K neighbors and others (R#4: 7-3) The hyperparameter values are selected based on our experiments. We will add analysis in relevant sections.

  1. Figure Quality (R#1: 3-1; R#4: 7-2) We have redrawn the diagram.

  2. Discussions (R#4: 7-4) We will add discussions accordingly.



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    The rebuttal is quite thorough and has addressed the reviewers’ concerns regarding the experimental setup details, comparison to state-of-the-art methods, and computational complexity analysis. Overall this paper has sufficient contributions and the performance is promising. The final version should include more discussions to highlight the novelty of the paper.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    The rebuttal is quite thorough and has addressed the reviewers’ concerns regarding the experimental setup details, comparison to state-of-the-art methods, and computational complexity analysis. Overall this paper has sufficient contributions and the performance is promising. The final version should include more discussions to highlight the novelty of the paper.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    N/A

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    N/A



back to top