Abstract

The B-mode ultrasound based computer-aided diagnosis (CAD) has demon-strated its effectiveness for diagnosis of Developmental Dysplasia of the Hip (DDH) in infants. However, due to effect of speckle noise in ultrasound im-ages, it is still a challenge task to accurately detect hip landmarks. In this work, we propose a novel hip landmark detection model by integrating the Topological GCN (TGCN) with an Improved Conformer (TGCN-ICF) into a unified framework to improve detection performance. The TGCN-ICF in-cludes two subnetworks: an Improved Conformer (ICF) subnetwork to gen-erate heatmaps and a TGCN subnetwork to additionally refine landmark de-tection. This TGCN can effectively improve detection accuracy with the guidance of class labels. Moreover, a Mutual Modulation Fusion (MMF) module is developed for deeply exchanging and fusing the features extracted from the U-Net and Transformer branches in ICF. The experimental results on the real DDH dataset demonstrate that the proposed TGCN-ICF outper-forms all the compared algorithms.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3284_paper.pdf

SharedIt Link: https://rdcu.be/dV19m

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72086-4_65

Supplementary Material: N/A

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Hua_TopologicalGCN_MICCAI2024,
        author = { Huang, Tianxiang and Shi, Jing and Jin, Ge and Li, Juncheng and Wang, Jun and Du, Jun and Shi, Jun},
        title = { { Topological GCN for Improving Detection of Hip Landmarks from B-Mode Ultrasound Images } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15005},
        month = {October},
        page = {692 -- 701}
}

Reviews

Review #1

Please describe the contribution of the paper

This paper proposes a method for diagnosing the development of pediatric hip joints using landmark detection. Firstly, local and global feature information is extracted using CNN and Transformer respectively, obtaining an initial landmark feature map. Secondly, a graph network is utilized to model the topological relationship among landmarks for achieving high-quality localization results. The proposed method outperforms five other methods on a validation based on the authors’ private dataset.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

• The authors realized the importance of both local and global features for pediatric hip joint diagnosis. They employed CNN and Transformer to obtain the local and global features separately and designed a fusion module to effectively utilize the information.

• The graph network was innovatively utilized and validated in the task of pediatric hip joint diagnosis. The localization performance was successfully improved by the proposed graph network which modeled the topological relationship among landmarks based on Graf method.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

• The explanation of how the graph model work is insufficient. What are the meanings of labels “normal” and “abnormal” in the graph network classification module? There are no specific descriptions for them. Besides, there is no explanation of how these labels provide additional constraints and why they can improve the localization performance in the graph network.

• The description of the dataset is not comprehensive enough. What do “normal” and “abnormal” mean in the dataset, and what are their distributions in this dataset? How many kinds of Graf types are included in the dataset? What is the inclusion and exclusion criteria for the data? What are the levels of experience of the doctors involved, and so on.

• The diagnostic capability of the proposed method was not validated from the perspective of clinical metrics such as alpha and beta angle errors, accuracy of Graf types, etc. Adding these assessment indicators will make this paper more convincing.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

• Adding specific definitions for the classification labels ‘normal/abnormal’ will make the paper clearer. Do these labels refer to ‘healthy/unhealthy’ hip joints or ‘standard/non-standard’ planes? Why these classification labels can provide additional constraints for the graph network.

• In section 3.1 Dataset, adding the detailed information of the dataset will make the paper more convincing. The following aspects should be considered: 1. the specific meanings and distribution of “normal/abnormal” data. 2. the kinds of Graf types that included in the dataset.

• Hip joints of subluxation or dislocation significantly differ from that of common Type I and Type II, but there is no analysis for these kinds of Hip joints. Does the dataset include Hip joints of subluxation or dislocation? If such hip joints are not included in the dataset, please specify the limitations of this study. If included, presenting the analysis results for these types will make the paper better.

• Even small localization errors can lead to significant angle measurement errors, affecting the classification judgments and following diagnoses. Thus, comparing the method with sonographers in terms of the clinical metrics such as alpha, beta angle and the accuracy of Graf typing is significant. However, the paper did not mention these assessments while several cited references included. Supplementing these assessments will make the paper more convincing.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Accept — could be accepted, dependent on rebuttal (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The method presented is innovative. However, the specific meanings of the classification labels are unclear, which hinders understanding how the graph network benefit from them. Additionally, there is a lack of clinical metric validation to assess the performance in the clinical scenarios.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

This work proposes a hip landmark detection model by integrating the Topological GCN (TGCN) with an Improved Conformer (TGCN-ICF). Authors design an Improved Conformer (ICF) subnetwork to generate the related heatmaps and a TGCN subnetwork to refine landmark detection. Moreover, a Mutual Modulation Fusion (MMF) is developed for deeply exchanging and fusing features extracted from the U-Net and Transformer branches in ICF. The motivation is interesting and clear. The experimental results indicate the effectiveness of the proposed method.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1 The motivation of this work is clear and interesting. Skeletons have a natural structural morphology, thereby introducing global structural dependencies is meaningful for improving landmark detection. 2 Applying GCN for medical image landmark detection is relatively rare, thus this paper has good reference significance. 3 Authors propose a novel method, Mutual Modulation Fusion (MMF), for deeply exchanging and fusing features extracted from the CNN and ViT.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

1 The Adjacency Matrix is design with a strong prior knowledge, which could be suboptimal solution. 2 Missing some related work about Hip landmark detection, especially published in MICCAI and TMI.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

This paper is written with clear description and good reproducibility. Of course, it could be much better to provide the code and dataset in future.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

1 The core contribution of this paper is applying GCN to modeling the structural dependencies between hip landmarks. However, authors construct the Adjacency Matrix with a strong prior knowledge, as shown in Eqn.6. Will this design harm the learning process? 2 Does the global structural relationship still exist for developmental abnormalities (like DDH case), and will the Adjacency Matrix change in this case? 3 Why do you introduce a Binary Cross Entropy (BCE) loss between ground truth labels and predicted classes? The identification of landmarks has already been presented by the index of heatmaps. 4 Please attach figures with higher resolution. At present, some figures, like fig.4, are with poor resolution, making it difficult for reviewers to observe the visualization effect of this method.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Accept — should be accepted, independent of rebuttal (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The reviewer is a senior researcher in medical image landmark detection and has conducted long-term exploration on modeling the global structural relationships of bone landmarks. For a long time, reviewer has been eagerly anticipating the research to introduce GCN for structural modeling. This work precisely solves this problem and has good reference significance for research in this field.
Reviewer confidence

Very confident (4)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

This paper propose a hip landmark detection model by integrating the topological GCN with an improved conformer into a unified framework to improve detection performance. The proposed model learns the spatial topological relations among landmarks with the guidance of class labels, so as to improve the detection accuracy.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The authors integrate the topological GCN modual and conformer modual into an unified framework, which can further refine the heatmaps generated from the ICF subnetwork.
2. The authors develope an MMF model to fully exchange and fuse the local and global features. The experimental results show its effectiveness.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The proposed method depends on sufficiate labelled b-mode ultrasound images to achieve an accurate performance, how to address the overfit issue in your framework.
2. The ‘LTR-GCN’ in Fig.2 seems not be defined until Subsection 3.2.
3. The sentence ‘where λ a hyperparameter…’ in Subsection 2.3 is a mistake.
4. The abbreviations in this manuscript is to much, which decrease its readability. For example, ‘ICF’ and ‘Improved Conformer Subnetwork’ are used mixed in Section Method.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The authors proposed an automatic landmark detection model for DDH, significantly effective in clinical practice. The idea is interesting, and the method is easy to reproduce. However, I have the following concerns:
1. The proposed TCGN-ICF is still a data-driven deep learning method, which always suffer from the overfit issue. The dataset used in work only has 500 hip ultrasound images from 294 infants. How to address the overfit issue and make sure the generalization of the model.
2. The graf’s method is the classical tool for diagnosing DDH. As an important process, the authors should calculate the \alpha and \beta angles using the outputs to compare its performance with the clinical sonologists.
3. The hip ultrasound images seems not match the number of infants. Does the right or left hip affect the detection performance.
4. Other concerns: (1) The abbreviations are too mach in this manuscript, which reduce its readability. Besides, some of the abbreviations are also not defined in the first mentioned, such as ‘LTR-GCN’. (2) The sentence ‘where λ a hyperparameter…’ in Subsection 2.3 is a mistake. (3) ‘ICF’ and ‘Improved Conformer Subnetwork’ are used mixed in Section Method.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Strong Accept — must be accepted due to excellence (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This paper presents an interesting idea that can effectively enhance landmark detection performance for DDH in clinical practice. Although some modules of the proposed method are not described in detail, and the generalizability, a major concern in clinical settings, is not thoroughly discussed, this does not diminish the value of the idea.
Reviewer confidence

Very confident (4)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Author Feedback

Summary: Thank all the reviewers for acknowledging our methodological contribution. We are pleased that they find this work interesting (R1), of good reference significance (R3), and innovative (R4). Our responses are as follows. Q1 (R1): “Overfit issue” A1: The overfit issue is common when the training samples are limited. To alleviate it, we adopted the Random Horizontal Flip as a data augmentation operation to enlarge our dataset during the model training. Q2 (R3): “Construction of the Adjacency Matrix” A2: The adjacency matrix is constructed by the spatial relations among the six hip landmarks. Since the topological relations will not be changed within the BUS images, the corresponding adjacency matrix will also keep unchanging for any DDH case (normal or abnormal). Additionally, such a design will help refine the generated heatmaps by ICF subnetwork instead of harming the learning process. Q3 (R3, R4): “BCE loss” A3: We introduce the BCE loss between ground truth labels and predicted classes in the TGCN subnetwork for further refining the heatmaps generated by ICF subnetwork. The classification labels can provide guidance for the graph subnetwork, so as to enhance the landmark detection performance. Q4 (R1, R4): “DDH dataset” A4: According to the Graf’s method, the α and β angles are two key metrics to diagnose the normal/abnormal of the hip joints (Normal if α > 60˚ and β < 77˚). Our dataset includes 500 BUS images (458 normal subjects and 42 abnormal subjects). Each image is scanned from either the right or left hips of infants, as this distinction does not affect the detection performance. It worth noting that the data is unevenly distributed, which is typical in real clinical settings. However, our TGCN-ICF achieves superior results to SOTA compared algorithms, indicating its effectiveness and robustness. We acknowledge that we do not focus extensively on the DDH classification in this work, as the primary aim is to improve the detection accuracy of hip landmarks. Q5 (R1, R4): “The following angles calculation” A5: Due to the page limit, we are unable to include extensive additional results in the manuscript. In future work, we will focus on developing a complete model together with experiments for DDH diagnosis, which is not limited only to detect the key points from hip BUS images. In addition, we have carefully checked and revised some errors in the manuscript, including improving the resolution of all figures. Thank you again for all the valuable comments!

Meta-Review

Meta-review not available, early accepted paper.

back to top

Topological GCN for Improving Detection of Hip Landmarks from B-Mode Ultrasound Images

Author(s):