List of Papers Browse by Subject Areas Author List
Abstract
Chest radiography is a commonly used diagnostic imaging exam for monitoring disease severity. Machine learning has made significant strides in static tasks (e.g., segmentation or diagnosis) based on a single medical image. However, disease progression monitoring based on longitudinal images remain fairly underexplored, which provides informative clues for early prognosis and timely intervention. In practice, the development of underlying disease typically accompanies with the occurrence and changes of multiple specific symptoms. Inspired by this, we propose a multi-stage framework to model the complex progression from symptom perspective. Specifically, we introduce two consecutive modules namely Symptom Disentangler (SD) and Symptom Progression Learner (SPL) to learn from static diagnosis to dynamic disease development. By explicitly extracting the symptom-specific features from a pair of chest radiographs using a set of learnable symptom-aware embeddings in SD module, the SPL module can leverage these features for obtaining the symptom progression features, which will be utilized for the final progression prediction. Experimental results on the public dataset Chest ImaGenome show superior performance compared to current state-of-the-art method.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3074_paper.pdf
SharedIt Link: https://rdcu.be/dVZiV
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72378-0_56
Supplementary Material: N/A
Link to the Code Repository
https://github.com/zhuye98/SDPL.git
Link to the Dataset(s)
https://physionet.org/content/chest-imagenome/1.0.0/
https://physionet.org/content/mimic-cxr-jpg/2.0.0/
BibTex
@InProceedings{Zhu_Symptom_MICCAI2024,
author = { Zhu, Ye and Xu, Jingwen and Lyu, Fei and Yuen, Pong C.},
title = { { Symptom Disentanglement in Chest X-ray Images for Fine-Grained Progression Learning } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15001},
month = {October},
page = {598 -- 607}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes an end-to-end symptom-level progression learning framework to capture subtle changes in CXRs. This framework consists of two modules: Symptom Disentangler (SD) and Symptom Progression Learner (SPL). SD extracts symptom level features from CXRs and SPL learns to capture the symptom-level changes between two CXRs. The model is evaluated on the Chest ImaGenome dataset.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The symptom Disentangler (SD) and Symptom Progression Learner modules are novel approaches to disentangle symptom representations and capture changes between two CXRs.
- The paper is well written and organized.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The proposed method is evaluated only on one dataset and compared with only one baseline. This is a primary weakness of the proposed method. In CheXRelFormer, several comparisons such as Local, Global and CheXRelNet are shown.
- In Table 2, Mean precision, Mean Recall and Mean F1-score is reported, is there any reason why accuracy is not reported similar to CheXRelFormer (Table 2). Also, statistical significance should be reported for these results.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Address the concerns mentioned in weakness. Minor: Introduction Paragraph 5 Line 4 “cam” should be “can”. Specify X and X’ in Figure 1.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper proposed a novel method for disentangled symptom representation for disease progression learning however lacks detailed quantitative comparison with other baselines.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
The authors addresses my comments and hence I upgrade my review.
Review #2
- Please describe the contribution of the paper
The paper proposes end-to-end symptom-level progression learning framework to capture subtle changes in a pair of chest X-ray images.
The authors also introduce a symptom-disentangler to extract symptom-specific features from chest X-ray images, obviating the requirement for image registration techniques.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Experimental results show the effectiveness of the proposed method.
The proposed method utilizes symptom-level information to enhance the prediction accuracy compared to using only image-level features.
The proposed method shows stable performance across different age groups and outperforming compared methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Limitations and computational requirements are not discussed.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Please revise the paper to correct writing mistakes and typos.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed method achieves good performance, but the complexity and limitations are not mentioned in the paper.
- Reviewer confidence
Not confident (1)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
After carefully reading the rebuttal and comments of other reviewers, I keep my initial score.
Review #3
- Please describe the contribution of the paper
The authors proposed a novel end-to-end symptom-level progression learning framework that can accurately capture the subtle changes ina a pair of chest X-ray images. This metod extract symptom-specific features from chest X-ray images, obviating the requirement for any image registration techniques.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
It is very interesting paper, which can be clinically applied to the one of the most common used medical imaging examination, X-ray. The paper is well described.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- I am not sure about the third contribution. Why the authors claim and list contribution by using one of the datasets?
- There are lack of comparison with other state-of-the-art methods.
- There are no ablation studies, which can claim the contributions
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The authors proposed an interesting paper with interesting task. However, I would recomment to add more comprehensive evaluation of this method, including more state-of-the-art methods as well as ablation studies of key components used in the proposed method.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This is an interesting work with a minor flaws, described in the “main weaknesses”.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
Based on Reviewers’ comments and the rebuttal letter, I will keep my previous decision.
Review #4
- Please describe the contribution of the paper
The paper proposes two modules, Symptom Disentangler (SD) and Symptom Progression Learner (SPL), for symptom-level progression learning in chest X-ray images. SD extracts symptom-specific features, while SPL utilizes these features to capture subtle changes for accurate progression prediction.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper explores disease progression monitoring based on longitudinal images, a valuable contribution to the clinical domain. The proposed approach for symptom-level progression learning appears novel and outperforms the baseline. It is well-written, and the figures are precise with good captions.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The paper needs more detailed information about the data preprocessing steps, which could affect reproducibility and understanding. To enhance reproducibility, open-sourcing the code is encouraged.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
To enhance reproducibility, open-sourcing the code is encouraged and more information about the data preprocessing steps.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Overall, the proposed symptom-level progression learning shows improvement over the baseline and appears promising for progression learning in chest X-rays. Section 3.1 needs clarification on whether “CheXFormer[15]” refers to the CheXRelFormer mentioned previously or another model. Additionally, open-sourcing the code is highly encouraged. If possible, more details about the data preprocessing steps should be included.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper presents two novel modules for symptom-level progression learning, showing improvement over the baseline. It contributes to the field of disease progression monitoring and holds promise for future research in this area.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Author Feedback
Thanks all the reviewers for their valuable and constructive feedback, and for recognizing my work as interesting and novel. Our responses to your comments are as follows:
C1. Comparison with the state-of-the-art method on one dataset (R2, R6):
In this study, we introduce a new task namely, fine-grained symptom-level progression classification, which requires additional symptom labels for training (please note that all existing methods did not consider symptoms, and therefore could not predict the progression of multiple symptoms simultaneously). To the best of our knowledge, the Chest ImaGenome dataset is the only one publicly available dataset which provides such symptom-level diagnostic labels (obtained from MIMIC-CXR). We notice that SOTA method, CheXRelFormer [MICCAI 2023] outperforms all existing methods on Chest ImaGenome dataset in 2023. In their paper, it has also utilized this dataset (only) for experiments. Our proposed method in this paper outperforms CheXRelFormer and exhibits a significant improvement of 10.7% in the overall F1 score. In future, we will conduct additional experiments using our in-house dataset.
C2. (i) Why accuracy is not reported similar to CheXRelFormer (Table 2) (instead of Mean precision, Mean Recall and Mean F1-score) and (ii) statistical significance should be reported (R4): (i) From the source code provided by CheXRelFormer, the calculation of “mean weighted overall accuracy” reported in CheXRelFormer (Table 2) is actually that of the weighted F1-score in this paper. That means, the evaluation metric is the same as that in this paper. Moreover, this paper provides two additional metrics: macro precision and macro recall. (ii) We agree that statistical significance should be reported, the p-value can be calculated based on results in Table 1 in the manuscript.
C3. Discussion of limitations and computational requirements (R3):
Thanks for the suggestions! Two limitations in this study could be addressed in future research. First, this study only focuses on chest-related disease progression, while it should include a wider range of disease types. Second, this study should conduct further exploration of the model’s ability in the survival analysis task, given its capability to capture both static disease features and dynamic disease changes.
Details of the models: The number of model parameters of CheXRelFormer is 41.0M while ours is 32.2M. The computational complexity (FLOPs) of CheXRelFormer is 20.4G while ours is 9.5G, indicating that our model is superior to CheXRelFormer regarding parameter size and computational complexity.
C4. Access to source code (R3-R6):
The source code will be released for public after acceptance.
C5. Minor typo mistakes (R3-R5):
Thank you for pointing out the typo mistakes. In the Introduction, Paragraph 5 Line 4, the “cam” should be “can”. In Figure 1, X and X’ represent the current and the prior image, respectively. In section 3.1, the “CheXFormer” should be “CheXRelFormer”.
C6. Ablation studies (R6):
Thank you for your suggestions! We conducted ablation studies and reported in section 3.4 in the manuscript to investigate the effectiveness of the Symptom Disentangler module. The results in Table 3 indicate that directly learning disease progression from image-level features without the Symptom Disentangler will lead to a significant performance decline. We also investigated the importance of the symptom labels. The results reported in Table 3 show that even training without the symptom labels, our method still exhibits a notable improvement compared to the baseline. The improvement can be attributed to the newly proposed Symptom Disentangler, which can be trained on only progression labels that indirectly imply the presence of symptoms.
All of the above modifications and discussions will be added to the final version of the manuscript.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
All reviewers have provided either Accepts or Weak Accepts
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
All reviewers have provided either Accepts or Weak Accepts
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
All reviewers have indicated that this paper covers a topic well-worth representing at MICCAI. The progression modeling in chest x-rays is under-explored and the authors address this by separating the symptom recognition from understanding its progression.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
All reviewers have indicated that this paper covers a topic well-worth representing at MICCAI. The progression modeling in chest x-rays is under-explored and the authors address this by separating the symptom recognition from understanding its progression.