Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Emphysema is defined as an abnormal alveolar wall destruc- tion exhibits varied extent and distribution within the lung, leading to heterogeneous spatial emphysema distribution. The progression of emphysema leads to decreased gas exchange, resulting in clinical worsening, and has been associated with higher mortality. Despite the ability to diagnose emphysema on CT scans there are no methods to predict its evolution. Our study aims to propose and validate a novel prognostic lobe-based transformer (LobTe) model capable of capturing the complexity and spatial variability of emphysema progression. This model predicts the evolution of emphysema based on %LAA-950 measurements, thereby enhancing our understanding of Chronic Obstructive Pulmonary Disease (COPD). LobTe is specifically tailored to address the spatial heterogeneity in lung destruction via a transformer encoder using lobe embedding fingerprints to maintain global attention according to lobes’ positions. We trained and tested our model using data from 4,612 smokers, both with and without COPD, across all GOLD stages, who had complete baseline and 5-year follow-up data. Our findings from 1,830 COPDGene participants used for testing demonstrate the model’s effectiveness in predicting lung density evolution based on %LAA-950, achieving a Root Mean Squared Error (RMSE) of 2.957%, a correlation coefficient (ρ) of 0.643 and a coefficient of determination (R2) of 0.36. The model’s capability to predict changes in lung density over five years from baseline CT scans highlights its potential in the early identification of patients at risk of emphysema progression. Our results suggest that image embeddings derived from baseline CT scans effectively forecast emphysema progression by quantifying lung tissue loss.



Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2275_paper.pdf

SharedIt Link: https://rdcu.be/dVZiQ

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72378-0_52

Supplementary Material: N/A

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Cur_Lobar_MICCAI2024,
        author = { Curiale, Ariel H. and San José Estépar, Raúl},
        title = { { Lobar Lung Density Embeddings with a Transformer encoder (LobTe) to predict emphysema progression in COPD } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15001},
        month = {October},
        page = {557 -- 566}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The authors introduce a method for predicting the development of emphysema. COPDGene CTscabs and available lobe segmentations were used. A prognostic lobe-based transformer (LobTe) model was trained and validated (split: 60:40). LobTe was able to predict the lung density evolution base on %LAA-950 with Mean Squared Error (RMSE) of 2.957% and a coefficient of determination (R^2) of 0.643.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • A method that can predict the progression of emphysema would potentially offer opportunities to treat COPD in a more personalized way.
    • A fairly large number of scans were used for training (n=2782) and testing (n=1830).
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • Maybe I misunderstand Figure 3, but you can clearly see, e.g. in the linear fit plot, that for many patients a decrease in %LAA-950 is predicted and you can clearly see that for many patients a decrease is actually measured. I am very surprised by the number of patients for which %LAA-950 is decreasing - given the fact that emphysema is defined as irreversible parenchymal destruction. However, a quick literature search made it clear to me that this distribution can be true. The authors should e.g. cite https://doi.org/10.1148/radiol.213054 and clarify this. This is of great importance in order to be able to better understand the development and also the prediction of %LAA-950.
    • Used CT data needs a more detailed description. Scans were done with comparable dose? Same reconstruction kernels? The quality of the data and its comparability is of fundamental importance. This should be explained (briefly). COPDGene data is used - clarification should be no problem.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?
    • What registration method was used? Was the influence of the registration method on the whole procedure investigated?
    • LEP: Have other pixel region sizes been tried? Or just 32x32? Why this size?
    • Figure 1 should be improved: Give some information about the patient, %LAA-950 at baseline and at follow-up. Marked emphysema should be shown.
    • Figure 3 needs explanation, the average %LAA-950 with SD for baseline and follow-up scans should be given – for follow-up the values for co-registered and not co-registered CT scans. Maybe it would help if the points in the fit plot were differentiated by color according to No COPD, mild and severe COPD. Or plots for the different groups.
  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    • What registration method was used? Was the influence of the registration method on the whole procedure investigated?
    • LEP: Have other pixel region sizes been tried? Or just 32x32? Why this size?
    • Figure 1 should be improved: Give some information about the patient, %LAA-950 at baseline and at follow-up. Marked emphysema should be shown.
    • Figure 3 needs explanation, the average %LAA-950 with SD for baseline and follow-up scans should be given – for follow-up the values for co-registered and not co-registered CT scans. Maybe it would help if the points in the fit plot were differentiated by color according to No COPD, mild and severe COPD. Or plots for the different groups.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Predicting the progression of emphysema is an interesting task per se – but where it gets really interesting is in the “No COPD” group, i.e. will healthy lung tissue develop emphysema.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    The paper introduces a novel predictive model, LobTe, which utilizes a transformer encoder and lobe embedding fingerprints to predict the progression of emphysema based on %LAA-950 measurements from CT scans, achieving high predicting accuracy. This offers a significant advancement in the ability to predict emphysema progression, potentially aiding in early diagnosis and tailored treatment strategies for individuals at risk.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The paper propose a novel problem: predicting the progression of emphysema, and validate the feasibility of this problem via the proposed method. The author also introduces the LobTe model as a novel approach.
    2. Experiments shows that such prediction is accurate and can provide valuable guidance for clinical assessments and treatment planning.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. Although the author claims that their method is unique and cannot be compared with other methods. the problem itself, an image regression problem, is not so complex. The author should provide the results achieved by a simple CNN or transformer as a baseline to enhance the contribution of LobTe.
    2. The technological details of the LobTe are not clearly presented in the Method section, especially for the development of the Local Density Model. The author should provide a more detailed description of this section, or provide the source code to improve its reproducibility.
    3. The definition of Local Emphysema Progression seems to lack more related work support and descriptions of clinical significance.
    4. LobTe is performed on the 2D CT slices. However, for one CT scan, there are many CT slices. The author should claim how they chose which slice will LobTe work on.
    5. The author should provide the environment and the expected running time of LobTe to increase its clinical utilization.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    1. The paper would benefit greatly from a comparative analysis with existing models, although existing methods may not be applicable to this specific problem. This will help clarify the contribution of the Local density model and Lobe Embedding Fingerprint. It would be better if the results of baseline and ablation experiments could be provided.
    2. The author should provide more details about the model or release the code.
    3. The author should add some data, preferably from an external study, to increase the credibility of the conclusion, although this is difficult. 2000 cases could be too small for 2D medical image processing, and overfitting could also exist.
    4. For the progression study, the possibility from No COPD to Mild COPD, and from Mild COPD to Severe COPD, can have larger significance. The author should show the accuracy of such prediction achieved by LobTe.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Although this article is not technically innovative from a technical perspective and the experimental part needs further improvement, it raises and resolves an issue of clinical concern. The article shows the feasibility of solving such clinical problems and the generalizability to other similar issues. I still think this is a good article.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    Implementation of transformer based models to predict the progression of emphysema in COPD based on CT scans.

    Contribution: Novel application,authors claim that no other publication has yet worked on predicting changes in emphysema based on %LAA-950

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper addresses a highly relevant medical use case that has not yet been sufficiently researched. The clinical relevance and motivation of this application are presented well and comprehensibly in the paper. It is therefore a novel application and an original way of using the available data.

    A data-efficient training was used in order to be able to use transformer-based models even with a less extensive data set. An interesting approach also for other use cases.

    Complex network architecture, which consists of different stages, has been clearly illustrated in a graphic, which is very helpful for understanding the basic pipeline. The architecture used in the paper is described well and comprehensibly.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    There is no baseline for the approach presented. The authors justify this by stating that no other publications have dealt with this use case to date: “Despite the ability to diagnose emphysema on CT scans there are no methods to predict its evolution.” After a brief search, I came across the paper “CT Imaging With Machine Learning for Predicting Progression to COPD in Individuals at Risk” (https://www.sciencedirect.com/science/article/abs/pii/S0012369223008097). Could this be a possible baseline to better evaluate the performance of the models? (Reviewer does not have access to the paper mentioned). Otherwise, the implementation of a standard CNN model as a baseline or an ablibation study would have been useful to better assess how relevant the individual steps of the model are and how good the performance is.

    The split of the data and its preprocessing are not sufficiently (and partly incorrectly) described. on page 2, 2,782 training samples are mentioned, from page 4 onwards 984 training samples are used for training the local density model, and the remaining 2,226 for fitting the LobTe model. This would result in 3210 training samples instead of the originally mentioned 2782. It would make sense to discuss the split information in the first section of the materials and methods chapter instead of spreading it throughout the paper. The split of the data is unusual, only about 60% is used for training, from which a small validation set is formed. No further justification is given for this decision.

    The choice of hyperparameters is not justified; no search is described. It is not possible to assess whether this is a complex procedure or the originally selected parameters.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    Despite the more complex training, the authors unfortunately do not share their code. Together with an unclear data split, this means that the results cannot be reproducible for other researchers, or only with great difficulty.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Check the paper for format errors, e.g. the short title is suppressed due to excessive length, there are typos (even in the header of table 2)

    Check the data split information again (see main weaknesses of paper) and mention all data splits in one abstract, this can lead to a better overview. Also discuss the distribution of the dataset regarding the GOLD scores, to better understand results and training behaviour.

    Include for a journal paper more insights into the choice of hyperparameters

    Include for the journal paper also a section where you describe the different evaluation steps before discussing the results, this could lead to a better understanding of the evaluation. While the materials and methods section was easy to follow, the results section was not described as well, e.g.Table 1 is not discussed but only mentioned, and it is difficult to follow the different evaluation steps.

    The abstract mentions technical terms like %LAA-950, which are not defined in the abstract and are not common for the majority of researchers.

    A standardized representation of numerical values with regard to the decimal place would be good.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The missing baseline or ablation study makes it difficult to fully evaluate the performance of the given complex solution

    The not yet resolved errors regarding the data split and number of training samples maybe lead to further problems, and underline the missing insights of the paper regarding the dataset and it’s split

    However the paper is discussing an interesting and novel use case and a well-described data-efficient machine learning solution.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

N/A



Meta-Review

Meta-review not available, early accepted paper.



back to top