Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Accurately discriminating progressive stages of Alzheimer’s Disease (AD) is crucial for early diagnosis and prevention. It often involves multiple imaging modalities to understand the complex pathology of AD, however, acquiring a complete set of images is challenging due to high cost and burden for subjects. In the end, missing data become inevitable which lead to limited sample-size and decrease in precision in downstream analyses. To tackle this challenge, we introduce a holistic imaging feature imputation method that enables to leverage diverse imaging features while retaining all subjects. The proposed method comprises two networks: 1) An encoder to extract modality-independent embeddings and 2) A decoder to reconstruct the original measures conditioned on their imaging modalities. The encoder includes a novel ordinal contrastive loss, which aligns samples in the embedding space according to the progression of AD. We also maximize modality-wise coherence of embeddings within each subject, in conjunction with domain adversarial training algorithms, to further enhance alignment between different imaging modalities. The proposed method promotes our holistic imaging feature imputation across various modalities in the shared embedding space. In the experiments, we show that our networks deliver favorable results for statistical analysis and classification against imputation baselines with Alzheimer’s Disease Neuroimaging Initiative (ADNI) study.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1615_paper.pdf

SharedIt Link: https://rdcu.be/dV1Op

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72069-7_32

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/1615_supp.pdf

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Bae_OCL_MICCAI2024,
        author = { Baek, Seunghun and Sim, Jaeyoon and Wu, Guorong and Kim, Won Hwa},
        title = { { OCL: Ordinal Contrastive Learning for Imputating Features with Progressive Labels } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15002},
        month = {October},
        page = {334 -- 344}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    In this paper, the authors have proposed an ordinal contrastive learning for imputing feature with progressive labels with application of discrimination of progressive stages of Alzheimer’s Disease (AD). The proposed method has been tested on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset, which is an asset to the research community.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    In this paper, the authors have proposed a novel method to solve the missing data problem. The missing data problem will lead to limited sample-size and decrease in precision in downstream analyses. The proposed method can use diverse imaging features while retaining all subjects.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The title of the paper is too broad, since it only has been tested on AD application. The paper is only validated on ADNI dataset, the reviewer would like to know the generalization to other dataset of the proposed method. The methods used for comparson may be out-of-date, i.e., the most recent method has been proposed in 2020. The implementation details of the proposed method and the training process of the model are missing. So the reviewer is worrying about the reproducibility.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    The proposed method has been validated on the publicly available ADNI dataset. So the only issue for reproducibility would be the access to the code.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    1. The title of the paper is to broad, the reviewer would suggest to include its application “AD” to better guide the audiences.
    2. The implementation of the proposed method is missing. The authors will need to describe which open-soucre of frame (i.e., pytorch or tensorflow) work has been used to implement the model.
    3. The author may want to compare the proposed method to recent proposed methods (i.e., published in 2022, 2023, and 2024.)
    4. Some typo need to be fixed e.g., on page 7, “as summarized in () of Fig. 4”.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
    1. The paper solve the missing data problem which is of great importance.
    2. The organization and writing of the paper is great. The paper is easy to follow.
    3. The techniques used in this paper is reasonable and inspiring.
    4. The work has been tested on a publicly available dataset, which can be easily compared by following researchers.
  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    In this work the authors implement a method to impute missing imaging data (region of interest level in brain imaging) and to learn an disease class sensitive embedding to improve classification of dementia stages. They advance on the concept of supervised contrastive learning.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1) Very clearly written and easy to follow paper. 2) Interesting methodology, addressing a relevant problem of imputing missing data and learning a disease severity based embedding in the latent space. 3) Good set of baseline experiments.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1) no mention of code availability 2) one of the baselines (no imputation) uses very little data, could use an alternative baseline here 3) No mention to an earlier paper that introduces supervised ordinal contrastive loss (https://arxiv.org/pdf/2307.12006)

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    Recommend to provide link to github for the code

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    1) The method uses different loss terms. It appears as if they are unweighted. Could a weighted combination of the loss functions improve the method? 2) Acknowledge that the Supervised Ordinal Contrastive Loss was already previously introduced (https://arxiv.org/pdf/2307.12006). It is therefore not a novel contribution in this paper. 3) One of the baseliines (no imputation) uses vastly less data than the other methods. It would be interseting to add a baseline that uses the most abundand modality to train the classifier, to maximize sample size without imputation. 4) can the authors clarify why there are more AMY and FDG PET imaging data compared to Cortical Thickness? ADNI has T1 weighted MRIs for every visit. So this should be the most abundand modality.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall a nice paper. However, the lack of acknowledging the SOCL (https://arxiv.org/pdf/2307.12006) is a major weakness.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    This paper proposes a method to impute unobserved imaging measures of subjects using existing measures. To enable holistic imputation accurately reflecting individual disease conditions, the framework devises modality-invariant and disease-progress aligned latent space using three components: 1) domain adversarial training, 2) maximizing modality-wise coherence, and 3) ordinal contrastive learning. Experimental results on the ADNI study show that the model offers reliable estimations of unobserved modalities for individual subjects.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The method is intersting and novel. The paper is well-organized and very easy to follow. The results show a clear advantage of the proposed method.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The paper is good in general. The figure and tables can be further improved to better understanding.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    1. Does the Domain Adversarial Training need a pretrained classifier for different domains?

    2. The model is trying to Eliminate modality-specific information, but why not decouple it into a modality-specific and a modality-independent one?

    3. The disease progression information is modeled by considering that values of diagnostic label y ∈ {1, · · · , V } that are aligned according to their severity. But how other confounder information, like age, is considered here?

    4. Figure 1 is not self-contained. What are the shapes here? What are the colors for? maximize the similarity of embeddings. What embeddings? Embeddings from different modalities?

    5. Figure 1 (b) : The source modality condition would not make any difference as the Auto-encoder model is automatically to reconstruct the input? Will you use target modality condition during the training of decoder? My concern is Figure 1 (c) maybe would not work after the training.

    6. When the method trying to Maximize modality-wise coherence within a subject, will this also remove the complementary information from different modalities? I guess that is the core for multi-modal learning.

    7. Figure 3 Visualizations of embeddings, can be further improved as first the colors are very hard to distinguish; second there are layers of different classes. One class is over the other, It is hard to see the overlap.

    Minor: The x_k,· denotes the k-th subject, not k-th sample. penalizing strength based on the label distances.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Like above

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #4

  • Please describe the contribution of the paper

    This paper proposes an ordinal contrastive learning method to classify structures of various alzheimer stages brain images modalities (for predictive and diagnosis uses). They introduce an imaging feature imputation method that enables to leverage diverse imaging features while retaining all subjects by using encoder (with the particular loss) and decoder.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The method has been proposed for various biomarkers of the alzheimer disease (Ab, tau, FDG and the cortical thickness). The paper seems reproducible with a clear explanation of the loss and architecture. Losses are compared. Several architectures from the litterature are compared with the new methodology. Illustrations are readable and understable.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The main weaknesses consist in the validation of the method (see 10) and its reproducibility (see 9).

  • Please rate the clarity and organization of this paper

    Excellent

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    Paragraphs describing the architecture are well described, illustrations are readable and understadable. However, an open source code is missing, and the framework, exploitation system, memory resources and time execution are lacking to reproduce the implementation of this article.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    The t-SNE groups may be compared with distance calculations for instance in Figure 3 in future works. Here only a qualitative comparison of the various loss embedding have been made.

    The illustration of classification for various architectures (listed in Table 2) may be also illustrated and verified as in Figure 4 to compare the results in the same way.

    There is a lack of informations such as the framework to implement the architecture, the exploiting system or time execution to reproduce this work.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper provide a new way to classify biomarkers of a neurodegenerative disease using an ordinal contrastive learning method. It compares fairly the new architecture with some architectures from the litterature.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

We thank the reviewers for their constructive comments and generous scores.

[Q] Reproducibility (R1, R3, R4, R5) [A] The code will be released, which can be used for any multi-modal neuroimaging analysis with ordinal labels. ADNI data is publicly available but ROI-wise measures may slightly vary depending on pre-processing pipeline. [Q] Broad title (R1) [A] Although OCL is a general method applicable to various datasets, we will mention in the title that this paper is for Alzheimer analysis. [Q] Out-of-date baselines (R1) [A] The baselines in the current version may be traditional but still serve as benchmark references in most imputation studies. We have found HyperImpute [1] as a recent literature, and we anticipate that it would yield results similar to existing baselines we used as demonstrated in [1]. [Q] Decoupling modality-specific and -independent contents (R3) [A] While decoupling modality-specific and -independent contents is a feasible idea, the modality-specific contents become redundant in our model as it does not require modality-specific information when generating a target modality from an existing one. [Q] Potential impact of confounders (R3) [A] This is a good question. As in other deep models, our model does not directly deal with confounders assuming the disease-specific effect is the most dominant. In the embedding stage, as we discard modality-specific contents only, we expect the confounding effects, e.g., age and sex, to be naturally incorporated in the generation stage. [Q] Figure clarification (R3) [A] We consistently use shapes to represent modalities and colors to represent labels in every figure. However, we will clarify this in Figure 1. In Figure 3, while the overlap may not be apparent with numerous (a) training samples, the overlap is more evident with sparse samples in (b) test data. [Q] Training mechanism (R3) [A] If different modalities share similar embeddings, the target modality with a similar context can be generated by changing the condition of modality during inference. This information is given in section 2.2. [Q] Further analyses (R4) [A] Due to page limit, only preliminary results are presented. We hope to present extended analysis, including group-wise distances from each method per the reviewer’s suggestion, in a journal version in the near future. [Q] Weighted combination of loss functions (R5) [A] We experimented with different combinations of losses, but the results were only marginally better than our current unweighted combination. [Q] Comparison to SCOL [2] (R5) [A] While SCOL suggests a similar concept, our method is designed in a totally different and more sophisticated way: (1) The mere addition of distance metrics to similarity scores in SCOL may result in the shrinkage or dispersion of the embedding space over prolonged training periods. Conversely, we first assign different temperatures for each negative sample for gradient calculation rather than simple addition. Then, we assign the temperature of positive samples to preserve the embedding space deterministically by gradient analysis as detailed in the supplementary material. (2) While SCOL adopts L2 distance for the label distance metric, we choose L1 distance. In scenarios with numerous labels, adding the distance within the exponential function could introduce errors or lead to overfitting. [Q] Why ‘no imputation’ baseline? (R5) We need the baseline to validate if the imputation helps a downstream task, i.e., no imputation vs. imputation. [Q] Why more AMY and FDG than CT? (R5) [A] We have processed a subset of ADNI data to study the effect of PET measures, which resulted in less number of MRI samples. The processing is still ongoing.

[1] Jarrett, et al., “Hyperimpute: Generalized iterative imputation with automatic model selection.”, ICML, 2022. [2] Saleem, et al., “SCOL: Supervised Contrastive Ordinal Loss for Abdominal Aortic Calcification Scoring on Vertebral Fracture Assessment Scans”, MICCAI, 2023.



Meta-Review

Meta-review not available, early accepted paper.



back to top