Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Prenatal drug exposure, which occurs during a time of extraordinary and critical brain development, is typically associated with cognitive, behavioral, and physiological deficits during infancy, childhood, and adolescence. Early identifying infants with prenatal drug exposures and associated biomarkers using neuroimages can help inform earlier, more effective, and personalized interventions to greatly improve later cognitive outcomes. To this end, we propose a novel deep learning model called disentangled hybrid volume-surface transformer for identifying individual infants with prenatal drug exposures. Specifically, we design two distinct branches, a volumetric network for learning non-cortical features in 3D image space, and a surface network for learning features on the highly convoluted cortical surface manifold. To better capture long-range dependency and generate highly discriminative representations, image and surface transformers are respectively employed for the volume and surface branches. Then, a disentanglement strategy is further proposed to separate the representations from two branches into complementary variables and common variables, thus removing redundant information and boosting expressive capability. After that, the disentangled representations are concatenated to a classifier to determine if there is an existence of prenatal drug exposures. We have validated our method on 210 infant MRI scans and demonstrated its superior performance, compared to ablated models and state-of-the-art methods.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3719_paper.pdf

SharedIt Link: https://rdcu.be/dY6fB

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72390-2_7

Supplementary Material: N/A

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Che_Disentangled_MICCAI2024,
        author = { Cheng, Jiale and Wu, Zhengwang and Yuan, Xinrui and Wang, Li and Lin, Weili and Grewen, Karen and Li, Gang},
        title = { { Disentangled Hybrid Transformer for Identification of Infants with Prenatal Drug Exposure } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15012},
        month = {October},
        page = {67 -- 76}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper presents a novel hybrid framework combining volume and surface-based transformers for diagnosing prenatal drug exposure. The objective is to effectively capture both unique and shared characteristics across non-cortical and cortical regions, thereby improving classification performance.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1) The paper is clear and well motivated. 2) The proposed method is interesting and the results are good and seem significant. 3) The results show not only the means but also the std for each metrics. 4) This work provide a new strategy for neuroscience learning,

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1) The link of source code and data public are not provide. 2) More detailed activate region may need further clarification and more statistic should be included. 3) The appearance of surfaces from various ages may appear similar, yet the T1 and T2 data from these distinct ages exhibit significant disparities. Such variations can profoundly impact the visual style perceived by ViT at different ages, potentially compromising model stability. How did the authors address this challenge?

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    The reproducibility of this paper is good except for the link of the code that is not provided.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    To improve this paper, here are my suggestions: 1) The number of parameters and computation time of all the methods studied should be added. 2) The link for the code should be added. 3) The abalation analysis of influence from distinct appearance at differnt infant develop phases should be clarify.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Even if the paper suffers from some weaknesses, I find it interesting, with an interesting application case, and with convincing results.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    The paper proposes a disentangled hybrid volume-surface transformer model that combines 3D MRI and cortical surface features using transformers. The results demonstrate promising performance in identifying infants with prenatal drug exposure from MRI and surface data.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • This paper is well-written and easy to follow.
    • Combining multi-modal data to predict the drug exposure is interesting
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • The dataset is a bit small.
    • The experiments can be imporved
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    I suggest the authors release their code to imporve the reproducibility.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    • The multiplication symbol (for representing the concatenation) in Figure 1 is misleading. Please revise it.
    • Figure 1 shows the specific code has been average, which is a discrepancy to the text. Please clarify.
    • The formula of the disentanglement loss is a bit tricky. I feel it will lead to a local minima during optimization by forcing the common code to be as small as possible (even though the ablation study shows some evidence that it can help boost the performance)
    • It would be nice to compare other hybrid multi-modal approaches.
    • What is the meaning of the number after plus and minus signs in Tables 1 and 2?
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall, this paper is good without any major weaknesses.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    This paper proposes a novel disentangled volume- and surface-transformer classifier model for identification of infants with prenatal drug exposure. The authors validate their proposed method on a cohort of MRI scans of 210 infants with prenatal opioid exposure and show that they achieve better results than other volume-only or surface-only SOTA methods. Moreover, they conduct an ablation study to showcase the importance of including both surface and volume data, as well as the proposed disentangled latent codes.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1. The idea of designing a hybrid transformer model, together with the latent code disentanglement of common and specific features (between surface and structural MRI volumetric data), is novel.

    2. Moreover, it is an interesting and novel way of combining multimodal data, thus leveraging complementary information from the volume and surface datasets.

    3. The paper is well written and clear.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1. One of the most interesting contributions of this paper is the disentanglement itself. Although the authors do provide illustrations of the discriminative regions, I think the paper would benefit from an extra figure visualising the learned latent space.

    2. The introduction has a very short section on prior work, which I recommend expanding.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    1. Please add some of the notations used in Section 2.2 to Figure 1 to aid understanding (e.g., are Oi and Os the “Transformer” blocks in the Figure?)

    2. Can you provide a bit more information on the architecture itself? What size is the dimension of the unified embedding space (i.e., C)? I think this would aid reproducibility of your work.

    3. The analysis on the discriminative regions seems very important to me. Could you possibly produce these as averages across controls vs. patients, instead of just 2 subjects? Would there be differences in what the network is looking at in controls vs. what it is looking at in patients?

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I believe this is an interesting and novel paper and the MICCAI community would benefit from discussions around this topic.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

N/A



Meta-Review

Meta-review not available, early accepted paper.



back to top