Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

The dynamic 3D shape of a cell acts as a signal of its physiological state, reflecting the interplay of environmental stimuli and intra- and extra-cellular processes. However, there is little quantitative understanding of cell shape determination in 3D, largely due to the lack of data-driven methods that analyse 3D cell shape dynamics. To address this, we have developed MorphoSense, an interpretable, variable-length multivariate time series classification (TSC) pipeline based on multiple instance learning (MIL). We use this pipeline to classify 3D cell shape dynamics of perturbed cancer cells and learn hallmark 3D shape changes associated with clinically relevant and shape-modulating small molecule treatments. To show the generalisability across datasets, we apply our pipeline to classify migrating T-cells in collagen matrices and assess interpretability on a synthetic dataset. Across datasets, our pipeline offers increased predictive performance and higher-quality interpretations. To our knowledge, our work is the first to utilise MIL for multivariate, variable-length TSC, focusing on interpretable 3D morphodynamic profiling of biological cells.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/3839_paper.pdf

SharedIt Link: https://rdcu.be/dV546

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72117-5_45

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/3839_supp.pdf

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{De_Interpretable_MICCAI2024,
        author = { De Vries, Matt and Naidoo, Reed and Fourkioti, Olga and Dent, Lucas G. and Curry, Nathan and Dunsby, Christopher and Bakal, Chris},
        title = { { Interpretable phenotypic profiling of 3D cellular morphodynamics } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15010},
        month = {October},
        page = {481 -- 491}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper presents an interpretable pipeline for analysing and classifying 3D cell shape dynamics using multiple instance learning (MIL). This approach aims to quantify the changing shapes of cells, providing insights into the physiological states and responses to environmental stimuli. The authors apply this pipeline to two distinct datasets: perturbed cancer cells and migrating T cells, demonstrating its generalisability. They claim their method offers increased predictive performance and higher-quality interpretations compared to previous approaches.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Novel Approach: The use of MIL for multivariate, variable-length time series classification (TSC) is an interesting method for analyzing 3D cell shape dynamics. It provides a new perspective for understanding cellular behavior and could be valuable in various biomedical applications.

    Generalizability: The authors have applied their pipeline to different datasets, showing that the approach can work across diverse types of cell shape dynamics. This cross-application is crucial for demonstrating the method’s versatility.

    Interpretability: The focus on interpretable 3D morphodynamic profiling is commendable, as it provides a way to understand the underlying mechanisms behind cell shape changes.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Lack of Exceptional Results: The paper indicates that the results achieved by the proposed pipeline are not significantly better than existing approaches. This lack of dramatic improvement limits the impact of the work.

    Poor Manuscript Organization: The structure of the manuscript could be improved. Key information about metrics and quality assessment appears in the results section, which could confuse readers. A clearer organization would enhance the readability of the paper.

    Inadequate Table and Figure: The captions for tables and figures lack sufficient detail, making it difficult to understand the data presented. More descriptive captions are needed to guide the reader through the findings.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    The authors could consider to make public the source code for interpretable reproducibility of their framework.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Reorganize Manuscript Structure: Restructure the paper to ensure a logical flow of information, separating key aspects such as metrics and quality assessment from the results section. This will improve readability and understanding.

    Enhance Table and Figure Captions: Provide more detailed captions for tables and figures to facilitate a clearer interpretation of the data. This is crucial for conveying the paper’s findings effectively. Moreover, Fig. 2 needs to be increased in size.

    Address Similarities with Existing Work: Clarify the unique contributions of this work compared to existing studies. Highlight the specific differences that make this pipeline novel and valuable in its domain. The author need to make very clear their contribution compare with other studies especially the https://arxiv.org/pdf/2311.10049.pdf.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The proposed pipeline for analyzing 3D cell shape dynamics through MIL offers an interesting approach with a focus on interpretability. However, the lack of significantly better results ( in classification), poor manuscript organization, and inadequate table and figure captions detract from its overall impact. Moreover, the potential overlap with existing work—without clearly distinguishing contributions in comparison to the state-of-the-art—further undermines the paper’s uniqueness and value.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    This manuscript proposed an interpretable, variable-length multivariate time series classification (TSC) pipeline based on multiple instance learning (MIL). This pipeline to classify 3D cell shape dynamics of perturbed cancer cells and learn hallmark 3D shape changes associated with clinically relevant small molecule treatments

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1. Interpretable framework that can be used for multivariate, variable length time series classification.

    2. Application of this framework to the 3D cellular morphodynamic classification of drug-treated cancer cells in collagen matrices, learning the hallmark shape changes induced by clinically relevant cancer therapies.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    none

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    none

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    none

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    1. Interpretable framework that can be used for multivariate, variable length time series classification.

    2. Application of this framework to the 3D cellular morphodynamic classification of drug-treated cancer cells in collagen matrices, learning the hallmark shape changes induced by clinically relevant cancer therapies.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    This paper presents an innovative framework for the interpretable phenotypic profiling of 3D cell morphodynamics, overcoming challenges associated with variable-length multivariate time series classification (TSC). The core contribution of the paper is the development of a model that combines graph transformers and conjunctive pooling modules, significantly enhancing predictive performance without compromising interpretability. The authors validated the effectiveness of the model on synthetic and two biological datasets, demonstrating its superiority over existing technologies through quantitative metrics and visualization results.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper introduces a novel framework that utilizes Multiple Instance Learning (MIL) for multivariate, variable-length time series classification of 3D cell shapes. This approach addresses the challenge of analyzing variable-length time series without the need for padding or cropping, which can distort the natural dynamics of cellular processes. The use of MIL allows for interpretation at both the instance and bag levels, facilitating detailed insights into how specific changes in cell morphology relate to biological processes or drug treatments.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1. The paper mentions the use of Deep Graph Convolutional Neural Networks (DGCNN) and SimCLR for feature extraction. While effective for point cloud data, the paper lacks a thorough analysis of how these techniques perform across different cell types or under varying data characteristics. Future work should explore the adaptability and sensitivity of these techniques on a broader range of cell types and other biological datasets to assess the model’s generalizability and robustness.

    2. The model structure involves multiple layers of graph transformers and conjunctive pooling operations, potentially introducing high computational complexity, especially in large-scale data processing. The paper should discuss the computational demands in more detail, including training time and required hardware resources. Developing more efficient algorithmic versions, such as optimizing the graph structure or simplifying the attention mechanism, is recommended to reduce computational resource consumption and enhance practical application value.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    1. Although the model demonstrated good classification and interpretability performance on multiple datasets, discussions on statistical significance are lacking. Future work should include more statistical tests, such as calculations of confidence intervals and hypothesis testing, to further validate the statistical significance and reliability of the conclusions.
    2. The paper primarily focuses on short-term cellular morphological changes, with insufficient analysis of long-term dynamics. Future research should consider integrating long-duration tracking data to analyze the effects of prolonged drug treatment on cell morphology, which would help deepen the understanding of drug mechanisms and guide clinical applications.
    3. Although the paper demonstrated model interpretability through visualization techniques, it lacks user studies or biological validations to support the biological significance of the interpretations. It is recommended that future work collaborate with cell biology experts to experimentally validate the accuracy and relevance of the model’s interpretations, thus enhancing the biological value of the model’s explanations.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Accept — should be accepted, independent of rebuttal (5)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I value the design motivation of the method and the logical coherence of the article more than the operation and results.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

We genuinely thank all the reviewers for their positive feedback and constructive criticism. We appreciate the reviewers recognising this work as innovative and novel, which could help deepen the understanding of drug mechanisms in discovery endeavours. We address a few points below:

  • Significance of results: Reviewers suggested statistical significance tests of classification results. We have performed paired t-tests comparing our proposed IGT method with other methods (LSTM, TransMIL, and GTP) for ACC and AUC metrics across datasets. On the melanoma dataset, IGT significantly outperformed LSTM (p=1.7e-15), TransMIL (p=0.0002), and GTP (p=0.001) in terms of AUC. IGT significantly outperformed TransMIL (p=0.001) and GTP (p=0.0457) in ACC on this dataset. On the T cell dataset, IGT significantly outperformed LSTM (p=0.0423), GTP (p=0.003), and TransMIL (p=0.123) in ACC. Regarding AUC on this dataset, IGT outperformed LSTM (p=0.011) and TrasMIL (p=0.026). IGT has no accuracy-interpretability trade-off, as we have outperformed in both.
  • Manuscript organisation, table and figure legends: We appreciate this constructive feedback and have reorganised our manuscript as suggested. We introduced the evaluation metrics before the results and updated our figures and tables to be more explicit with informative legends.
  • Unique contributions: We have clarified our differences from previous work, highlighting our application of graph transformers and MIL to variable-length multivariate TSC of 3D morphodynamics. We also highlight the unique novelty of our graph construction in accounting for cell cycle shape similarities and using graphs to handle variable lengths. Our similarity to previous work is in our conjunctive pooling operation used. Other than this, our method is novel. Furthermore, to our knowledge, this is the first work to classify 3D cellular morphodynamics interpretably, which is fundamental for drug discovery applications. Although we compare our model to existing MIL models for interpretable variable length multivariate TSC, these are not necessarily existing pipelines for this task. Neither TransMIL nor GTP have been used to classify 3D shape dynamics of biological cells or any time series data. Our problem formulation, as well as our model, is novel. This novel perspective can pave the way for innovative interpretable methods in studying cellular behaviour, ultimately advancing our understanding of complex biological processes.
  • Computational costs: Reviewers suggested a discussion on the computational complexity. All models were trained on a single Nvidia Quadro RTX 6000 GPU with 24GB of memory. Our final IGT model on the WM266.4 (Melanoma) dataset had 186,771 trainable parameters, with the 2-layered graph transformer module having 184,192 trainable parameters and the conjunctive pooling operation having 2,322. On average, training the IGT for one epoch on the WM266.4 (Melanoma) dataset took 4.91 seconds (std = 0.48) with an average inference time of 3.19 s (std = 0.11).
  • Generalisability of DGCNN: Reviewers suggested future work to analyse the generalisability of DGCNN. We appreciate this comment and agree on its importance. We believe that we may not have been evident in the original version of the manuscript in the fact that the feature extractor, DGCNN, was trained initially as an encoder in an autoencoder with FoldingNet as the decoder on a dataset of 3D melanoma cells fixed in collagen matrices, and then fine-tuned using SimCLR on the live melanoma dataset (two different datasets). This DGCNN encoder was then fixed and used on an entirely different (publically available) dataset of intravital two-photon images of migrating T cells in various parts of mice anatomy and our synthetic shapes dataset, showing that DGCNN learns shape features in a generalisable manner. Future work: All reviewers have suggested excellent ideas for future work, which we will discuss in detail in the final version of the paper.



Meta-Review

Meta-review not available, early accepted paper.



back to top