Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Generative models, such as GANs and diffusion models, have been used to augment training sets and boost performances in different tasks. We focus on generative models for cell detection instead, i.e., locating and classifying cells in given pathology images. One important information that has been largely overlooked is the spatial patterns of the cells. In this paper, we propose a spatial-pattern-guided generative model for cell layout generation. Specifically, a novel diffusion model guided by spatial features and generates realistic cell layouts has been proposed. We explore different density models as spatial features for the diffusion model. In downstream tasks, we show that the generated cell layouts can be used to guide the generation of high-quality pathology images. Augmenting with these images can significantly boost the performance of SOTA cell detection methods. The code is available at https://github.com/superlc1995/Diffusion-cell.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2613_paper.pdf

SharedIt Link: https://rdcu.be/dY6iY

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72083-3_45

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/2613_supp.pdf

Link to the Code Repository

https://github.com/superlc1995/Diffusion-cell

Link to the Dataset(s)

https://github.com/TopoXLab/Dataset-BRCA-M2C

BibTex

@InProceedings{Li_Spatial_MICCAI2024,
        author = { Li, Chen and Hu, Xiaoling and Abousamra, Shahira and Xu, Meilong and Chen, Chao},
        title = { { Spatial Diffusion for Cell Layout Generation } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15004},
        month = {October},
        page = {481 -- 491}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper proposes a learning-based generative method that can learn spatial distributions of positions (of cells in a plane in this case; the distributions are termed in the paper as “layout maps”) and that can generate new samples (sets of spatial positions) from the distributions. The learning of the distributions honours the fact that position density is not the same (uniform) everywhere in the images.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    • Important problem. I agree with the Authors that most of the generative approaches focus on textures of cells (a cell shape must be given), less also focus on the shape (that will be “texturized” afterward) and very few also on the “placement” of the shapes within an image. The latter is exactly at the heart of the paper.

    • The method is based on diffusion models.

    • The paper shows a complete pipeline, that is from generating the layout, to generating realistically looking images with controlled density of cells.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    • For me the presentation: I wasn’t always sure if I understand the sentence, and overall I had to read the paper several times to finally grasp everything.

    • The evaluation (Table 1), in fact, measures the performance of a full pipeline, which is three networks (layout generator, texture generator, detection network). The outcome of the detection network is reported, and it is not guaranteed that any improvement is predominantly because of the better generated layout.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    Since most format restrictions on papers (true also here) does not allow to provide all details, I would ask the Authors to at least share their source code and models. I understand it is harder in double-blind review, but perhaps a sentence, e.g., “The source codes can be find at …..published after review……” would have been a good signal for the reviewers, provided sharing the code is actually really the intention of the Authors…

    I don’t believe I would be able to reproduce the reported study from this paper alone.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Please, fix the “Keywords:” line.

    Please, explain the headers in Table 1. I suspect the “Stro.” refers to Stromal cells, to their specific distribution. What are the other columns? Can you please, at least, explain in the Table caption. The numbers in the table can then be detection accuracy on those particular cells, but what is Det then? Is it the DET measure from the CellTrackingChallenge.net?

    In the Supplementary Fig. 1, it would have been much better to overlay each sub-figure with a letter A), B), C), etc. and refer to the sub-figures using these letters, rather than as “right medium” (btw, medium -> middle?).

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I like the topic and the approach to the topic, but the manuscript would greatly benefit from a proper proof-reading, perhaps even with language correction, and the reproducibility in the current manuscript is very difficult to achieve.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • [Post rebuttal] Please justify your decision

    I thank the Authors for their rebuttal. I rely on their promise to carefully revise the readability of the manuscript and that they would provide at least the details explained in their rebuttal. I agree that spatial layout generating networs are needed.



Review #2

  • Please describe the contribution of the paper

    Paper propose interesting problem of using diffusion based model for improving cell detection using density map.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1) Using diffusion to generate cell layout with count conditioning is an interesting application of diffusion models. 2) Paper propose novel spacial-FID metric for evaluating layouts.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1) The writing in the paper can be significantly improved to provide a broader perspective of the problem, as the motivation behind it is not clear. 2) Spatial FID is calculated using an autoencoder, providing details on convergence and the size of the training dataset would be helpful. Additionally, major part of the layout is background, will FID will be able to give a good indication is similarity? 3) While generating cell images is commendable, augmentation using these generated images doesn’t seem to substantially improve performance. For example, MCSpatNet’s accuracy increased only from 0.849 to 0.855. 4) Does the model consider the density of various cell types when generating the density map? This isn’t clearly explained in the text. 5) Can authors comment on the cell shape and neighbourhood of the generated images?

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    NA

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    1) The paper can be restructured to improve readability. 2) The comparison of various density estimation methods takes up too much space, which could be used more efficiently elsewhere. 3) The paper would be improved if its generalisability could be demonstrated on other cell detection datasets, such as: [1]: Hover-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images [2]: A Multi-Organ Nucleus Segmentation Challenge [3]: DeGPR: Deep Guided Posterior Regularization for Multi-Class Cell Detection and Counting.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    I liked the idea of using diffusion based models to generate the cell layout. However, writing of the paper can be improved to highlight the main contribution. Additionally, applicability of the method is shown on single dataset, with marginal gain in performance. Extending the idea to at least 2 dataset will strengthen the paper.

  • Reviewer confidence

    Somewhat confident (2)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Reject — could be rejected, dependent on rebuttal (3)

  • [Post rebuttal] Please justify your decision

    I thank authors for the rebuttal. I feel paper will benefit from the another round of revision.



Review #3

  • Please describe the contribution of the paper

    This paper proposes diffusion model based cell layout generation with cell annotation masks and estimated cell density maps. The layout maps are then used to generate realistic histopathology images. The proposed method was evaluated on a breast cancer dataset and the generated cell images were further leveraged as augmented data for cell detection and classification of the dataset.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Despite that the idea of layout map generation using cell density maps is somewhat similar to TMCCG (Bousamra, et al CVPR, 2023; reference No 2 in this submission), the application to diffusion model based generation of cell images is novel.
    • Evaluation and ablation studies are comprehensive.
    • The paper is well organized and easy to read.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • For MCSpatNet, the baseline model tailored for cell detection, the performance with the best of the proposed methods was identical with that of TMCCG (Bousamra, et al CVPR, 2023): classification mean 0.667 vs 0.669. With GMCM, the performance was lower than that of TMCCG and identical to the baseline itself. Hard to tell if the proposed methods are really better than TMCCG.
    • Different from the results for MCSpatNet, the other worse performing baseline, U-Net, benefitted more from the proposed method than that of MCSpatNet. Would this mean the proposed method would not be as helpful for the more optimally designed baseline cell models like MCSpatNet?
    • Not much analysis, justification and discussion on the different performances with different density estimation models.
    • The proposed “spatial-FID” is not new in terms of formulation and design: the authors switched the feature extractor used to calculated FID from off-the-shelf models to custom trained cell density based model.
    • There seems a lack of key details about the diversity of generated layout maps and/or generated realistic images. Including more examples may help with better assessment.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    There’s no mention about source code but a public dataset is used. No implementation details provided in terms of training hyperparameters, hardware, training time and resource need.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    • For main concerns, please refer to the weakness section.
    • For Table 1 comparison of cell detection/classification results, the authors are suggested to include repeats with mean and standard deviations. As mentioned in the weakness section, it is hard to assess whether the proposed methods are better than TMCCG.
    • Observing the different levels of benefits the proposed methods provide with the two baselines, it would help further assessment if the authors can include additional baseline models designed for cell modeling and/or apply the proposed methods on additional datasets that are more challenging and with larger sample sizes, for example, PanNuke (J Gamper, et al, 2020).

    Minor:

    • Typo in Figure 1 legend: “For clearance” -> “For clarity”
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall this paper propose relatively novel designs for diffusion modeling of cell layout generation and demonstrated potential for realistic cell image generation to augment cell modeling data. The main concern was the unclear advantage over previous reports (TMCCG).

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

Thanks for the insightful reviews. Source code will be made public upon acceptance. We will follow the suggestions of R3&R4 to improve the readability. We will include the new results on CoNSeP for cell detection, more generated examples, and repeats with mean and standard deviation in the revised version.

Q1: Performance improvement. (R1, R4) Our method performs better than TMCCG for both U-Net and MCSpatNet. For U-Net, the means of classification accuracy are 0.666 vs. 0.644. For MCSpatNet, our method improves the performance on all three classes consistently, while TMCCG worsens the performance on Stromal. We explore different density estimators and GMM is best. Besides, our method performs well with both GMM and KDE, indicating that it is robust for different density estimators. The major difficulty of cell detection is classifying detected cells into correct categories. As shown in Tab. 1, our method significantly improves the mean classification f-score from 0.572 to 0.666 for U-Net and from 0.658 to 0.669 for MCSpatNet.

Q2: More helpful for weaker methods. (R1) Augmentation is indeed more effective for weaker models and this applies to all augmentation methods.

Q3: Analysis of different density estimator performance. (R1) The cell layout distribution of BRCA-M2C data has distinct subgroups or clusters, and GMM can effectively capture these patterns. The flexibility of KDE leads to noisy density estimation results in our case, preventing our framework from getting better layout generation. GMCM models dependencies between variables using copulas, which can introduce additional complexity. The mismatches between GMCM’s assumption and data distribution lead to inferior layout generations.

Q4: Concerns about spatial-FID. (R1, R4) Spatial-FID is an adaption of FID on layout generation. We train the autoencode on the training layouts of BRCA-M2C, which has 80 images with size of ~500*500. The dice and cross-entropy losses for training autoencoder decrease from 2.98 and 2.16 to 1.11 and 0.16, respectively. We will provide the convergence curve in the revised version. We will also show that spatial-FID can reflect the increasing disturbance in layout maps to validate that it can indicate the similarity of layouts well.

Q5: Does better layout generation lead to better cell detection performance? (R3) We introduce the spatial-FID metric to evaluate the quality of the generated layouts and we find that the density model with the best spatial-FID score is also the model with the best scores on the downstream cell classification task. This validates that the improvement is mainly due to the better generated layouts.

Q6: Explain headers in Tab. 1. (R3) “Stro.”, “Infl.”, and “Epi.” refer to the Stromal, inflammatory, and epithelial cells, respectively. “Det” is the detection of all cells. The values in Tab.1 are the F-score.

Q7: Motivation of our method. (R4) The previous methods focus on improving image quality for textures, but overlook the importance of cell layouts. To tackle this problem, we propose a diffusion-based framework to generate high-quality cell layouts. Different from GAN-based model (TMCCG), our model adopts the diffusion model as backbone. Benefiting from training stability and high quality generation of diffusion model, our method performs better in layout and pathology image generation.

Q8: Density of various cell types. (R4) We generate spatial density maps for each cell type independently.

Q9: The cell shape and neighborhood of the generated image. (R4) As shown in Fig. 3, different cell types have different cell morphology. Lymphocyte cells are small in dark color, while tumor cells are larger in light color. The shape of stromal cells is more varied than other cells. In the third row of Fig. 3, the lymphocyte cells are gathered in high density to surround/confront tumor cells. This indicates our layout generative framework captures the pattern behind multi-labeled cell neighborhoods very well.



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Reject

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    The paper introduces a spatial-pattern-guided generative model for cell layout generation using diffusion models, a novel approach that includes the use of spatial features to generate realistic cell layouts. Although the paper addresses a critical component in enhancing the realism and utility of synthetic histopathology images for cell detection tasks, the reviewers expressed mixed opinions. Their critiques primarily focus on the need for clearer writing, more detailed information, and a more convincing demonstration of the advantages offered by the proposed methods over existing methods. In rebuttal, the authors have promised to provide more details and improve the readability in the revised version. However, given the page limits, it may not be feasible to incorporate all the promised changes substantially within the final version. Considering other papers in my batch, I am inclined to recommend rejection for this borderline paper in its current form.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    The paper introduces a spatial-pattern-guided generative model for cell layout generation using diffusion models, a novel approach that includes the use of spatial features to generate realistic cell layouts. Although the paper addresses a critical component in enhancing the realism and utility of synthetic histopathology images for cell detection tasks, the reviewers expressed mixed opinions. Their critiques primarily focus on the need for clearer writing, more detailed information, and a more convincing demonstration of the advantages offered by the proposed methods over existing methods. In rebuttal, the authors have promised to provide more details and improve the readability in the revised version. However, given the page limits, it may not be feasible to incorporate all the promised changes substantially within the final version. Considering other papers in my batch, I am inclined to recommend rejection for this borderline paper in its current form.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    N/A

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    N/A



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    All reviewers found the topic relevant and solution interesting. The major concern is the writing (other recommendations include testing on other datasets and performance gain margin). This is indeed a borderline, but considering the main strength in method design and main weakness in writing, I would lean towards acceptance.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    All reviewers found the topic relevant and solution interesting. The major concern is the writing (other recommendations include testing on other datasets and performance gain margin). This is indeed a borderline, but considering the main strength in method design and main weakness in writing, I would lean towards acceptance.



back to top