Paper Info Reviews Author Feedback Meta-Review Back to top

List of Papers Browse by Subject Areas Author List

Abstract

Examining pathology images through visual microscopy is widely considered the most reliable method for diagnosing different medical conditions. Although deep learning-based methods show great potential for aiding pathology image analysis, they are hindered by the lack of accessible large-scale annotated data. Large text-to-image models have significantly advanced the synthesis of diverse contexts within natural image analysis, thereby expanding existing datasets. However, the variety of histomorphological features in pathology images, which differ from that of natural images, has been less explored. In this paper, we propose a histomorphology-focused pathology image synthesis (HistoSyn) method. Specifically, HistoSyn constructs instructive textural prompts from spatial and morphological attributes of pathology images. It involves analyzing the intricate patterns and structures found within pathological images and translating these visual details into descriptive prompts. Furthermore, HistoSyn presents new criteria for image quality evaluation focusing on spatial and morphological characteristics. Experiments have demonstrated that our method can achieve a diverse range of high-quality pathology images, with a focus on histomorphological attributes.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0215_paper.pdf

SharedIt Link: https://rdcu.be/dY6iw

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72083-3_19

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0215_supp.pdf

Link to the Code Repository

https://github.com/7LFB/HistoSyn

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Yin_HistoSyn_MICCAI2024,
        author = { Yin, Chong and Liu, Siqi and Wong, Vincent Wai-Sun and Yuen, Pong C.},
        title = { { HistoSyn: Histomorphology-Focused Pathology Image Synthesis } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15004},
        month = {October},
        page = {200 -- 210}
}

Reviews

Review #1

Please describe the contribution of the paper

This paper presents a pathology image synthesis method called HistoSyn with an emphasis on modeling spatial and morphological attributes. It also introduces a new evaluation metric named HistoD for assessing synthetic pathological image quality based on histomorphology attributes, which strongly correlates with down-stream task performance.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. This paper proposes a novel pathology image synthesis method, which translates visual details of spatial and morphological attributes into descriptive textual prompts to guide the image generation process towards the user-defined histomorphology.
2. This paper introduces a histomorphology-focused evaluation metric, which leverages Wasserstein distance to statistically measure of the discrepancy between the attribute distributions of real and synthetic datasets.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. Incorporating spatial and morphological attributes into pathological image synthesis is the key contribution of this paper. However, the description in Section 2.1 is very unclear: (1) What exactly do the objects referred to by these attributes represent in pathological images? (2) The notation in the formulas is very confusing. What is the numerical significance of the ‘l’ in the ‘l-th kind of spatial attributes’ and the ‘k’ in the ‘k-th kind of morphological attributes’? How are they related to the number of objects ‘N’, and the number of attributes N_attr? How many spatial attributes and morphological attributes does each object have, and what exactly do these attributes include? (3) What is the representation format of these attributes in the text prompts? Raw texts? Or text embeddings?
2. The comparison experiment does not sufficiently validate the effectiveness of incorporating histomorphology attributes into text prompts. According to the text prompt examples shown in the supplementary materials, apart from the placeholders, the text prompt templates of HistoSyn contain a substantial amount of descriptive words and phrases about the content of pathological images, which are not present in the text prompts of other methods. I wonder how the image generation would turn out if we simply remove the placeholders and use the text prompt template itself directly.
3. The visualization results in Fig. 2 do not show any significant difference between HistoSyn and Morphology-enriched method.
4. The calculation of HistoD involves different histomorphology attributes, and simply taking the averaged HistoD of all attributes is not reasonable. It does not allow for a comparison of the strengths and weaknesses of the generated results at different attributes.
5. Section 3.4 is confusing. It does not clarify the meaning of the each subscript number under HistoD. Readers are left unaware of which attributes correlate with histological findings and whether this correlation aligns with clinical reality. Additionally, when analyzing the results of the Pearson correlation coefficients, the statistical significance levels were not provided.
6. This paper does not provide details regarding the data usage in the histological findings recognition experiment, such as the quantities of real and generated data, and how attribute conditioning was applied. This information is crucial for assessing the augmentation effectiveness of the generated data on downstream tasks.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not provide sufficient information for reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

Refer to the weaknesses above.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Reject — could be rejected, dependent on rebuttal (3)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper has not presented many details about the method and experiments. The experimental results presented are not convincing due to the lacking details. The organization of language leaves room for improvement.
Reviewer confidence

Very confident (4)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

Accept — should be accepted, independent of rebuttal (5)
[Post rebuttal] Please justify your decision
Thanks for authors’ rebuttal response. A good paper should be self-contained, and with that in mind, I offer the following suggestions to improve the clarity and the readability of your writing in the final version, if this paper is accepted.
1. It is essential to clearly define each symbol used in the mathematical expressions within the main text, especially Secion 2.1. For example, what the objects exactly refer to, the exact number of N and N_attr should be explicitly stated, rather than presented in the supplementary material.
2. To better illustrate the differences between the proposed method and others, I recommend adding highlights to the visualization results in Figure 2.
3. The paper would benefit from a more thorough explanation of the experimental setup, such as the mixed ratio of real and synthetic images, the meaning of the subscripts in HistoD.

Review #2

Please describe the contribution of the paper

The paper introduces HistoSyn, a novel framework tailored for synthesizing pathology images with a primary emphasis on histomorphology, a pivotal aspect in clinical diagnosis. Additionally, the authors introduce a novel evaluation metric that prioritizes spatial and morphological characteristics, complementing existing criteria. Through rigorous experimentation, they demonstrate the efficacy of HistoSyn in generating high-quality pathology images with a focus on preserving histomorphological attributes.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper’s primary strengths lie in two key areas. Firstly, it addresses a critical challenge in digital pathology by offering a solution to the arduous task of creating high-quality datasets. This contribution holds significant promise for enhancing the efficacy of downstream deep learning algorithms in pathology detection through improved training data availability. Secondly, the authors demonstrate a commendable understanding of the unique characteristics of pathology images, particularly their diverse morphological and spatial distributions. By acknowledging and addressing this complexity in image synthesis, their work stands out as a valuable addition to the field.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. One notable weakness of the paper is the absence of specific information regarding the bandwidth of variations for individual features in pathology images. For instance, the diversity in the size of cells within the liver NAS dataset remains unspecified. Providing such quantitative details would enhance the clarity and depth of the analysis, enabling a better understanding of the dataset’s characteristics.
2. Another area for improvement is the quality of the figures presented in the paper. It is suggested that the authors enhance the resolution of Figure 3 to improve readability. Additionally, in Figure 2’s title, it would be beneficial for the authors to emphasize how their method surpasses others. Highlighting qualitative observations that distinguish their approach would provide valuable insights into the superiority of their method over existing techniques.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
1. While the authors provide a qualitative assessment of diversity in pathology images, particularly considering morphology and spatial characteristics, it would be beneficial to include a quantitative assessment. For instance, detailing the statistical distribution of values for each attribute, both in original images and synthesized ones, would offer a more comprehensive understanding of the performance of the image synthesis model from a statistical perspective.
2. Although the authors mention utilizing Hover-Net for nuclei segmentation, they lack clarity on the methods employed for extracting other histological features. Given the pivotal role of accurate segmentation in image synthesis, it would be advantageous if the authors elaborate on any additional segmentation techniques explored. Moreover, it would enhance the paper if they conducted a qualitative assessment of the segmentation performance on the dataset, providing insights into the efficacy of different segmentation methods employed.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Accept — could be accepted, dependent on rebuttal (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper demonstrates commendable efforts in addressing the scarcity of high-quality datasets in digital pathology and acknowledges the complexity of pathology images. While the paper addresses significant challenges in digital pathology and offers promising solutions for image synthesis, it lacks certain critical elements. For example, there is not much clarity on the statistical distribution of histological features that leads to diversity in pathology images. Further, their method works well only if the segmentation is accurate and hence more information on the validation of segmentation strategy is required. However, with some revisions to address these shortcomings, the paper holds potential for making a valuable contribution to the field.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

Weak Accept — could be accepted, dependent on rebuttal (4)
[Post rebuttal] Please justify your decision

The authors have addressed my concerns and specific questions. I would emphasize to improve the quality of graphics and captions for the camera-ready version of this paper. Further, also provide statistical diversity of the morphologies taken into account while synthesizing images.

Review #3

Please describe the contribution of the paper

This paper proposes a method for synthesizing pathology images with a focus on histomorphology. The proposed method enhances image synthesis by incorporating a wide range of histomorphology attributes depicted through statistical measures. This paper designs a new reliable metric to assess the quality of synthetic pathology images based on histomorphology attributes.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This paper represents a comprehensive effort, as it not only presents a framework that incorporates spatial and morphological attributes into prompts, but also introduces a novel metric for evaluating the quality of generated images based on morphology. The incorporation of morphology attributes into prompts is highly heuristic for applying the diffusion model to pathology images.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

In the introduction, authors should include a comparison between this paper and related work [17] and another related work “A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Images”.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

authors should publich the code for reproduction and promote the development of related work
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

see weakness
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Accept — should be accepted, independent of rebuttal (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

see main strength
Reviewer confidence

Very confident (4)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Author Feedback

Thanks to all reviewers for their valuable feedback. We are encouraged by all reviewers to find our idea and method novel. Below are responses to each point. @Reviewer #1

More stats on Real vs. Synthetic data attributes Thanks for your instructive suggestions. More statistics could be derived based on the two distributions (in Eq (7)). These highlight the differences in terms of mean, variance, and bandwidth.

Explore other nuclei extraction methods We opted for Hover-Net, a trusted baseline for nuclei segmentation, to demonstrate our method’s effectiveness in attribute-enhanced image synthesis, noting its use is optional. White regions are extracted and denoised using morphological operations within a specific color range. Other segmentation techniques (e.g., CellPose) may also be optional. The exploration of various nuclei extraction methods is beyond the focus of this paper and may be addressed in future studies.

Quality and clarity of figures and captions We would increase the resolution of the figure and revise the caption as “Our method excels in synthesizing pathology images by capturing diverse morphological attributes of white regions (e.g., area) and the varied spatial arrangement of nuclei surrounding these regions while preserving essential histological findings.’’ @Reviewer #3

(i)-(ii) Notion clarity on attributes N objects (2 types: nuclei and white regions) are analyzed based on N_attr attributes (7 types). For both nuclei and white regions, 4 spatial attributes are evaluated. 3 morphological attributes are considered specifically for white regions. Detailed definitions (4+3) of these attributes are provided in Tab. 2 of the Supplementary Material. The subscript (1-7) under HistoD in Section 3.4 corresponds to the seven attributes we analyzed.

(iii) Representation format of attributes The attributes are represented as calculated numerical values and are inserted into placeholders within a predefined text prompt template. These text prompts are then converted into embeddings through a text encoder mentioned in Sec. 2.

More explanation for Tab. 1 The detailed descriptive words and phrases are necessary to locate the . For your interest, we tested the textual descriptions without numerical and found that the performance dropped compared to ours and was better than baseline. We would add them to the final version if allowed.

Visual comparison in Figure 2 Variations like the size of white areas and nuclei positioning are present in the figure. These differences are further revealed by the quantitative metrics shown in Table 2.

Calculation of HistoD We agree that averaging HistoD scores oversimplifies the analysis. The average was meant to quickly gauge overall performance. You’re correct in emphasizing the importance of individual attribute analysis for a comprehensive understanding. As demonstrated in Figure 3, each attribute was analyzed individually, providing clearer insights.

Analysis with P-value Sorry for the missed p-value. Analyzing p-values, which can be derived during Pearson calculations. We would add them to the final version if allowed.

Details of synthesizing data We maintained a 1:1 (total 224) ratio of real to synthetic data, with each prompt generating one image to match real data. When generating synthetic images, prompts are provided based on real training data, similar to the compared method [17]. We followed the standard practice in conditional LDM [18] to apply Eq.2. The details will be added to the final version. The code will be released after acceptance. @Reviewer #4

More discussion of two related works in the Introduction As suggested, we’ll add a discussion on how these studies solely focused on studying morphological attributes indirectly without specifying quantitative attributes.

Meta-Review

Meta-review #1

After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

Accept
Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

The paper introduces a novel methodology for synthesizing pathology images, concentrating specifically on histomorphology attributes. Initially, the reviews included one weak reject, but following the author’s rebuttal, all reviewers have moved to accept the paper. It is important that the authors adhere to the commitments made in their rebuttal, such as regarding the enhancements to figure quality and captions for improved clarity. Please also make sure the supplementary material is within the page limit.
What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

The paper introduces a novel methodology for synthesizing pathology images, concentrating specifically on histomorphology attributes. Initially, the reviews included one weak reject, but following the author’s rebuttal, all reviewers have moved to accept the paper. It is important that the authors adhere to the commitments made in their rebuttal, such as regarding the enhancements to figure quality and captions for improved clarity. Please also make sure the supplementary material is within the page limit.

Meta-review #2

After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

Accept
Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

All reviewers accept the paper after rebuttal.
What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

All reviewers accept the paper after rebuttal.

back to top

HistoSyn: Histomorphology-Focused Pathology Image Synthesis

Author(s):