Abstract

In semi-supervised medical image segmentation (SSMIS), existing methods typically impose consistency or contrastive regularizations under basic data and network perturbations, and individually segment each voxel/pixel in the image. In fact, a dominating issue in medical scans is the intrinsic ambiguous regions due to unclear boundary and expert variability, whose segmentation requires the information in spatially nearby regions. Thus, these existing works are limited in data variety and tend to overlook the ability of inferring ambiguous regions with contextual information. To this end, we present Multi-Formation Soft Masking (MOST), a simple framework that effectively boosts SSMIS by learning spatial context relations with data regularity conditions. It first applies multi-formation function to enhance the data variety and perturbation space via partitioning and upsampling. Afterwards, each unlabeled data is soft-masked and is constrained to give invariant predictions as the original data. Therefore, the model is encouraged to infer ambiguous regions via varied granularities of contextual information conditions. Despite its simplicity, MOST achieves state-of-the-art performance on four common SSMIS benchmarks. Code and models will be released.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0906_paper.pdf

SharedIt Link: https://rdcu.be/dV58L

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72120-5_44

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0906_supp.pdf

Link to the Code Repository

https://github.com/CUHK-AIM-Group/MOST-SSL4MIS

Link to the Dataset(s)

https://github.com/yulequan/UA-MT/tree/master/data https://wiki.cancerimagingarchive.net/display/Public/Pancreas-CT https://www.creatis.insa-lyon.fr/Challenge/acdc/databases.html https://github.com/HiLab-git/SSL4MIS/tree/master/data/BraTS2019

BibTex

@InProceedings{Liu_MOST_MICCAI2024,
        author = { Liu, Xinyu and Chen, Zhen and Yuan, Yixuan},
        title = { { MOST: Multi-Formation Soft Masking for Semi-Supervised Medical Image Segmentation } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15011},
        month = {October},
        page = {469 -- 480}
}

Reviews

Review #1

Please describe the contribution of the paper
This paper addresses the semi-supervised medical image segmentation problem using weak and strong perturbations. They augment this strategy by
1. Partitioning images and
2. A soft masking strategy: masks are rectangular patches, soft-masked ones are boundary-liner-interpolated “rectangular” patches. The method partitions the image (e.g., to 2*2), stacks the (four) partitions, upsamples each partition to match the original image size and concatenates with the original image. With this concatenated structure, soft masking follows strong augmentations. A combination of pixel-wise pseudo classification cross entropy for the unsupervised portion (unlabeled images), DICE coefficient loss for the supervised portion (labeled images) and a copy-paste loss are used.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The results surpass the existing results with 5% and 10% labels.
2. Ablations studies show the effectiveness of strong augmentations, multi-formation function, and soft masking strategy.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. I cannot agree with the statement that “abrupt transitions [in medical images] are essentially artefacts”.
2. Soft masking infuses information from the unmasked areas into the masks. This, in my opinion, violates the original concept of masking, where no image information is present in the masked regions. Can the authors justify this choice?
3. What will happen if hard masks are used?
4. What is the reason for the sequence of operation of partitioning, staking partitions, upsampling to the original image size, augmentation, and marking? What would be the difference if masking is done on the upsampled original image without stacking?
5. Although promised in the introduction, the paper does not address the distribution shifts.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The authors claimed to release the source code and/or dataset upon acceptance of the submission.
Do you have any additional comments regarding the paper’s reproducibility?

None.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

Please see my comments above.

Minor partitioning the data and uniformly upsample them -> partitioning the data and uniformly upsampling them Consider using different letter symbols for datasets (D_l and D_u) and the dept (D). Please consider reporting SD values. Provide an ablation on the components of the loss function.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Accept — could be accepted, dependent on rebuttal (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

I selected “Weak Accept” due to results that surpass existing work and the ablations provided. Please provide ablation results with hard masks too.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

Existing methods for semi-supervised medical image segmentation (SSMIS) focus on individual voxel/pixel segmentation and overlook the challenge of ambiguous regions in medical scans. To address this, a simple framework called Multi-Formation Soft Masking (MOST) is introduced. MOST effectively enhances SSMIS by learning spatial context relations and inferring ambiguous regions using contextual information. It achieves state-of-the-art performance on four SSMIS benchmarks and will release code and models.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The problem of the paper is interesting and practical.
2. The performance outperforms previous models by a large margin.
3. The experiments are complete, and the paper is technique solid.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1 .The organization of the paper can be improved. For example, there is no related work section, which can help readers better understand your paper and your idea.
1. It would be better if you can provide more your open-source codes, as well as the datasets.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The submission does not provide sufficient information for reproducibility.
Do you have any additional comments regarding the paper’s reproducibility?

It is recommended to add a anonymous github link for reproducibility.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

See strengths and weaknesses
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Accept — could be accepted, dependent on rebuttal (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

See strengths and weaknesses
Reviewer confidence

Somewhat confident (2)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

MOST integrates a multi-formation function to partition and upsample data, enhancing its variety and perturbation space. It employs soft masking to encourage the model to infer ambiguous regions with varying granularity of contextual data. MOST achieves state-of-the-art performance on several datasets.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. Implements a multi-formation function that partitions and upsamples data, improving data diversity and perturbation design space.
2. Utilizes soft masking to effectively infer ambiguous regions.
3. Achieves state-of-the-art performance on four datasets.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The explanation and discussion of semi-supervised learning are insufficient, and only the illustration of framework for unsupervised learning is provided.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The authors claimed to release the source code and/or dataset upon acceptance of the submission.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

See the weaknesses
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Accept — should be accepted, independent of rebuttal (5)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The proposed framework is simple yet interesting. Extensive experiments have demonstrated the effectiveness of the proposed semi-supervised learning framework.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Author Feedback

N/A

Meta-Review

Meta-review not available, early accepted paper.

back to top

MOST: Multi-Formation Soft Masking for Semi-Supervised Medical Image Segmentation

Author(s):