Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Medical image segmentation is a critical task in computer-assisted diagnosis and disease monitoring, where labeling complex and ambiguous targets poses a significant challenge. Recently, the alpha matte has been investigated as a soft mask in medical scenes, using continuous values to quantify and distinguish uncertain lesions with high diagnostic values. In this work, we propose a multi-scale regions-aware implicit function network for the medical matting problem. Firstly, we design an regions-aware implicit neural function to interpolate over larger and more flexible regions, preserving important input details. Further, the method employs multi-scale feature fusion to efficiently and precisely aggregate features from different levels. Experimental results on public medical matting datasets demonstrate the effectiveness of our proposed approach, and we release the codes and models in https://github.com/xuyanyu-shh/MedicalMattingMLP.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1619_paper.pdf

SharedIt Link: https://rdcu.be/dV51B

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72114-4_45

Supplementary Material: N/A

Link to the Code Repository

https://github.com/xuyanyu-shh/MedicalMattingMLP

Link to the Dataset(s)

LIDC-IDRI: https://wiki.cancerimagingarchive.net/display/Public/LIDC-IDRI ISIC: https://www.isic-archive.com/#!/topWithHeader/wideContentTop/main Brain-growth: https://qubiq21.grand-challenge.org/QUBIQ2021/ Processed Data: https://github.com/wangsssky/MedicalMatting/tree/main/dataset

BibTex

@InProceedings{Xu_Multiscale_MICCAI2024,
        author = { Xu, Yanyu and Xia, Yingzhi and Fu, Huazhu and Goh, Rick Siow Mong and Liu, Yong and Xu, Xinxing},
        title = { { Multi-scale Region-aware Implicit Neural Network for Medical Images Matting } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15009},
        month = {October},
        page = {467 -- 477}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper proposes a multi-scale regions-aware implicit function network for medical matting problems to generate high-quality and resolution-free alpha matte. Specifically, the proposed region-aware implicit function can interpolate over larger and more flexible regions, and the proposed multi-scale feature fusion can further improve prediction quality by leveraging features from different levels.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Overall the paper is technically sound and easy to follow.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The novelty is limited. Experiments are not solid and the visualization results do not show significant improvements.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    No.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
    1. The work is incremental, just a combination of the implicit neural representations and deformable convolution.
    2. Authors need to explain the motivation and function of ‘region-aware’ more clearly. The story stated in the paper is that region-aware INR aims to better preserve important details by interpolating in a larger area, However, for the preservation of detailed information, according to the locality prior, the four nearest latent codes can actually represent the input coordinates well enough. In addition, the purpose of deformable convolution is to expand the receptive field of the network, but in the INR-based decoder, the receptive field can be expanded through the coarser scale latent code in the multi-scale feature map (like IFA and IOSNet). So it seems that there is no need to introduce additional offset mechanisms on feature maps of each scale to supplement the receptive field.
    3. Figures and texts need further checking. E.g. The flying arrows in Fig 2, the missed key codes in Fig 3, the Eq (4) that seems to be wrong, the mismatched subsection titles (‘Qualitative Comparison’ and ‘Quantitative Comparison’).
    4. Why only a subset of LIDC-IDRI and a part of ISIC 2018 dataset are selected as datasets, instead of the overall ones?
    5. Results in Fig 4 do not show the significant superiority of the proposed method compared with Medical Matting.
    6. Baselines are too weak. The newest one is merely from 2021.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Reject — should be rejected, independent of rebuttal (2)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The novelty is limited. Experiments are not solid and the visualization results do not show significant improvements.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • [Post rebuttal] Please justify your decision

    The rebuttal addresses most of my concerns although I still think the novelty is limited.



Review #2

  • Please describe the contribution of the paper

    This paper introduces the implicit neural representations to matting method for medical images. The implicit neural representations are recently used in 3D reconstruction and image super-resolution.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    New modules to the previous work:

    • Introducing implicit neural representations to matting method
    • Multi-scale feature fusion
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Upon review, the paper’s organization needs to be improved. Besides:

    1. The figure captions are overly simplistic, hindering reader comprehension.
    2. Portions of figure 4 seem to be directly replicated from prior work without proper citation.
    3. The references are outdated, warranting an update for current literature incorporation.
  • Please rate the clarity and organization of this paper

    Poor

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    An ideal figure caption should be informative, offering ample information to aid the reader in comprehending the pivotal elements of the figure.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Reject — should be rejected, independent of rebuttal (2)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Marginal improvement & unclear figures

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • [Post rebuttal] Please justify your decision

    The authors are committed to updating this paper to align it with academic norms.



Review #3

  • Please describe the contribution of the paper
    1. The paper designs a regions-aware implicit neural function to interpolate over larger and more flexible regions, preserving important input details.
    2. The method employs multi-scale feature fusion to efficiently and precisely aggregate features from different levels.
    3. Experimental results on public medical matting datasets demonstrate the effectiveness.
  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The paper designs a regions-aware implicit neural function to interpolate over larger and more flexible regions, preserving important input details.
    2. The method employs multi-scale feature fusion to efficiently and precisely aggregate features from different levels.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1. In my opinion, novel or new clinical values achieved by a new method are more important than the evaluation metrics for medical image matting tasks. Since the goal is to highlight ambiguous regions, slightly modifying the alpha value does not bring any clinical value. The authors should demonstrate novel clinical values instead of focusing solely on SAD and Grad metrics.

    2. Practically speaking, recall is important for medical image segmentation tasks. In row 2, fig. 4, the prior work “MM” recognizes lesions in the bottom right, but the method proposed in this paper does not. Although the method may achieve better evaluation metrics for alpha matte, it does not demonstrate better clinical value in this case.

    3. Unlike natural image matting, achieving a good-looking or GT-similar alpha matte is meaningless in fig. 4. The authors should provide visual examples to explain what novel or new clinical values are achieved by their new method.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Do you have any additional comments regarding the paper’s reproducibility?

    N/A

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    1. In my opinion, novel or new clinical values achieved by a new method are more important than the evaluation metrics for medical image matting tasks. Since the goal is to highlight ambiguous regions, slightly modifying the alpha value does not bring any clinical value. The authors should demonstrate novel clinical values instead of focusing solely on SAD and Grad metrics.

    2. Practically speaking, recall is important for medical image segmentation tasks. In row 2, fig. 4, the prior work “MM” recognizes lesions in the bottom right, but the method proposed in this paper does not. Although the method may achieve better evaluation metrics for alpha matte, it does not demonstrate better clinical value in this case.

    3. Unlike natural image matting, achieving a good-looking or GT-similar alpha matte is meaningless in fig. 4. The authors should provide visual examples to explain what novel or new clinical values are achieved by their new method.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Weak Accept — could be accepted, dependent on rebuttal I am not an expert in the field of medical image segmentation. If my views on clinical applications deviate from facts, my comments can be disregarded.

  • Reviewer confidence

    Not confident (1)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #4

  • Please describe the contribution of the paper

    The paper proposes a novel method for medical image matting using a multi-scale region-aware implicit neural network. The method is based on an alpha matte representation that quantifies and distinguishes uncertain lesions. More specifically a soft alpha matte function is used along with probability Unet and Matting Unet to carry out the regions’ segmentation.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Major strength includes the use of a regions-aware implicit neural function, which allows for interpolation over larger and more flexible regions while preserving important input details. This enables the network to capture fine-grained information and improve the quality of predictions. Additionally, the method incorporates multi-scale feature fusion, which enhances the aggregation of features from different levels and further improves the performance.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The methodology is built on several key components which includes probability Unet, Matting Unet and Multi scale feature aggregation, majority of the components already exists in literature and are well explored in medical imaging domain individually. The method explains the region aware implicit function however doesn’t describes the processing of soft labels interms of image matting. The solution of alpha matte equation explained in introduction section interms of the proposed implicit function and matting network is is not presented clearly.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Do you have any additional comments regarding the paper’s reproducibility?

    The method seems to work satisfactorily as presented.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

    Further research and evaluation on a wider range of medical images would be valuable to fully assess its capabilities. The visual comparison of dermoscope in Fig.4. (Second Row) doesn’t seems promising as compared to ground truths.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

    Weak Accept — could be accepted, dependent on rebuttal (4)

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    In general, this paper presents a promising approach for medical image matting, addressing the challenges of labeling complex targets and presenting the target regions in terms of alpha matte rather than hard labels. The use of the region-aware implicit neural network and multi-scale feature fusion contribute to the effectiveness of the method.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Author Feedback

We thank the reviewers for the high-quality reviews and will carefully revise our submission, correct typos, add more details and discussions, and release the codes and models in the revision.

Q1. Clinical Values (R1) We thank for the insightful comments. Alpha matte segmentation offers more detailed and clinically useful information about lesions than binary segmentation. Clinical values of new modules: 1) Detailed information from ambiguous and small regions enhances the accuracy of identifying and characterizing lesions, improving the overall diagnostic process. 2) Clear delineation of ambiguous and small regions reduces the risk of misdiagnosis, ensuring that both benign and malignant areas are correctly identified and treated appropriately.

Q2. Justification of the contributions (R3) The main contributions of this paper can be summarized as follows: 1) Clinically, our method enhances the accuracy of lesion segmentation by preserving intricate details in ambiguous areas, which is crucial for precise diagnosis and treatment planning. Additionally, the multi-scale feature fusion mechanism ensures that even tiny and arbitrary regions are accurately captured, preserving critical details that might be overlooked by traditional methods. 2) Technically, we introduce a region-aware implicit neural representation that interpolates over larger and more flexible regions, preserving important details missed by conventional locality-based approaches. Our multi-scale feature fusion mechanism integrates features across different scales, enhancing prediction quality and ensuring detailed information capture.

Q3. Motivation of ‘region-aware’ contribution. (R3) Thank you for the feedback here is a clearer explanation. Deformable convolution expands the network’s receptive field, but coarser scale latent codes (as in IFA and IOSNet) have limitations in capturing fine-grained details, especially in small or arbitrary regions. Region-Aware mechanism addresses this by introducing offset mechanisms on feature maps at each scale, enhancing the capture of detailed information across scales and preserving tiny and intricate features that might otherwise be lost. As shown in the Right in Table 2, this mechanism interpolates over larger areas, capturing more detailed information, which is particularly beneficial for tiny and arbitrary regions. This ensures that even small area features are preserved and supplemented by information from other parts of the image, effectively preserving detailed information across different scales.

Q4. Qualitative Analysis in Fig. 4. (R1&R3&R5) The IF, LB, FBA methods tend to produce over-segmented or binary mask-like results, missing finer details. Both MM and ours provide a more detailed alpha matte closer to the ground truth (GT). Our method captures the complex texture of the lesion more effectively, maintaining details in ambiguous areas. We will provide additional visual examples highlighting finer details in ambiguous and tiny areas to demonstrate the new clinical values more comprehensively.

Q5. Datasets (R3) We follow the settings in [23] to use part of two datasets for a fair comparison.

Q6. Baselines (R3) The medical matting problem is still evolving, meaning related work is limited. We are working on updating our baselines with more recent models in our revision.

Q7. Figures and Captions (R3&R4) Thank you for your valuable feedback. We will correct the issues in Figures 2 and 3, including the flying arrows and missed key codes, and verify and rectify any mistakes in Equation (4). Subsection titles will be reviewed for consistency. We will revise figure captions to provide more detailed descriptions and ensure all portions of Figure 4 are appropriately cited.

Q8. Extension (R5) We agree that further research and evaluation on a wider range of medical images is valuable. We plan to expand the datasets and conduct additional experiments for a more comprehensive evaluation in our revision.



Meta-Review

Meta-review #1

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    N/A

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    N/A



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Reject

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    First, the paper lacks solid comparison with SOTA methods, which makes the effectiveness of the proposed method less convincing. The authors promise to add more results in the final version, which can not be counted for the paper decision.

    Second, the rebuttal does not address the gap between the motivation and the result. It is clear that “accurate matting” matters, but it is unclear if the 0.01 improvement in MSE can make a difference in the clinical setting. Thus, it is unclear if the improvement is impactful.

    Third, the paper lacks novelty. The proposed method is a straightforward combination of dynamic conv and implicit function. Without significantly better results, it is hard to justify such a simple approach.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    First, the paper lacks solid comparison with SOTA methods, which makes the effectiveness of the proposed method less convincing. The authors promise to add more results in the final version, which can not be counted for the paper decision.

    Second, the rebuttal does not address the gap between the motivation and the result. It is clear that “accurate matting” matters, but it is unclear if the 0.01 improvement in MSE can make a difference in the clinical setting. Thus, it is unclear if the improvement is impactful.

    Third, the paper lacks novelty. The proposed method is a straightforward combination of dynamic conv and implicit function. Without significantly better results, it is hard to justify such a simple approach.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Reject

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    I have read the manuscript, review comments, rebuttal letter and the opinions of two meta-reviewers.

    Although four reviewers tends to weakly accept this paper, I tend to agree with Meta-Reviewer #4. The main problems of this paper include:

    1. The methodology looks the incremental modification of the neural network structure. Moreover, little explanation about the claimed contributions are provided.

    2. Insufficient comparison with SOTA methods

    3. The rebuttal letter tends to re-claim the conclusion by authors, rather than provide the convincing explanations.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    I have read the manuscript, review comments, rebuttal letter and the opinions of two meta-reviewers.

    Although four reviewers tends to weakly accept this paper, I tend to agree with Meta-Reviewer #4. The main problems of this paper include:

    1. The methodology looks the incremental modification of the neural network structure. Moreover, little explanation about the claimed contributions are provided.

    2. Insufficient comparison with SOTA methods

    3. The rebuttal letter tends to re-claim the conclusion by authors, rather than provide the convincing explanations.



Meta-review #4

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    All four reviewers recommend acceptance (after rebuttal). This meta reviewer believes that the authors did a good job in addressing concerns.

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    All four reviewers recommend acceptance (after rebuttal). This meta reviewer believes that the authors did a good job in addressing concerns.



back to top