List of Papers Browse by Subject Areas Author List
Abstract
Abnormal structures in multi-modality medical images often lead to heterogeneous heavy-tailed distributions. However, traditional models, especially those relying on Gaussian distributions, struggle to effectively capture these outliers. To address this, we propose BayeSMM, a novel framework that leverages Student’s $t$ distribution mixture models (SMM) to simultaneously perform registration and segmentation for misaligned multi-modality medical images. Specifically, we construct a Bayesian Student’s $t$ mixture model incorporating the heavy-tailed nature of the Student’s $t$ distribution and develop variational inference to optimize the model. Guided by variational inference, we design a novel deep learning architecture that performs registration and segmentation jointly. We demonstrate the effectiveness of BayeSMM with experiments on the MS-CMR dataset, where the results show superior performance compared to existing combined computing methods, and yield enhanced robustness under the simulated heavy-tailed setting. The code is available at https://github.com/HenryLau7/BayeSMM.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2025/paper/0645_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/HenryLau7/BayeSMM
Link to the Dataset(s)
MSCMR dataset: https://zmiclab.github.io/zxh/0/mscmrseg19/
BibTex
@InProceedings{LiuYua_BayeSMM_MICCAI2025,
author = { Liu, Yuanye and Zhen, Ruoxuan and Gao, Shangqi and Luo, Xinzhe and Gao, Xin and Chen, Qingchao and Zhuang, Xiahai},
title = { { BayeSMM: Robust Deep Combined Computing Tackling Heavy-tailed Distribution in Medical Images } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2025},
year = {2025},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15972},
month = {September},
page = {44 -- 53}
}
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a method to jointly perform image registration and segmentation. Specifically, they introduce a generative model in which voxel intensities are assumed to be conditionally independent given the underlying segmentation mask. The key contribution of the work is the use of a mixture of Student’s t-distributions, rather than the more common Gaussian mixtures, to better account for the presence of outliers in images. To optimize the model, the authors develop a variational inference approach based on a mean-field approximation, enabling joint optimization of registration and segmentation.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The methodology is sound, relying on a well-defined probabilistic graphical model with clear assumptions regarding the parameterization of the different components.
The optimization strategy based on mean-field variational inference is well-suited to the problem.
Unlike most previous approaches that used graphical models for image registration, this work extends them by incorporating a neural network to estimate the posterior parameters. As such, it provides a modernized methodology on these classical models.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
My main concern lies in the lack of comparison with relevant literature and in an evaluation protocol that presents several weaknesses.
- The proposed approach is grounded in probabilistic graphical models, which was widely explored few years ago in the MICCAI community. However, the authors do not sufficiently acknowledge this body of literature. For instance, [1] also performs joint segmentation and registration using a probabilistic graphical model. Furthermore, the authors omit key references related to Conditional Random Fields (CRFs) for image registration [2,3,4], which share conceptual similarities with their approach but often incorporate more advanced spatial modeling. The authors should have mentioned the differences and advantages of their approach compared to these approaches.
- Additionally, the proposed model doesn’t seem to be extendable to multimodal settings, as the generative model appears to be modality-specific.
In terms of validation:
- One of the main motivations of this work is to better handle outliers via a Student’s t-distribution rather than a Gaussian. However, the empirical evidence provided is weak: Table 1 shows only a marginal improvement in segmentation Dice score (+0.5%), with standard deviations around 1%. No statistical tests are performed.
- The experiments are also limited in scope, being conducted on a single dataset. It seems that the images were heavily preprocessed as the warped images have been masked in Fig 3, without any mention in the text, which may dramatically simplify the problem.
- Moreover, several widely-used baselines (e.g., NiftyReg, ConvexAdam) are missing, making it difficult to assess how the method compares to standard or more established approaches.
Typos: page 3: “an defined” page 4: Eq.(2,
[1] Wyatt, P. P., & Noble, J. A. (2003). MAP MRF joint segmentation and registration of medical images. Medical Image Analysis, 7(4), 539-552.
[2] Parisot, S., Duffau, H., Chemouny, S., & Paragios, N. (2012). Joint tumor segmentation and dense deformable registration of brain MR images. MICCAI
[3] Parisot, S., Wells III, W., Chemouny, S., Duffau, H., & Paragios, N. (2014). Concurrent tumor segmentation and registration with uncertainty-based sparse non-uniform graphs. Medical image analysis, 18(4), 647-659.
[4] Xiaohua, C., Brady, M., & Rueckert, D. (2004). Simultaneous segmentation and registration for medical image. MICCAI
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — could be rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed approach addresses the important problem of joint registration and segmentation using a mathematically grounded framework based on a probabilistic graphical model. The use of this formalism is elegant, as it relies on clear assumptions and provides a solid methodological foundation for potential extensions.
However, the authors do not reference the (not-so-recent but MICCAI) literature that has explored similar ideas, which weakens the originality of the method and the need to develop new modeling. For example, we could wonder if the proposed method couldn’t have been used in a more advanced probabilistic model that introduces spatial consistency in the segmentation masks, which were proposed before the deep-learning era.
Lastly, the evaluation is quite limited and the results are not entirely convincing.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
The authors have adequately addressed most of my concerns, particularly regarding the literature review and the handling of multimodality. They also provided a reasonable justification for the effectiveness of their method, highlighting its robustness to noise despite the marginal improvements without noise.
Overall, this is a principled proof-of-concept for image registration, grounded in a well-designed graphical model, and is well aligned with MICCAI interests.
Review #2
- Please describe the contribution of the paper
The paper introduces a statistical framework for joint registration and segmentation of multi-modality medical images. The proposed method leverages Student’s t-distribution to model heavy-tailed intensity variations and employs variational inference to guide learning of the distribution parameters. BayeSMM has achieved superior performance compared to existing state-of-the-art methods for joint segmentation and registration tasks.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
Modeling multi-modality medical images using a Student’s t mixture model allow for incorporating an outlier-detection variable that enhances robustness against abnormal regions in images.
-
An efficient variational inference-based deep learning framework is proposed that jointly estimates distribution parameters while jointly performing registration and segmentation (termed as combined computing).
-
Without any affine preprocessing, BayeSMM has outperformed existing combined computing methods in registration by 18.7%, demonstrating strong alignment capabilities under severe misalignment conditions, as shown in Table 1
-
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
The variational inference employed here relies on a mean-field assumption, which factorizes the joint posterior into independent distributions over latent variables. However, this independence assumption may be overly restrictive — particularly if the latent mean parameter (μ) and the outlier indicator or variable (u) exhibit statistical dependence. Ignoring such correlations can lead to suboptimal posterior approximations and poor uncertainty quantification in regions influenced by outliers. Given the potential for correlation between these variables—particularly in settings with heavy-tailed behavior or structured outliers—could the authors clarify or justify this modeling choice? A brief discussion of the implications of this assumption on posterior accuracy and model robustness would strengthen the paper.
Although the model is framed within a Bayesian paradigm, it does not appear to provide uncertainty quantification for its predictions or latent variables. This is a missed opportunity, as one of the key advantages of Bayesian methods is the ability to capture and reason about uncertainty — particularly valuable in settings where data may be noisy, sparse, or contain outliers.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html
The authors may consider expanding the modeling capacity beyond heavy-tailed distributions by incorporating more flexible intensity models, such as multimodal or skewed distributions.
Additionally, relaxing the mean-field variational assumption — for example, through structured inference — could allow the model to better capture dependencies in the posterior.
Finally, incorporating uncertainty quantification would strengthen the Bayesian framework and enhance the clinical interpretability of the results.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — could be accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper presents a novel approach to modelling multimodal images and makes a solid contribution to joint registration and segmentation in multi-modality medical imaging. While there are some limitations in the inference approximation and model assumptions, the robustness under misalignment and the strong performance compared to other state-of-the-art proofs its merit.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This paper presents a thoughtful and well-constructed approach to jointly performing registration and segmentation on multi-modality medical images through a Bayesian Student’s t Mixture Model (BayeSMM). The authors target a real and often overlooked challenge in medical image analysis: the heavy-tailed, heterogeneous intensity distributions that arise from abnormalities and modality mismatches. Instead of relying on conventional Gaussian assumptions, they model the intensity distributions using Student’s t mixtures, which offer robustness against outliers by introducing heavier tails through an explicit outlier-detection variable. This choice is well motivated and fits naturally within a Bayesian hierarchical framework.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Overall, this is an interesting and well-motivated paper that combines rigorous statistical modeling with practical deep learning components. The Bayesian formulation and use of Student’s t mixtures are meaningful contributions to the field, especially in dealing with outlier-prone datasets. While there are a few open questions regarding the optimization and inference strategies, these do not detract from the novelty or value of the work. I recommend accepting the paper, or at least giving it fair consideration, with the hope that some of the open questions can be addressed in future iterations or supplemental materials.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
First, while the hierarchical probabilistic graphical model is promising, the way the variational distribution is factorized assumes a fully mean-field approximation. This simplification may limit the expressivity of the posterior, especially when correlations exist among latent variables (e.g., between Have the authors considered structured variational inference or amortized inference as alternatives to better capture these dependencies?
Second, the Student’s t distribution introduces an additional degree-of-freedom parameter ν, which governs the tail heaviness. It is not fully clear whether this parameter is fixed or learned during training. If it is fixed, some insight into how it is chosen would be helpful, as this parameter has a strong influence on the shape of the distribution. If learned, how is the inference handled for v, given its non-conjugacy?
Third, while the formulation is theoretically grounded, the implementation relies partially on neural networks to estimate posterior components. The manuscript mentions an alternating approach between analytical and learning-based inference, but this is not fully elaborated. How are the posteriors over discrete labels z handled during training in the presence of neural networks? Is there a relaxation or sampling strategy applied (e.g., Gumbel-softmax)? Clarifying this interaction would enhance both interpretability and reproducibility.
Another area that could benefit from improvement is the literature review. While the paper cites relevant work on variational inference and image registration, it lacks coverage of recent developments in deep learning models that perform joint registration and segmentation, for instance, in applications like tumor/lesion/abnormalities analysis. Methods such as MetaMorph (“Learning Metamorphic Image Transformation with Appearance Changes”) and MetaRegNet (“Metamorphic Image Registration Using Flow-Driven Residual Networks”) have proposed effective architectures that explicitly address joint modeling tasks in the presence of substantial appearance shifts. These works highlight the practical need and feasibility of combining registration, segmentation, and even removal of pathological tissue—an area where the proposed BayeSMM model could make an even stronger impact. Including and comparing to such works would help contextualize the novelty and practical relevance of this paper more clearly.
On the empirical side, the performance on the MS-CMR dataset looks promising. The results show robustness in handling simulated heavy-tailed conditions, which supports the core motivation of the work. It would be beneficial to see more visualizations or diagnostic plots showing how the outlier-detection mechanism adapts across samples or modalities. This would provide better intuition for the model’s behavior and interpretability in clinical settings.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(5) Accept — should be accepted, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Innovation, clarity and the expression of the story.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
This paper presents interesting findings and partially addresses the questions raised. However, the proposed model appears somewhat complex to reproduce. Providing the implementation code would significantly enhance its reproducibility and impact.
Author Feedback
We thank all reviewers for their constructive feedback. As for the method and evaluation part, R1 and R2-R3 posit different opinions, where R1 raised concerns about the applicability to multimodal settings and experimental results, while R2 and R3 acknowledged well-motivated novelty and strong performance instead. Our source code is ready for release at GitHub.
*Regarding Writing Q: R1 and R3 expected more introduction, discussion, and comparisons with related works. A: We will revise the literature review to include classical PGM-based methods and clarify our contributions: (1) our Student’s t mixture model improves robustness to outliers, which Gaussian models often fail to capture; (2) our hybrid framework combines variational inference and deep learning, improving efficiency over iterative solutions. Though we tackle the joint segmentation-registration task in a unified framework, we will also discuss registration-only baselines (e.g., NiftyReg, ConvexAdam), and include joint segmentation-registration works involving sub-region segmentation as a future work.
*Regarding Experiments Q: R1 questioned the benefit of using Student’s t, noting marginal Dice improvement. A: Rather than achieving the highest Dice score, BayeSMM aims to enhance robustness without sacrificing accuracy. Prior studies show a known trade-off principle between accuracy and robustness, where robust models tend to capture semantically meaningful features (BayeSMM explictly capture abnormal region with outlier-detection variable u). The following results demonstrate the robustness of BayeSMM. Table 1 shows that BayeSMM achieves competitive and SOTA Dice scores, and maintains stable performance under synthetic corruptions, outperforming other methods by a large margin. Fig. 3 demonstrates BayeSMM can identify abnormal regions via u using the student t formulation.
Q: R1 noted the experiment is limited to one dataset, and masked warped images (Fig. 3) suggest heavy preprocessing. A: Considering fair and controlled comparisons with existing works, we follow the same preprocessing pipeline in [1]. In terms of comprehensive evaluations, we evaluated on MSCMR on two different settings (before and after pre-registration). In future work, we will evaluate BayeSMM on more datasets.
*Regarding Methodology Q: R1 claimed the model is not extendable to multimodal settings due to modality-specific design. A: We would like to clarify that our model is explicitly designed for multimodal settings and evaluated on a three-modality cardiac dataset. Theoretically, it follows a conditional independence assumption as in [1], where each modality is modeled independently given a shared label. Practically, the deep inference network is modality-shared, allowing scaling to more modalities without structural changes.
Q: R2 and R3 asked why mean-field factorization is assumed. A: We adopt this assumption to ensure tractable inference with efficient updates under conjugate priors, which enables stable optimization and is commonly used in prior works [2]. Despite this simplification, our results (Fig. 3) show that the model captures meaningful outlier structure. We will explore structured inference in future work.
Q: R3 asked how the degrees-of-freedom parameter \nu is treated and how discrete labels z are handled under neural inference. A: The nu parameter is not modeled as a random variable but serves as a parameter of the Gamma prior for u, and is implicitly updated through the network predicting u per voxel. For the discrete label z, we compute expectations over the predicted categorical posterior gamma_{x,k} and directly evaluate the variational loss L_z, avoiding sampling during training. We will clarify both aspects in our revision.
[1]X-Metric: An N-Dimensional InformationTheoretic Framework for Groupwise Registration and Deep Combined Computing. TPAMI2023 [2]BayeSeg: Bayesian Modelling for Medical Image Segmentation with Interpretable Generalizability. MedIA2023
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- If your recommendation is “Provisional Reject”, then summarize the factors that went into this decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. You do not need to provide a justification for a recommendation of “Provisional Accept” or “Invite for Rebuttal”.
N/A
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
N/A
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
Generally this is a solid, innovative and well-written work, and ready to be published in MICCAI. However, I agree with R1’s view about comparing the proposed joint seg-reg method with popular registration methods and segmentation methods, even as ablations if possible, yet it is not fully feasible with limited length of this conference paper. I suggest to perfom these additional comparison if the authors decided to extend this paper into a journal version. After all, at the present stage, well done.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
I recommend acceptance of this paper.
The reviewers acknowledge that the paper makes a principled and well-motivated contribution to the field of image registration, with a graphical model approach that is both methodologically sound and aligned with MICCAI interests. The authors have adequately addressed key concerns raised during the initial review, particularly with regard to the literature review, handling of multimodality, and robustness of the method under noise conditions.
One remaining concern is the reproducibility of the method, as the proposed model is relatively complex. While this does not preclude acceptance, I strongly encourage the authors to release the implementation code to facilitate reproducibility and broaden the impact of their work.
Overall, this paper presents a solid proof-of-concept backed by reasonable empirical evidence and contributes meaningfully to the MICCAI community.