Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Deep learning-based medical image segmentation models are trained using annotations that exhibit systematic bias and variability across raters. While probabilistic multi-rater approaches can emulate annotator-specific delineations, annotator characteristics are typically encoded implicitly in deep latent feature space, making direct analysis of their influence on predictive distributions less straightforward. We propose a logit-space probabilistic segmentation framework based on stochastic variational Gaussian Process that explicitly decomposes predictions into an image-dependent reference logit distribution and annotator specific perturbations parameterised by bias and variance. This formulation enables more explicit analysis on how intra- and inter-rater variability propagate to predictive distributions. We evaluate the method on a multi-annotator medical image dataset, which shows that explicitly modelling annotator specific perturbations improves uncertainty calibration while maintaining comparable segmentation accuracy, compared with state-of-the-art multi-rater probabilistic segmentation method. The learned bias and variance parameters quantitatively reflect annotator-specific behaviour. Furthermore, controlled perturbation experiments over bias and variance demonstrate how changes in annotator parameters systematically influence predictive performance. The code used in this paper is made publicly available at https://github.com/QiLi111/GPS-Var.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2379_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/QiLi111/GPS-Var

Link to the Dataset(s)

N/A

BibTex

@InProceedings{LiQi_Interpretable_MICCAI2026,
        author = { Li, Qi AND Huang, Yuliang AND Saeed, Shaheer U. AND Yang, Qianye AND Stavrinides, Vasilis AND Baum, Zachary M. C. AND Barratt, Dean C. AND Noble, J. Alison AND Vercauteren, Tom AND Hu, Yipeng},
        title = { { Interpretable Probabilistic Medical Image Segmentation via Gaussian Process with Explicit Modelling of Annotation Bias and Variability } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16882},
        month = {September},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper addresses the multi-rater segmentation problem, where the goal is to recover a latent gold-standard segmentation from multiple annotators who may exhibit different biases and variances in their labeling behavior. The proposed approach uses a Gaussian Process as a generative prior over the latent segmentation, and models each annotator’s output through two scalar parameters: a bias term mu and a variance term sigma. The core contribution of the model is its interpretability — rather than learning implicit, non-linear annotator representations as in PIONONO (ICCV 2023), each rater is described by two clinically meaningful quantities, mu and sigma. Experiments are conducted on a trans-rectal ultrasound dataset, showing Dice performance comparable to PIONONO and improved Expected Calibration Error (ECE).

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    • Clarity and presentation. The paper is well-written and easy to follow. The problem is clearly introduced, the relationship to prior work is well articulated, and the notation is consistent throughout.

    • Principled probabilistic formulation. Framing the problem with latent variables that encode the gold segmentation and the annotator-style segmention is neat and clean. It allows a better and principled understanding of this problem.

    • Interpretable annotator characterization. Reducing each annotator to a (mu, sigma) pair enables interpretability and makes the model’s assumptions about rater behavior explicit and auditable. These parameters can support downstream tasks like annotator quality assessment or data curation.

    • Improved calibration. The reported improvement in ECE over PIONONO is a positive point. However, I wonder if this calibration gain came from the annotator modeling or the latent Gaussian Process. Anyway, this kind of results are really useful for trustworthy deployment of AI in the clinic.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    • Insufficient baselines. The empirical evaluation compares the proposed method against only two baselines: a deterministic U-Net and PIONONO. This is a significant gap. The multi-rater segmentation literature has a number of well-established methods that should be included — most notably reference [30] in the paper’s own bibliography, which is one of the foundational works in this area utilizing confusion matrices for annotator modeling. Additional relevant baselines would include Probabilistic U-Net (Kohl et al. , 2019), which also models annotation uncertainty via a latent variable formulation or STAPLE, which is considered the gold-standard for multi-rater segmentation fusion.

    • Single-dataset evaluation. All experiments are conducted on a single, highly specific trans-rectal ultrasound dataset. This severely limits the generalizability of any conclusions drawn from the results. The trans-rectal ultrasound setting has particular characteristics — specific image quality, anatomical structure, and annotator pool — that may or may not be representative of multi-rater disagreement in other modalities or tasks. The fact that performance is nearly equivalent to PIONONO on this dataset makes it especially unclear what the proposed method actually contributes. It is entirely possible that the GP prior is beneficial in certain regimes (e. g. , small datasets, high annotator disagreement, specific noise structures) and detrimental in others. Without evaluation across multiple datasets — such as LIDC-IDRI for lung nodule segmentation, QUBIQ for multi-organ uncertainty, or the datasets used in PIONONO and [30] — the paper cannot make general claims about the value of the proposed approach.

    • Insufficient justification for the GP prior. The choice to use a Gaussian Process as the segmentation model is not adequately motivated. GPs are not a standard choice for pixel-level segmentation: their computational complexity scales cubically with the number of input points in the naive formulation, making them poorly suited for high-resolution medical images without substantial approximation. The paper utilizes 512 inducing points and reports time of processing per image, but a comparative study with PIONONO and baselines at training/inference time would strengthen this design choice.

    • Annotator model cannot capture structure-dependent disagreement. This is the most fundamental conceptual limitation of the paper. The two-parameter annotator model assumes that each rater has a global, structure-agnostic bias and variance. In other words, if rater A tends to over-segment, the model assumes this is true for all structures in all images. This may be empirically false in most real annotation scenarios. In clinical practice, annotators typically agree on unambiguous structures but disagree specifically on ambiguous boundaries, rare pathologies, or regions where their training or domain expertise differs. A rater might consistently over-segment the prostate but have no systematic bias on the seminal vesicles. The current formulation has no capacity to represent this, because mu and sigma are scalar quantities associated with the rater as a whole, not with specific structures or image regions. The current state of the literature, as the authors themselves acknowledge by citing PIONONO and [30], has moved beyond simple over/under-segmentation models toward representations that capture richer, structure-dependent opinion distributions. Although expressed in the limitations of the paper, I encourage the authors to consider this comment to improve their work.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    The authors can improve the paper by expanding the experimental evaluation across multiple datasets and baselines (as the ones commented above), provide a computational analysis of the GP formulation, and design experiments that concretely demonstrate the value of interpretable annotator parameters over deep non-linear mapping produce by deep neural nets.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper is well-written and tackles a relevant problem with a coherent probabilistic framework. The improved ECE and the interpretable annotator parameterization are genuine contributions. I personally value and acknowledge the emphasis on interpretability. However, the evaluation is too narrow to support the paper’s claims: one dataset, two baselines, and results that are similar to an existing method. The methodological choices — particularly the GP prior and the global annotator parameters — are not adequately justified or analyzed.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Reject

  • [Post rebuttal] Please justify your final decision from above.

    After this review process, my main concerns remain: (i) the simplicity of the method and (ii) the single-dataset evaluation. However, as argued by the authors, the lack of a baseline can be overlooked because they compare their approach with current state-of-the-art methods, which should suffice. However, these two concerns, while not decisive individually, together weaken the evidence regarding the usefulness of the reported results. Therefore, I encourage the authors either to develop a more sophisticated model or to provide a more thorough evaluation using the current model.

    The authors acknowledge the use of a single dataset as a limitation and justify the simplicity of the approach by claiming good generalization by simple models (namely, CM global), as reported in PIONONO [20]. However, the results presented in Table 3 are not statistically significant. Therefore, further experimentation is necessary.



Review #2

  • Please describe the contribution of the paper

    The authors propose a framework based on stochastic variational Gaussian processes to model inter-rater variability in medical image segmentation. They apply their framework to segmentation of trans-rectal ultrasound images (249 volumes), approached as a 2D segmentation task (6644 2D slices for the whole annotated train/val/test data).

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Presentation and clarity: The mathematical framework is presented in a sound manner The paper is overall clear

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    • I am not convinced at this stage by the way the inter-rater bias is accounted for, using a single uniform “shift” for the logits independently of the location in the image and anatomical structure. Annotation biases tend to be highly region-dependent with significant variability in the type of “mistakes” depending on the location (might over-segment in one region, under-segment in another, if the segmentation protocol was not fully applied or not clear enough). Thus I am concerned that the modeling might not be complex enough to effectively handle real use-cases.

    • The experimental validation is currently quite limited in scope for a MICCAI paper, both regarding the number and choice of baselines and regarding the number of use cases.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    • Annotator induced logit perturbations p(li|l0i,a) are modeled via a uniform shift in logit, N(l0i + mu_a, sigma_a**2), independent of the location in the image and anatomical structure. This can account for a global over-segmentation for one class vs. another, but it can only very indirectly account for region-dependent biases due to an ambiguous annotation protocol in specific locations. This is a weakness of the work in its current form given that the focus is on inter-rater variability. Only by increasing sigma_a can it account for region-specific effects, in a way that would be inefficient in terms of calibration / NLL of the model.

    • Some simple natural baselines are missing, which make it difficult to assess the relevance of the contribution, for instance (but not only) :

    1.The authors could have trained separate Gaussian process based models similar to the proposed one, separately on the 3 annotator-specific subsets of data, just removing the part that accounts for annotator bias. This would have enabled us to assess if there is a particular benefit in training a model jointly on the 3 annotation sets, accounting for annotation-bias vs. training 3 separate models with similar expressivity to the proposed one.

    2.At the very least, the calibration of the U-Net baseline could be adjusted post-hoc on the validation set by maximizing the ECE; as it is known that standard U-Nets are often poorly calibrated

    3.A more complex baseline, maybe based on hyper-networks could be included: Joutard et al., HyperSORT: Self-Organising Robust Training with hyper-networks, MICCAI 2025

    • I would suggest to expand the number of use-cases using as many of the standard multi-annotator datasets as possible.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (2) Reject — should be rejected, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    In its current form, it seems to me that the weaknesses of the paper outweigh its strengths.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Reject

  • [Post rebuttal] Please justify your final decision from above.

    Many thanks to the authors for taking the time to respond to the points raised by the reviewers. My two main concerns were:

    • lack of important baselines to assess the relevance and effectiveness of the contributions
    • a simplistic modeling of the annotator-bias, as was pointed out by all reviewers In the end, these two concerns remain. I understand the authors’ responses but I would maintain that the missing baselines are essential to assess the quality of the contribution, rather than an addition that might dilute the main point of the paper. Given that the paper is positioned as a proof-of-concept with limited experimental validation, I feel that the above two limitations justify rejecting the paper in its current form.



Review #3

  • Please describe the contribution of the paper

    When training deep learning models for segmentation, annotations are often subjective and vary across annotators. This paper aims to disentangle annotator-specific effects from an underlying, annotator-independent reference prediction. While prior multi-rater methods typically model annotator behavior implicitly through latent variables within deep networks, the main contribution of this work is to explicitly model annotator-specific bias and variability in a probabilistic framework.

    Specifically, the paper proposes a Gaussian process-based formulation that models a distribution over reference logits and introduces annotator-specific perturbations applied to these logits and parameterized by bias and variance. This explicit decomposition enables a more interpretable representation of annotator behavior and allows direct analysis of how annotator variability affects predictive uncertainty.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    S1.The paper proposes a principled probabilistic formulation that explicitly models annotator-specific bias and variability, in contrast to prior approaches that rely on implicit representations learned by deep networks. This design leads to parameters with clear interpretations and improves model transparency.

    S2.The proposed method replaces the standard prediction head of deep segmentation models with a Gaussian Process-based formulation that maps latent features to logits. Combined with annotator-specific Gaussian perturbations and a probit link function, the model enables analytic tractability and closed-form predictive probabilities of segmentation maps. This results in a computationally efficient and conceptually elegant formulation that avoids the need for Monte Carlo sampling-based inference required by alternative approaches (e.g., Pionono, discrete generative models).

    S3.Experimental results show that the proposed method significantly improves the calibration of predictive probabilities, leading to substantially lower NLL while maintaining, and in some cases improving, the segmentation performance compared to deep learning-based methods. This supports the idea that replacing the standard prediction head does not degrade segmentation quality, while providing well-calibrated and interpretable probability estimates. This enables a reliable use of uncertainty for downstream analysis (e.g., active learning, robust clinical decision-making).

    S4.To support the validity and interpretability of learned annotator-specific parameters, the paper provides an interesting analysis of annotator behavior by examining the estimated bias and variance terms. In particular, the results reveal a correlation between the annotator-specific variance and the performance of a deterministic model trained on individual annotators.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    W1.The modeling of annotator-specific behavior appears too simplistic. Equation (2) assumes that annotator-specific variability is conditionally independent of the input image given the reference logits. This seems a limiting assumption: annotator behavior can depend on image-specific factors (e.g., some annotators may be more skilled in noisy regions while others may be more careful delineating complex textured boundaries). Furthermore, Equation (3) factorizes the annotator-specific distribution across pixels, ignoring potential spatial dependencies in annotator patterns (e.g., some annotators may be more prone to find small structures or delineate individual connected components than others). Finally, modeling annotator effects as a global Gaussian perturbation characterized by bias and variance assumes that annotators differ only in terms of systematic over-/under-segmentation and noise levels. While this leads to a simple and interpretable formulation, it may be insufficient to capture more complex and context-dependent annotator behaviors.

    W2.Related to W1, the experimental evaluation is limited to a single dataset (TRUS). This makes it difficult to assess how well the proposed approach generalizes. It remains unclear whether the simplifying assumptions on annotator-specific modeling hold across different image modalities, annotation settings, and annotators. The observed performance in TRUS may be specific to the characteristics of that dataset, where the annotator behavior may be well approximated by global bias and variance. Evaluating the method on one or two additional datasets would have helped to support the generalizability of the method.

    W3.[Minor] While the method is formulated as a principled probabilistic model with analytical tractability, the final training objective combines several heuristic terms (BCE, Dice, and KL), rather than following a purely likelihood-based formulation (e.g., ELBO). Although this loss may be motivated by the need to train the deep feature extractor, it appears somewhat out of place given the otherwise elegant model formulation and could benefit from further clarification.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    I believe the approach has strong potential and could be further extended in future work. It would be valuable to consider:

    • evaluating the method on additional datasets with different annotation characteristics,
    • exploring more expressive models of annotator behavior (e.g., allowing image-dependent or spatially structured variability),
    • clarifying the connection between the probabilistic formulation and the final training objective.
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    While the model relies on relatively strong simplifying assumptions and some aspects of the evaluation could be further improved, I find the proposed approach to be technically sound and conceptually elegant. Its modular design and its theoretically grounded formulation suggest that it could be adapted to more expressive variants if needed, for instance by relaxing some of the assumptions highlighted in W1, without sacrificing the benefits of a principled probabilistic framework and calibrated predictions.

    Overall, I consider the contribution to be relevant and potentially valuable for the MICCAI community in its current form. Nevertheless, I encourage the authors to address the identified limitations for the final version or in a future version.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    In contrast to many methodological MICCAI papers that primarily focus on architectural variations, engineering refinements, or incremental benchmark improvements, this work addresses a more fundamental probabilistic modeling question and proposes an end-to-end interpretable formulation. I do not believe that its main limitations (namely the absence of some baselines and the use of simplifying assumptions) diminish its potential relevance or scientific value.

    While additional baseline comparisons would strengthen the empirical section, I think the main value of the paper is not necessarily in outperforming existing methods, but in showing how interpretable probabilistic modelling can be incorporated into modern medical image segmentation pipelines. In fact, even if the method underperformed relative to other approaches, I would still consider the contribution important, as the paper explores the trade-off between predictive performance and interpretability (a trade-off highly relevant in medical imaging).

    I also think that this work could spark interest in a promising research direction that combines expressive deep feature extractors with interpretable probabilistic modeling. Compared to purely opaque latent representations, such approaches may offer improved transparency, controllability, and scientific insight. This direction remains comparatively underexplored in medical image segmentation despite its potential importance.



Author Feedback

We thank the reviewers and meta-reviewer for their thoughtful comments and for recognising the clarity (R1&R2), the principled probabilistic formulation (R1&R3), the interpretability (R1&R3), and the improved model calibration (R1&R3). We address reviewers’ concerns below.

(1) Clarification of main contribution (R1). Our aim is to explain how annotation bias and variability propagate to the predictive distribution, whereas many existing multi-rater methods primarily emulate rater-specific segmentations with less emphasis on interpretability. Their use of implicit annotator representations and deep nonlinear mappings often makes the predictive distribution analytically intractable and requires Monte Carlo approximation. Thus, comparable performance to Pionono [20] does not diminish the contribution. In fact, it shows that our method achieves competitive accuracy while making annotator effects more interpretable, as supported by the correlation between learned annotator-specific variance and individual-annotator model performance (Page 7).

(2) Insufficient baselines (R1&R2). As clarified above, the proposed method addresses a different methodological objective from many existing multi-rater segmentation methods, including those suggested by R1.Likewise, the baselines mentioned by R2 address useful but complementary questions rather than the central interpretability claim. Further expanding the baseline set with different primary aims would dilute the interpretability focus, which is difficult to accommodate under the current space limit and will therefore be left to a journal extension. Moreover, our selected baseline, Pionono, is a recent probabilistic multi-rater method that already reports favourable performance over STAPLE, Probabilistic U-Net, and global/pixel-wise confusion-matrix baselines. Thus, comparison with Pionono contextualises our method against prior multi-rater approaches while preserving the focus on the proposed interpretability-oriented contribution.

(3) Single-dataset validation (R1&R2&R3). The present study is intended as a proof-of-concept for the proposed interpretable probabilistic framework, demonstrated on a clinical-relevant multi-rater segmentation dataset. As noted by R3, the modular design allows the framework to be extended for broader settings. We will clarify this scope and avoid overstating the generality of the empirical findings in the revision.

(4) Simplistic assumption (R1&R2&R3). We appreciate the concern that global annotator-specific parameters may not capture spatially or image-dependent disagreement. However, a more expressive annotator model does not necessarily guarantee better empirical performance. For example, as reported in Table 3 of Pionono [20], the simpler global confusion-matrix model showed better cross-dataset generalisation than the pixel-wise counterpart. This suggests that simplified annotator models can still provide robust approximations, possibly because they are less prone to overfitting dataset-specific annotation patterns. Overall, the trade-off between model complexity and generalisability remains a broader open question, and requires dedicated investigation beyond the scope of the present work.

(5) Justification of GP prior (R1). The GP prior was chosen because it provides a non-parametric, data-adaptive prior over the reference logits and empirically outperformed the alternatives we considered. Computationally, we adopted the scalable formulation in [10] rather than a naive implementation with cubic complexity, making the method feasible in practice. We will add a clearer discussion on training/inference cost in the revision.

(6) Training objective (R3). BCE corresponds to the expected negative log-likelihood under a Bernoulli observation model, so BCE+KL can be interpreted as an ELBO objective. Dice loss is added empirically to support segmentation training, following common practice. We will make this explicit in the revision.



Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    All three reviewers were positive about the work, however, several concerns were raised mostly on the limited experimental validation: is it sufficient to support the papers claims? This led to mixed reviews. I suggest the authors get a chance to address the raised concerns in a rebuttal.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    This is a tough call. Two reviewers mention the lack of proper baselines and the use of a single data set, which is in rebuttled by the authors by pointing out that improving on baselines is not the main contribution of the paper, rather it addresses a more fundamental question on probabilistic modeling. In spite of 2 out of 3 votes to reject, I would still suggest to accept this paper, motivated by the justification of Reviewer 3, who voted to accept and argued that this work could spark an interesting research direction and, with that, may have more impact that papers that merely improve on baselines with architectural variations.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Reject

  • Please justify your recommendation.

    While R3 makes a case, 2 out of 3 vote for reject with similar reasoning, and I do not feel as though it is correct to overrule them here, as the suggestions by R1 and R2 are actionable. I encourage the authors to resubmit with additional baselines and more extensive experimentation.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Reject

  • Please justify your recommendation.

    I agree with R3 that the probabilistic framing of the method has interpretability potential despite the limited results. However, I must agree with the other reviewers that the paper in its current form (one dataset and minimal baselines comparisons) is not sufficiently developed to be presented at the main conference. I hope that the authors further develop their framework and broaden the empirical scope in future work.



back to top