Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Breast ultrasound is widely used for screening, yet accurate annotations are costly, motivating anomaly detection methods trained only on normal data. Reverse Distillation (RD) is a popular teacher–student paradigm, but most RD approaches enforce global cosine consistency during training while relying on pixel-wise discrepancies at test time, leading to an objective mismatch. Moreover, directly optimizing cosine similarity is prone to poor local optima, which reduces anomaly sensitivity. To address these issues, we propose the Discriminative Directional Projection Loss (DDP Loss). First, we introduce a directional subspace projection mechanism, where a sampler generates directional bases along the channel dimension to project teacher and student features into a directional subspace. Pixel-wise consistency distillation is then enforced within this subspace to better align the training and testing objectives. Second, we design a Hard-Direction Mining strategy that alternates optimization to actively search for directions with the largest teacher–student discrepancy, thereby strengthening discriminative constraints. Third, we incorporate an orthogonality regularizer to promote directional diversity, mitigating direction collapse and improving subspace representational capacity. Experiments on BUS-UCLM and BUSI-WHU show that DDP Loss consistently improves AUC, F1, and FPR@TPR95 without extra architectures or pseudo anomalies. These results validate the effectiveness of improving the reverse distillation paradigm at the objective level and highlight its potential clinical value.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/6664_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{XuShi_Discriminative_MICCAI2026,
        author = { Xu, Shicheng AND Li, Jincheng AND Lin, Junhao AND Li, Zuoyong AND Lan, Hengrong AND Gao, Fei},
        title = { { Discriminative Directional Projection for Reverse Distillation in Breast Ultrasound Anomaly Detection } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16885},
        month = {September},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper
    • Identifies a key limitation of prior reverse distillation methods for breast ultrasound anomaly detection, namely the mismatch between the global training objective and pixel-wise inference-time anomaly scoring.
    • Proposes Discriminative Directional Projection (DDP) Loss, which projects teacher–student features into a directional subspace and enforces pixel-wise consistency in that space.
    • Introduces hard-direction mining and orthogonality regularization to encourage more discriminative anomaly representations.
  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The paper addresses a limitation of prior reverse distillation methods at the objective level, rather than only reporting empirical improvements.
    • The idea of replacing global cosine alignment with directional pixel-wise consistency is intuitive and methodologically interesting.
    • The alternating optimization strategy for mining hard directions is meaningful, as it reduces reliance on overly easy alignment directions.
    • The method shows competitive overall performance, and the ablation results reasonably support the contribution of each component.
    • The gains are achieved without substantial architectural changes, which is a practical advantage.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • Validation is limited to only two breast ultrasound datasets, so the generalizability of the method remains unclear.

    • If the paper primarily targets breast ultrasound anomaly detection, comparisons with stronger non-RD baselines, such as diffusion-based UAD methods (e. g. , REFLECT [1], AnoDDPM [2]), would strengthen the study. [1] Beizaee, F. , Hajimiri, S. , Ben Ayed, I. , Lodygensky, G. , Desrosiers, C. , & Dolz, J. (2025, September). Reflect: Re ctified Fl ows for E fficient Brain Anomaly C orrection T ransport. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 456-466). Cham: Springer Nature Switzerland. [2] Wyatt, J. , Leach, A. , Schmon, S. M. , & Willcocks, C. G. (2022). Anoddpm: Anomaly detection with denoising diffusion probabilistic models using simplex noise. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 650-656).

    • If the main claim is an improvement to the RD framework itself, additional validation on other domains and datasets is needed.
    • It is currently unclear why RD is particularly the most suitable formulation for ultrasound anomaly detection, or whether the proposed loss generalizes broadly across RD settings.
    • Regarding Figure 3, the qualitative examples appear relatively easy; showing improvements on more challenging or ambiguous cases would make the evidence more convincing.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Please refer to the weaknesses.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    I have read the authors’ rebuttal and appreciate their detailed responses. While some of my concerns remain only partially addressed, I still consider the paper to be marginally on the acceptable side.



Review #2

  • Please describe the contribution of the paper

    This paper proposes the Discriminative Directional Projection Loss (DDP Loss) to improve reverse distillation-based anomaly detection for breast ultrasound images.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The inclusion of FPR@TPR95 and consistent improvements on this metric provide meaningful clinical grounding beyond standard AUC reporting. 2.The stepwise ablation shows monotonic improvements from each component, reasonably supporting the design choices.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.Class definitions and splits for BUSI-WHU are never stated, i.e., what constitutes “normal” vs. “anomaly” is ambiguous, undermining result interpretability. 2.SCRD4AD achieves near-chance AUC (~63%) but high ACC, and Skip-TS shows high AUC but ~53% ACC, suggesting reproduction or evaluation protocol issues that weaken the claimed improvements. 3.key parameters (λ_orth, update frequency) are unreported.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Reasonable motivation and lightweight design, but unclear dataset definitions, questionable baseline reproductions, unablated critical hyperparameter , and Fig. 4 missing Skip-TS from the ROC comparison collectively undermine reliability and completeness.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The reviewers’ responses have largely satisfied my concerns, and I recommend acceptance.



Review #3

  • Please describe the contribution of the paper

    This paper proposes a Discriminative Directional Projection Loss to address the train-test objective mismatch and local optimum traps inherent in reverse distillation methods for breast ultrasound anomaly detection. By introducing directional subspace projection, a hard direction mining strategy, and an orthogonality regularization term, the proposed method aligns the training and testing objectives while enhancing the discriminative capability of anomalous features.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.Traditional Reverse Distillation methods impose global cosine consistency constraints on flattened features during training, yet rely on pixel-wise discrepancies during testing, leading to a significant objective mismatch. The proposed directional subspace projection mechanism maps features into a directional subspace to perform pixel-wise consistency distillation, effectively bridging this gap. 2.Directly optimizing cosine similarity is prone to falling into poor local optima. The authors designed an alternating optimization hard direction mining strategy, which actively identifies the directions with the greatest discrepancy between the teacher and student models. This prevents the optimization process from stalling on easily aligned directions. 3.Experimental results on the BUS-UCLM and BUSI-WHU datasets demonstrate consistent improvements across key metrics, including AUC, F1-score and Accuracy. Notably, the proposed method achieves the lowest False Positive Rate, while maintaining a True Positive Rate of 0.95.This is of significant clinical value for reducing unnecessary follow-ups in breast cancer screening.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.Inadequate Analysis of Hyperparameter Sensitivity: Setting the number of projection directions to K=4096 is a critical design choice,yet the paper lacks corresponding ablation studies.It remains unclear how this specific hyperparameter impacts model performance (e.g., assuming it conforms to the channel dimensions, would K=512 or K=1024 yield similar or better results?). The authors need to provide a comprehensive sensitivity analysis for K. 2.Significant VRAM Consumption and Computational Overhead: Although the sampler is described as “lightweight”, projecting features into a high-dimensional space (4096HW) to compute pixel-wise cosine similarity will inevitably lead to massive VRAM consumption during both training and inference. This is particularly concerning for high-resolution medical ultrasound images and is highly prone to causing CUDA Out-Of-Memory errors in practical deployments. The authors should provide detailed empirical data on VRAM usage and computational complexity (e.g., FLOPs) to justify the clinical deployability of this method on standard hardware. 3.Reproducibility Concerns Regarding Noise Initialization: The sampler utilizes Gaussian noise to generate channel directions. Because anomaly detection tasks are highly sensitive to subtle feature deviations, the stability of this random initialization is questionable. While orthogonal regularization helps prevent direction collapse, the authors must provide variance,standard deviation metrics across multiple runs with different random seeds to prove that the hard-direction mining strategy is genuinely effective and stable, rather than a result of lucky initialization. 4.Mathematical Paradox in Orthogonal Regularization: The paper sets the number of projection directions to K=4096 and introduces an orthogonal regularization term to encourage the Gram matrix of normalized directions to approach the identity matrix I. However, there is a fundamental mathematical contradiction here: the multi-scale feature channel counts are only 256, 512, and 1024.In a linear space with a maximum dimensionality of 1024, it is mathematically impossible to sustain 4096 mutually orthogonal vectors. Since the rank of the Gram matrix is upper-bounded by the channel dimension, it can never truly approximate a rank-4096 identity matrix. The authors must address this theoretical inconsistency.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    While the overall presentation of the figures in the paper is satisfactory, Figures 1 and 2 appear somewhat visually crowded. For the camera-ready version, I strongly recommend standardizing the visual style of the architectural diagrams. Optimizing the routing of connection lines, utilizing soft color palettes, and ensuring clean backgrounds will significantly enhance the paper’s readability and deliver a more professional academic presentation. Additionally, carefully reviewing the mathematical feasibility of the proposed dimensional expansions will greatly strengthen the theoretical foundation of this work.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper accurately identifies the core issues inherent in current RD methods for anomaly detection and proposes a custom loss function framework. The performance of the proposed method in reducing the False Positive Rate shows clinical value. However, my “Weak Accept” rating is highly conditional. The current manuscript presents a fundamental mathematical contradiction regarding the orthogonal regularization, lacks critical ablation analysis on the projection dimension parameter, and raises significant concerns regarding VRAM consumption, computational overhead and initialization stability. I evaluate this paper as a borderline Weak Accept. If the authors can provide a mathematically rigorous explanation for the dimensional paradox and adequately address the deployment concerns during the Rebuttal phase, I am willing to raise my rating. Conversely, if these theoretical and practical flaws remain unresolved, I will reconsider my score.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    Most of my concerns have been addressed, so I will retain the original score.



Author Feedback

We thank the reviewers and the meta-reviewer for their constructive comments. Following the meta-review, we organize our response around theoretical consistency, experimental clarity, baseline reliability, and other remaining points. 1.For the theoretical consistency of the orthogonality regularization, (R3.4) we agree that when K>C, the Gram matrix of normalized directions has rank at most C, and therefore it cannot exactly approximate a full-rank K×K identity matrix. The phrase approach the identity in Eq. (6) was imprecise and may incorrectly suggest an unrealizable strict orthogonality constraint. In fact, because each direction vector is L2-normalized, the diagonal entries of the Gram matrix are already fixed to one. Thus, Eq. (6) mainly penalizes the off-diagonal correlations between projection directions, making R_orth a soft inter-direction decorrelation constraint that reduces redundancy and alleviates direction collapse, rather than enforcing K strictly orthogonal directions. The rank limitation only implies that this regularizer cannot reach zero when K>C, but it does not invalidate its role in promoting directional diversity. (R3.1) K controls the number of candidate projection directions explored during hard-direction mining, rather than the dimension of an orthogonal basis. We use K=4096 to provide denser directional coverage for discovering discriminative teacher-student (TS) discrepancies, reflecting a trade-off between coverage and computational cost. We will clarify this design choice in the revision. (R3.3) The Gaussian noise is only the input to a learnable sampler, rather than fixed random directions. The final directions are learned through hard-direction mining, so the method is not designed to rely on lucky initialization. Moreover, using a large candidate direction set reduces dependence on any single initialized direction. 2.For the clarity of the experimental setup and dataset definitions, (R2.1) in our setting, normal breast ultrasound images are treated as the normal class, while all lesion-containing images, including benign and malignant cases, are treated as anomalies. We will explicitly report the class mapping, sample counts, and train/test splits. (R2.3) We used λ_orth=0.1 and an alternating schedule of 10 student updates followed by 1 sampler update. We will add these omitted implementation details in the revision. (R3.2) Projection-based scoring adds computation, but profiling under our test-time setting shows manageable overhead: the sampler has only 4.1M parameters, about 2.5% of the 165M full model, while full inference requires 108G FLOPs and 1.1 GB peak CUDA memory. 3.For the reliability of the baseline comparisons, (R2.2) AUC evaluates score ranking across thresholds, whereas ACC and F1 reflect threshold-based classification, so they need not vary consistently. We followed the evaluation protocol of prior UAD works and rechecked the code and data splits. Skip-TS and SCRD4AD show consistent trends on both datasets, suggesting that these differences arise from their score distributions rather than evaluation errors. 4.For other remaining issues, (R1.1/R1.3/R1.5) we agree that broader validation across anatomical regions, modalities, datasets, and more challenging qualitative cases would strengthen the evidence of generalizability. However, MICCAI rebuttal guidelines do not allow additional experiments or substantial experimental changes. Our current study therefore focuses on breast ultrasound anomaly detection, a clinically meaningful and annotation-scarce setting, to examine whether DDP Loss improves the RD paradigm. (R1.2/R1.4) Diffusion-based methods are valuable, but they belong to a different generative paradigm. Since our contribution is a loss-level improvement for RD, we compare mainly with representative RD-based methods. RD is suitable here because normal-only TS distillation matches scarce abnormal annotations and yields interpretable multi-scale discrepancy cues.



Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The motivation is sound and the ablation credibly supports each component. The authors should address the most important points raised by the reviewers, especially with regard to the theoretical consistency of the orthogonality regularization, the clarity of the experimental setup and dataset definitions, and the reliability of the baseline comparisons.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    All three reviewers moved to accept post-rebuttal. The theoretical inconsistency in the orthogonality regularization was resolved convincingly, dataset definitions and hyperparameters were clarified, and computational overhead was shown to be manageable. The rebuttal addressed the most critical concerns raised in the original reviews. As AC, I note one important limitation that should be acknowledged in the final version. The contribution is evaluated exclusively within the RD paradigm, and no comparison against diffusion-based UAD methods is provided despite these methods addressing the same task under the same normal-only training assumption. Establishing where DDP Loss stands relative to the broader UAD landscape, not just within RD, would substantially strengthen the clinical relevance claim and help readers contextualize the contribution appropriately.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    This paper presents a well-motivated and effective methodological refinement for medical image anomaly detection. The authors’ comprehensive rebuttal successfully resolved initial reviewer concerns regarding theoretical consistency, experimental clarity, and baseline evaluation. The proposed approach offers a practical, efficient solution that demonstrates strong clinical relevance and clear performance improvements. Following the rebuttal, all reviewers unanimously recommend acceptance.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The reviewers are satisfied with the rebuttal.



back to top