Abstract
Accurate nuclei instance segmentation in fluorescence microscopy is essential for quantitative cell analysis, yet models trained on a single dataset often degrade under changes in staining, density, morphology, and acquisition
protocol. We present FluoViT, a foundation model specifically tailored for
robust fluorescence nuclei segmentation. Built upon a DINO-pretrained ViT- 256 backbone, FluoViT learns a universal cellular representation by jointly
training across diverse large-scale datasets (BBBC039, DSB2018-Fluo, and S- BSST265). To ensure this foundation model scales effectively to highly dense
and overlapping cell populations, we introduce a lightweight distance head that regresses a normalized internal Euclidean distance map under a Smooth L1
objective. At inference, these distance peaks act as precise watershed markers, dynamically fused with horizontal-vertical (HV) evidence in a marker- controlled watershed decoder to separate dense and touching nuclei. Furthermore, we adopt an inverse cell-count weighted sampling strategy during multi-source pre-training to mitigate dataset and density biases. Extensive
experiments validate FluoViT’s foundational capabilities: on the in-domain Core-3 validation, it achieves a 0.7483 binary panoptic quality (bPQ), outperforming CellViT (0.7325). More importantly, demonstrating its strong
out-of-distribution generalization as a foundation model, FluoViT establishes
state-of-the-art performance on an entirely unseen clinical cohort (CAC40X) under domain shift, attaining the highest bPQ (0.5324) and binary detection
quality (0.7631 versus 0.7033 for the strongest baseline).
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026-sat/paper/COMPAYL_008.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to Open Review
BibTex
@InProceedings{KuaYin_DistanceGuided_MICCAISAT2026,
author = { Kuang, Yinglan AND Lu, Xing},
title = { { Distance-Guided Vision Transformer based Foundation Model for Fluorescence Nuclei Instance Segmentation } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026 Workshops and Challenges},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 17251},
month = {pending},
page = {pending}
}