Abstract

Light sheet fluorescence microscopy (LSM) enables high-resolution, three-dimensional (3D) imaging of biological specimens, providing rich volumetric data for studying cellular organization, pathology, and vascular networks. However, the size, dimensionality, and annotation burden of LSM data make supervised deep learning approaches costly and difficult to scale. Additionally, despite the abundance of unannotated LSM volumes, foundation models for this modality remain underexplored due to computational challenges and the complexity of volumetric representation learning. In this work, we introduce a 3D foundation model for LSM data, pretrained on a large curated collection of 3D images spanning multiple organisms, stains, and imaging protocols. We learn transferable volumetric representations by jointly optimizing for masked reconstruction and image-text alignment. The pretrained backbone drastically reduces the annotation burden, enabling efficient, few-shot adaptation for varied downstream tasks. We evaluate this approach on downstream segmentation, classification, and deblurring. Our results demonstrate consistent improvements over baselines, (1) when measured using standard evaluation metrics and (2) when rigorously assessed by domain experts. This highlights the potential of foundation model pretraining to reduce annotation requirements while improving performance across diverse LSM analysis tasks. Pretrained model weights and code for pretraining and finetuning are publicly available: https://github.com/AdinaScheinfeld/lsm_fm_public_repo.git.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/4243_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/AdinaScheinfeld/lsm_fm_public_repo.git

Link to the Dataset(s)

https://zenodo.org/records/20149070

BibTex

@InProceedings{SchAdi_AMultimodal_MICCAI2026,
        author = { Scheinfeld, Adina AND Zhang, Haotan AND Mu, Shang AND van Herten, Rudolf L. M. AND Stoffl, Lucas AND Ertürk, Ali AND Wu, Zhuhao AND Paetzold, Johannes C.},
        title = { { A Multimodal 3D Foundation Model for Light Sheet Fluorescence Microscopy Enables Few-Shot Segmentation, Classification, and Deblurring } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16879},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    The paper presents a foundation model for light sheet microscopy leveraging unannotated volumetric data and text descriptions to enable efficient and improved few-shot and many-shot learning across downstream tasks. The results of extensive experiments show that foundation model pretraining enables effective transfer to segmentation, classification, and deblurring tasks with minimal annotation. Pretrained model weights and code for pretraining and fine-tuning are made publicly available.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Well writen paper addressing an important need in biological image analysis. Foundation models for microscopy are most welcome especially for 3D analysis (segmentation, classification, deblurring).
    • Extensive efforts have been made to compile a dataset for pretraining and fine-tuning of the foundation model.
    • Likewise, extensive results are presented for three downstream tasks (segmentation, classification, deblurring) with promising results, including the relative importance of image versus text cues.
    • Evaluation is based not only on standard quantitative metrics but also human expert assessment.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • No new methods are presented. All presented work is based on existing methods and mostly also on existing datasets (except for the internal data). This is primarly an application paper and the main contribution is the resulting foundation model.
    • While the pretrained model weights and the code for pretraining and fine-tuning are made publicly available, it is not clear whether the actual data used in both will also be made available, or at least the recipe (code) for extracting the patches from the public datasets. The authors do state that the training patches from the internal dataset will be made available.
    • No mention is made of the computation times involved in the pretraining, fine-tuning, and inference for the different tasks.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    It would strengthen the paper if the weaknesses mentioned above were addressed.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall this is a strong experimental paper making a practically useful contribution but with little methodological contribution.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    The authors present a multimodal 3D self-supervised framework for Light-Sheet Fluorescence Microscopy (LSM). To achieve this, they curate a dataset of unannotated 3D LSM volumes from both internal and public sources, spanning various organisms, stains, and imaging protocols. A key novelty is the pairing of these volumetric patches with expert-written (and LLM-paraphrased) text descriptions. The models (UNet and SwinUNETR) are pretrained using a joint objective that combines masked image reconstruction, exponential moving average (EMA) teacher distillation, and image-text alignment. The resulting models are evaluated on downstream tasks including voxel-wise segmentation, patch-level classification, and deblurring, showing improvements over from-scratch models and task-specific baselines.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The paper introduces an innovative approach by combining free-text semantic descriptions with 3D structural data for light-sheet fluorescence microscopy, effectively bridging the gap between morphological features and biological semantics. 2.The methodology is tested across different architectures, specifically comparing convolutional networks and transformer-based models to show how different backbones benefit from the pretraining strategy. 3.The inclusion of a blinded qualitative assessment by six domain experts goes beyond standard quantitative metrics and provides strong evidence that the model predictions have genuine practical utility. 4.The framework demonstrates versatility by successfully adapting to fundamentally different downstream tasks including voxel-wise segmentation, patch-level classification, and image deblurring.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The claim of creating a foundation model is significantly overstated because the pretraining dataset consists of only 1023 small patches, which equates to a data scale much smaller than what is typically expected for true foundation models. 2.The evaluation relies on a dangerously small test set of only two patches per datatype per fold during cross-validation, and the absence of robust external validation makes it difficult to definitively prove the model can generalize to unseen data. 3.Because the model is evaluated in such an extreme small-data regime, it remains unclear whether the learned representations would genuinely translate to completely different tissue types or novel morphological structures without further extensive pretraining. 4.The quantitative results show that adding the textual modality sometimes degrades performance or fails to improve upon the image-only pretraining, and the authors do not adequately explain why this happens in certain structural downstream tasks.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The authors introduce a highly compelling, methodologically sound framework for 3D LSM pretraining. The idea of linking high-resolution whole-brain microscopy patches with semantic text prompts is innovative and well-executed, and the blinded domain-expert evaluation is a standout addition.

    However, calling this a “Foundation Model” is a significant overstatement given the severely limited scale of both the pretraining data (1,023 patches) and the downstream evaluation test sets (2 patches per fold). Despite these scale limitations, the paper serves as an excellent proof-of-concept that multimodal pretraining can reduce annotation burdens in the difficult domain of 3D microscopy. With minor tone adjustments regarding the “Foundation” claims, this paper provides valuable insights and baselines for the biomedical imaging community.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #3

  • Please describe the contribution of the paper

    This work has innovation and practical value. Targeting LSM — an important modality where foundation models remain relatively underexplored — the authors propose a 3D foundation model that combines image-text multimodal pretraining. The experimental design is reasonable, and both code and pretrained weights are publicly released, meeting MICCAI’s high standards for methodological innovation, practical application, and reproducibility.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The integration of masked reconstruction, teacher-student distillation, and CLIP-style image-text alignment for 3D LSM volumetric data effectively exploits the high signal-to-noise ratio and abundant unlabeled data of LSM, demonstrating a degree of novelty. 2.In few-shot settings, the proposed models significantly outperform training-from-scratch baselines and strong existing methods (uSAM, Cellpose, CellSeg3D). The improvements are further validated by blinded evaluations from domain experts, confirming the clinical and biological relevance of the quantitative gains.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The authors describe their collection as a “large curated collection,” yet it consists of only 1,023 patches of size 96³ (although sourced from multiple origins). This scale remains modest compared with the million-scale datasets typical for natural-image or medical-CT foundation models. A discussion in the Discussion section on the impact of dataset scale on representation quality, along with an explanation of why this scale is sufficient for the LSM scenario, would be beneficial. 2.The motivation and interpretability of the method are relatively weak. The core idea is that incorporating a language modality during pretraining yields better representations. However, the language modality is not directly utilized in any downstream task. It remains unclear how to establish a direct causal relationship between the linguistic priors and the observed improvements in downstream performance; further experiments are needed. The ablation study only compares image-only versus image+text pretraining. No systematic ablation is performed on text prompt quality (domain-expert written versus LLM-paraphrased). The influence of different text prompts on the quality of the pretrained model is not examined. Intuitive evidence, such as higher-quality feature clusters or clearer embedding visualizations when the text modality is included, would strengthen the interpretability of the method. 3.Generalization discussion: The current evaluation is primarily limited to brain tissue datasets from SELMA3D and related sources. Further discussion (or experiments) on generalization to other organs or imaging protocols would be valuable.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The contributions are clear, but the innovation is insufficient.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Author Feedback

We thank the reviewers for their thoughtful feedback. We are pleased that the reviewers recognize the value of our contribution to analysis of 3D LSM data and address key concerns below.

On “foundation model” framing and dataset scale [R1,R2,R3,MR]: We wish to clarify why 1023 patches represents a meaningful pretraining corpus for LSM, even if modest by standards of other fields. In LSM, each image patch is a volumetric sample capturing rich biological structure, a fundamentally more information-dense unit than a 2D natural image crop. Furthermore, the pretraining set spans multiple organisms, tissue types, stains, staining protocols, and imaging systems, making it diverse across biologically meaningful axes even if absolute patch count is modest. Finally, in coining the term, Bommasani et al. (2022) defines a foundation model as one that “is itself incomplete but serves as the common basis from which many task-specific models are built via adaptation.” We invoke this original definition by pretraining our model on heterogeneous unannotated data to serve as a reusable basis, finetuned for varied downstream tasks rather than trained end-to-end for a single task. While scaling to larger collections remains an important open direction, we maintain that the term “foundation model” is appropriate here.

On generalization and test set size [R2,R3,MR]: We agree that finetuning evaluation draws data exclusively from the SELMA challenge. The justification is 2-fold. First, the datatypes reflect diverse structural regimes common in LSM: amyloid plaques are sparse and punctate, nuclei are dense and punctate, and vessels reflect a network-type morphology. Together, the set spans the breadth of spatial organizations in LSM. Second, while distributed together, images from each datatype were sourced from different labs, reflecting varied preparation and imaging protocols, providing more diversity than a single-source evaluation would suggest. Broader evaluation would be valuable but remains challenging due to the scarcity of annotated LSM data, which is precisely what our work aims to address.

Likewise, we acknowledge that 2 test patches per datatype per fold is small, and results should be interpreted accordingly. This too is a direct consequence of the annotation burden in LSM. We note that the combination of 3-fold cross-validation and blinded expert evaluation help mitigate this concern.

On the contribution of the text modality [R2,R3,MR]: Consistent with prior CLIP-style work demonstrating that image-text alignment improves learned representations even when text is absent at test-time, here too, the benefit of the text is specific to representation shaping during pretraining. That said, exploring how text descriptions affect downstream task performance if integrated at inference time is an interesting future direction. For instance, we hypothesize segmentation performance would improve if descriptions of the stain or morphological context were provided alongside the input volume.

Regarding R3’s suggestion of prompt quality ablations, we previously experimented with using different LLMs to reword each text prompt in multiple ways, randomly selecting one rephrased prompt per training pass. This did not substantially affect performance in either direction, and we therefore opted to exclude the experiment from the manuscript.

On data release and reproducibility [R1,MR]: All components needed to reproduce our experiments or extend them to new datasets are available in our GitHub repository.

On computation times [R1]: As noted in the manuscript, pretraining required 10-12 hours on 2 A100/H100 GPUs. Finetuning times vary by task and dataset size but range from a few minutes to 2 hours on a single GPU. Inference is near-real-time per patch.

We again thank the reviewers for their careful assessment and hope these clarifications address the raised concerns while accurately characterizing both the contributions and limitations of our work.




Meta-Review

Meta-review #1

  • Your recommendation

    Provisional Accept

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper proposes a multimodal pretraining framework (here presented as foundation model) for light-sheet microscopy, combining 3D image data with text to improve performance on segmentation, classification, and deblurring tasks. All reviewers find the work valuable and highlight the strong experimental validation, where the proposed method obtains promising results against relevant baselines on several downstream tasks based on quantitative and human expert assessments. The release of dataset, code and models further strengthens its impact and reproducibility.

    However, the reviewers agree the paper is best viewed as a proof-of-concept rather than a true foundation model, especially given the limited dataset scale. Moreover, some details/explainations might be missing about the contribution of the text modality and about the limited generalisation assessment.

    Overall, this submission should be seen as an application paper rather than a methodological study. In this context, it offers a solid and useful empirical contribution. Therefore, I agree with the reviewers that the paper will foster interesting discussions and recommend acceptance. However, I strongly advise the authors to soften their claims, and clarify the limitations in terms of dataset scale, contribution of the text modality, and generalisation.



back to top