Abstract

Fetal ultrasound (US) is the primary imaging modality for prenatal screening, yet its interpretation relies heavily on the expertise of the clinician. Despite advances in deep learning and foundation models, existing automated tools for fetal US analysis struggle to balance task-specific accuracy with the whole-process versatility required to support end-to-end clinical workflows. To solve the problem, we propose FetalAgents, the first multi-agent system for comprehensive fetal US analysis. Through a lightweight, agentic coordination framework, FetalAgents dynamically orchestrates specialized vision experts to maximize performance across diagnosis, measurement, and segmentation. Furthermore, FetalAgents advances beyond static image analysis by supporting end-to-end video stream summarization, where keyframes are automatically identified across multiple anatomical planes, analyzed by coordinated experts, and synthesized with patient metadata into a structured clinical report. Extensive multi-center external evaluations across eight clinical tasks demonstrate that FetalAgents consistently delivers the most robust and accurate performance when compared against specialized models and multimodal large language models (MLLMs), ultimately providing an auditable, workflow-aligned solution for fetal ultrasound analysis and reporting. Source code is available at: https://github.com/birthlab/FetalAgents.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/0018_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/birthlab/FetalAgents

Link to the Dataset(s)

N/A

BibTex

@InProceedings{HuXia_FetalAgents_MICCAI2026,
        author = { Hu, Xiaotian AND Huang, Junwei AND Liu, Mingxuan AND Anmahapong, Kasidit AND Chen, Yifei AND Luo, Yitong AND Huang, Yiming AND Bai, Xuguang AND Li, Zihan AND Liao, Yi AND Qu, Haibo AND Tian, Qiyuan},
        title = { { FetalAgents: A Multi-Agent System for Fetal Ultrasound Image and Video Analysis } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16879},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper

    This paper presents a multi-agent AI framework for automated fetal ultrasound analysis, with the aim of supporting clinicians across the full diagnostic workflow. The system is structured around a set of specialized agents, each responsible for a distinct task such as plane classification, anatomical segmentation, biometric measurement, and report generation. A coordinator module orchestrates these agents by routing inputs—such as ultrasound images or video sequences—and determining the appropriate sequence of operations. The inclusion of a video understanding component allows the system to incorporate temporal information, which is particularly relevant in ultrasound where image quality varies across frames. Outputs from the individual modules are integrated by a summarization agent to produce structured clinical reports, thereby aligning the system with real-world clinical needs.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    A key strength of the work lies in its clinically grounded design. Rather than focusing on a single isolated task, the authors attempt to replicate the end-to-end workflow followed by practitioners, beginning with plane identification and proceeding through segmentation and measurement to final reporting. This makes the system more relevant for deployment settings. The modular, multi-agent formulation is another positive aspect, as it improves interpretability and allows individual components to be refined independently. For instance, segmentation errors in fetal abdomen detection can be addressed without retraining the entire system. The integration of multiple tasks into a coherent pipeline is also noteworthy, particularly the way segmentation outputs are utilized for downstream biometric calculations such as head or abdominal circumference. Additionally, the use of video data, as opposed to only static images, reflects a practical understanding of how ultrasound examinations are conducted and has the potential to improve robustness through temporal aggregation. The emphasis on generating structured clinical summaries further enhances the usability of the system from a clinician’s perspective.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    At the same time, the paper has certain limitations. The individual technical components largely rely on established architectures, such as CNN- and ResNet-based models, and do not introduce significant methodological novelty. As a result, the primary contribution is system-level integration rather than algorithmic advancement. Furthermore, the interaction between the different agents is not described with sufficient clarity. It remains unclear whether the coordination mechanism is rule-based or learned, and how decisions are made when multiple tasks are applicable. This lack of detail makes it difficult to fully assess the novelty and robustness of the proposed multi-agent framework. Finally, the evaluation appears to be conducted at the level of individual tasks, such as classification and segmentation, without a comprehensive assessment of end-to-end system performance. In the absence of such evaluation, it is challenging to determine how effectively the integrated system performs in realistic clinical scenarios.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall, the paper offers a meaningful contribution from a systems and application standpoint, particularly in its attempt to bridge multiple stages of fetal ultrasound analysis within a unified framework. Its strengths lie in clinical relevance and thoughtful integration, while its limitations are primarily related to novelty and depth of evaluation.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    This paper presents FetalAgents, a multi-agent system for comprehensive fetal ultrasound (US) analysis, integrating specialized vision models with a reasoning LLM to support a full clinical workflow, including plane classification, segmentation, biometry, and report generation from both images and video streams. The system dynamically orchestrates multiple task-specific models and aggregates their outputs into structured clinical reports. Extensive evaluation across multiple external datasets demonstrates consistent improvements over standalone models and recent multimodal approaches.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.Strong clinical motivation and workflow alignment The paper addresses a real and important clinical need: integrating multiple fetal ultrasound tasks into a unified automated workflow. Unlike many prior works focusing on isolated tasks, this system-level approach better reflects actual clinical practice and is a clear step toward deployable solutions. 2.Broad and well-structured evaluation The authors provide extensive evaluation across eight clinical tasks on external datasets (Table 1), including classification, segmentation, and biometry. This breadth of validation is uncommon and significantly strengthens the empirical contribution. 3.Consistent performance improvements The method demonstrates consistent improvements over strong baselines, including foundation models and task-specific architectures. These gains are particularly meaningful in biometry tasks and are supported by multi-center external validation. 4.Well-engineered system design The use of structured JSON outputs, deterministic fusion rules, and cross-task consistency checks (e.g., HC/AC vs GA) is a strong design choice that improves robustness and reduces hallucination risk. The modular structure is clear and practical 5.Practical relevance The inclusion of video summarization and structured report generation is particularly valuable and aligns with real-world

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    Overall, this is a well-conducted and meaningful study addressing an important problem; however, several weaknesses are outlined below: 1.Limited algorithmic novelty at the core model level: While the system is well designed, much of the performance gain appears to stem from integration and orchestration of existing specialized models, rather than the introduction of fundamentally new vision or learning methods. The novelty is therefore primarily at the system integration level, rather than a fundamental methodological advance, which should be more explicitly acknowledged and positioned. The framing around “agentic intelligence” somewhat overstates this novelty. 2.Insufficient ablation of the multi-agent design: The paper does not sufficiently disentangle the contributions of: the multi-agent coordination framework; the ensemble of models; or the deterministic fusion strategy. As a result, it is unclear how much of the observed improvement is attributable to the proposed framework versus the aggregation of strong existing components. A more detailed ablation would strengthen the central claim. 3.Limited quantitative evaluation of reporting components: While Table 1 provides strong quantitative evaluation of individual tasks, the key system-level contribution: image captioning and video summarization, is evaluated primarily qualitatively (Fig. 2). There is no formal assessment of report quality, clinical accuracy, or usability (e.g., reader studies or structured evaluation metrics). 4.Over-reliance on engineering aggregation: The performance improvements appear to stem largely from dynamic model selection, ensembling, and consistency constraints, which, while effective, are expected to improve performance when combining multiple strong models. 5.Lack of prospective or user-facing validation: Despite the strong clinical framing, the evaluation is entirely retrospective. There is no assessment of: usability, time savings, or impact on clinical decision-making. This limits the strength of claims regarding real-world applicability. 5.Positioning relative to prior work could be clearer: Although recent agentic and multimodal works are cited, the paper does not clearly distinguish itself from existing modular or pipeline-based systems that combine multiple models without LLM-based orchestration.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    Overall, this is a well-conducted and meaningful study addressing an important problem; however, several aspects could be further strengthened. Clarify the contribution and positioning: Clearly distinguish between the proposed system-level integration and existing agentic or modular pipelines, and moderate claims regarding methodological novelty. Add targeted clarification on the role of each component: Provide a clearer explanation (textual, not necessarily experimental) of how the coordinator, expert modules, and fusion strategy contribute to the overall performance. Expand discussion of evaluation scope: Explicitly acknowledge that report generation and video summarization are evaluated qualitatively, and discuss this limitation in the manuscript. Improve explanation of fusion and consistency mechanisms: Provide more detail on the deterministic fusion rules and cross-task consistency checks to enhance reproducibility and transparency. Clarify comparison to baselines: Better explain how the proposed system differs from standard ensembles or pipelines combining multiple models. Add discussion of limitations and failure cases: Include a short section discussing potential failure modes and generalization limits. If feasible, consider including additional ablation analysis to better isolate the contribution of the proposed framework, such as: Comparison between single-model and multi-agent performance Impact of the coordinator decisions Contribution of the structured fusion strategy

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper presents a well-executed and clinically meaningful system that integrates multiple components into a unified fetal ultrasound workflow, supported by strong empirical validation at the task level. While the contribution lies primarily in system integration rather than methodological novelty, and the evaluation of the reporting and video summarization components remains limited, overall it represents a solid and practically relevant contribution that fits well within MICCAI.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #3

  • Please describe the contribution of the paper

    This paper presents FetalAgents, a multi-agent framework for fetal ultrasound analysis that bridges specialized single-task models and clinical workflows. It introduces a coordinated system of Coordinator, Expert, and Summarizer agents, supporting standard-plane classification, brain sub-plane classification, segmentation, biometry, caption generation, and video summarization via keyframe identification and temporal aggregation. Across eight externally validated tasks, FetalAgents consistently outperforms strong standalone vision models and general/medical MLLMs, suggesting that agentic orchestration of specialized tools provides a more workflow-aligned solution.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1)The paper targets an important gap in fetal ultrasound AI: current methods are mostly task-specific, while real clinical workflows require coordinated analysis across multiple subtasks, temporal video streams, and reporting. This motivation is clear and relevant. 2)The Coordinator–Experts–Summarizer structure is easy to understand, and the use of structured expert outputs is a sensible design choice that improves traceability and reduces free-form hallucinations.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1)The core technical idea depends on existing models using an LLM-based coordinator and a summarizer. The novelty comes from combining existing ingredients into a workflow than from proposing a fundamentally new learning or reasoning mechanism. 2) It is unclear how much of the gain comes from multi-agent coordination itself versus simple routing, ensembling, and fusion of strong expert models. The paper lacks direct ablations to isolate this effect. 3) A major claim of the paper is that it supports end-to-end video stream summarization and structured clinical reporting. However, these components are evaluated almost exclusively through qualitative examples in Figure 2.There is no quantitative evaluation of keyframe selection quality, no report-level factuality/completeness assessment, and no user study. Given that video summarization is a headline contribution, the evidence supporting this component is currently limited. 4) FetalAgents is effectively a system-level ensemble, while many baselines are single models. This makes it hard to determine whether the advantage comes from the agent framework itself or simply from stronger combined modeling capacity.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    My main concern is the novelty and experimental attribution. The proposed method appears to derive much of its strength from coordinating and fusing multiple specialized models rather than from a clearly novel agentic reasoning mechanism. The paper does not sufficiently disentangle the contribution of the multi-agent framework from that of multi-model ensembling and deterministic routing/fusion. In addition, two of the headline contributions, image captioning and especially video summarizationj, are evaluated mainly qualitatively, without the level of quantitative or human evaluation that would be needed to fully substantiate those claims. The lack of systematic ablations and matched-capacity baselines makes it hard to determine how much of the gain is truly due to the agent framework. Overall, I think this is a promising systems paper with strong practical relevance, but in its current form it sits slightly below the MICCAI acceptance threshold for me because the methodological novelty is moderate and the evidence for the most distinctive workflow-level claims is not yet fully convincing. A strong rebuttal clarifying the novelty and providing more insight into the role of the agentic design may potentially improve assessment.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Author Feedback

N/A




Meta-Review

Meta-review #1

  • Your recommendation

    Provisional Accept

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    the authors should better disentangle the contribution of the multi-agent framework from that of standard ensembling, model selection, or deterministic routing. Even partial or qualitative ablation evidence, or a clear conceptual decomposition, would work here.

    the role and implementation of the coordinator should be described more precisely. It should be made clear whether the coordination mechanism is learned, rule-based, or hybrid, and how decisions are made when multiple processing paths are possible.

    please address the limited evaluation of the reporting and video summarization components. If additional experiments are not feasible, the limitations should be explicitly acknowledged and the claims moderated accordingly, with clearer discussion of how these components could be rigorously evaluated.

    the positioning of the work relative to existing modular or pipeline-based systems should be clarified, particularly to avoid overstating methodological novelty and to better highlight the practical contribution.



back to top