Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Point-of-care transthoracic echocardiography enables cardiac assessment in virtually any clinical setting, yet its diagnostic utility remains constrained by the expertise required for image acquisition and interpretation. Visual question answering (VQA) offers a promising paradigm for bridging this expertise gap through interactive clinical assistance, but existing echocardiography VQA datasets are limited in scale, restricted to high-quality images, and only cover a few views. We introduce EchoVQA, the first large-scale VQA dataset for echocardiography, comprising 14,299 images and 74,819 question-answer pairs. The dataset integrates public sources (EchoNet-Dynamic, CAMUS) with our own point-of-care acquisitions from two handheld probes (Lumify, Clarius), spanning diverse views and including both high-quality and suboptimal images. Uniquely, EchoVQA includes acquisition guidance questions to help users optimize transducer positioning toward a diagnostic apical 4-chamber view for left ventricular ejection fraction estimation—a challenging task for novice operators in point-of-care settings. We further develop a parameter-efficient method based on multimodal learnable prompts achieving state-of-the-art performance on most benchmarks, including EchoVQA, with significantly fewer trainable parameters than existing state-of-the-art approaches. Data and code: https://huggingface.co/datasets/filbel/echoVQA.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/6097_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/fil-mp/echovqa

Link to the Dataset(s)

https://huggingface.co/datasets/filbel/echoVQA

BibTex

@InProceedings{BelFil_EchoVQA_MICCAI2026,
        author = { Bellos, Filippos AND Li, Yutong AND Dong, Jessie N. AND Guo, Zaiyang AND Mackay, Emily AND Li, Yayuan AND Avrithis, Yannis AND Pouch, Alison AND Corso, Jason J.},
        title = { { EchoVQA: Enabling Conversational Assistance for Point-of-Care Cardiac Ultrasound } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16878},
        month = {September},
        page = {pending}
}

Reviews

Review #1

  • Please describe the contribution of the paper
    1. The paper introduces EchoVQA, a large-scale cardiac ultrasound VQA benchmark with 14,299 images and 74,819 QA pairs, combining public datasets (EchoNet-Dynamic, CAMUS) and newly collected point-of-care data. It features fine-grained stratification by view and quality, and uniquely includes acquisition-guidance questions.
    2. It proposes a parameter-efficient three-level prompting framework on top of LLaMA-2-7B, integrating visual prompts, dataset-level textual prompts, and retrieval-based instance prompts. With only 104M trainable parameters, it achieves performance comparable to LLaVA-Med and LLaVA-Tri.
  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The dataset fills a critical gap, expanding prior echocardiography VQA resources from only hundreds of QA pairs to a large-scale benchmark with real point-of-care data, significantly enhancing clinical relevance.
    2. The inclusion of acquisition-guidance questions is novel, extending VQA from image interpretation to assisting probe operation, directly addressing real-world needs of novice sonographers.
    3. The human-in-the-loop data generation pipeline is well-designed, combining expert-defined templates, LLM-based generation, and multi-expert verification to reduce hallucinations and improve reliability.
    4. The method is parameter-efficient, achieving near-SOTA performance with only 104M trainable parameters compared to much larger models like LLaVA-Med and LLaVA-Tri.
    5. The three-level prompting design is well-validated, with ablations showing complementary contributions from each component rather than redundancy.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    1. The point-of-care data are limited in scale and diversity (≤12 subjects), which weakens claims of generalizability and clinical robustness.
    2. The clinical validity of acquisition-guidance QA is not verified, lacking inter-rater agreement or user studies to confirm real-world usability.
    3. The expert review process is unclear given the dataset size, with no details on full vs. partial validation or quality control statistics.
    4. The methodological contribution is largely compositional, combining existing techniques, and the gains from test-time optimization are marginal relative to its computational cost.
    5. The evaluation lacks domain-specific echocardiography baselines and shows limited generalization beyond radiology-heavy settings.
    6. Potential dataset bias and incomplete ethical disclosure, along with clarity issues in figures and presentation, further limit reliability and transparency.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The dataset clearly fills an important gap (PoC echocardiography VQA with acquisition guidance), and the method provides a reasonable PEFT-based approach with solid SOTA-level results. However, concerns remain: (1) the PoC data involve only 12 subjects, limiting diversity and weakening claims of large-scale generalization; (2) the clinical validity of acquisition-guidance QA is not supported by user studies; and (3) the methodological novelty is largely a cross-domain combination of existing techniques. Clarifying data release plans, providing subject diversity statistics, and reporting the latency of TTO would strengthen the work.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The rebuttal confirms a data release commitment and provides subject demographics, but the two most critical concerns — clinical validity of acquisition-guidance QA and TTO computational cost — remain unsupported by evidence, and the 12-subject PoC cohort limitation is not resolved.



Review #2

  • Please describe the contribution of the paper

    This paper introduces a new echocardiography benchmark alongside a parameter-efficient fine-tuning (PEFT) strategy. The proposed benchmark consolidates existing public datasets with self-collected point-of-care ultrasound (POCUS) images to construct a comprehensive set of echocardiographic question-answering pairs. To improve model performance, the PEFT strategy incorporates three complementary components: visual adaptation prompts, dataset-level prompts, and test-time optimization.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The problem this benchmark aims to address is practically motivated. Despite its clinical importance, visual question answering for POCUS remains a largely underexplored area.

    2.The question types curated in the benchmark are closely aligned with real-world POCUS interpretation tasks, reflecting strong potential for clinical applicability.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.Some additional clarification on the benchmark construction would help readers better understand the design choices. For instance, it would be helpful to explain the rationale behind combining conventional echocardiography from public datasets (EchoNet-Dynamic and CAMUS) with self-collected POCUS images, as well as the key similarities and differences between these data sources. Additionally, since echocardiography is inherently temporal, it would be worth discussing the motivation for using images rather than video clips as input. Finally, as echocardiographic interpretation typically involves multiple views (e.g., A4C, A2C, PLAX, PSAX), clarifying why the benchmark focuses on A4C images — particularly given that existing benchmarks such as MIMIC-Echo-VQA already cover a broader range of views and conditions — would strengthen the paper’s positioning.

    2.Further elaboration on the benchmark construction pipeline would help establish its soundness. Specifically, clarifying the relationships among “expert templates,” “category labels,” and “questions” would be beneficial. It would also be helpful to describe how category labels are assigned to each image, how expert templates and category labels interact in the QA generation process, and to provide examples of the resulting QA pairs at each stage.

    3.The use of GPT-4o for fine-grained QA generation raises a few points worth addressing. First, given that GPT-4o’s availability may change over time, it would be helpful to discuss contingency plans for reproducing or extending the benchmark when the same model is unavailable. Second, providing some validation or evidence of GPT-4o’s echocardiographic analysis capability would strengthen confidence in the quality of the generated ground-truth annotations, as this is a specialized domain that may benefit from additional scrutiny.

    4.The connection between the proposed PEFT strategy and the Echo-VQA benchmark could be made more explicit. The introduction motivates the benchmark by highlighting the scarcity of echocardiography-specific data, yet the PEFT experiments rely on general-purpose medical VQA datasets for training rather than splits derived from Echo-VQA itself. It would be helpful to clarify two related points: first, how the constructed Echo-VQA dataset contributes to MLLM performance improvement, and second, since echocardiographic content appears to constitute a relatively small fraction of the general-purpose training data, what accounts for the observed performance gains on Echo-VQA specifically.

    5.The experimental comparisons could be further strengthened by including more recent medical MLLMs, such as MedGemma, Lingshu, and their LoRA fine-tuned variants, to provide a more up-to-date performance reference. In addition, reporting FLOPs for the compared methods would offer a clearer picture of computational costs, since these reflect the parameters actually involved in the optimization process and are important for fair comparison.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    The paper mentioned that they self-collected echocardiography data for constructing the benchmark. This may need ethical approval for data curation.

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    See weakness above.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (2) Reject — should be rejected, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    A strong benchmark paper should construct a rigorous and well-motivated benchmark that can meaningfully stress-test current methods from important yet underexplored perspectives, while also offering insights into existing limitations and potential future directions. Given that the EchoVQA dataset is the central contribution of this work, meeting these expectations is essential. While the paper addresses a practically relevant problem, the curation process and benchmark constitution would benefit from further clarification and refinement to better establish their soundness. Additionally, incorporating more recent methods and deeper analysis that more fully leverages the proposed benchmark would considerably strengthen the experimental section.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Reject

  • [Post rebuttal] Please justify your final decision from above.

    I thank the authors for their efforts. However, several issues remain unresolved. The concern regarding why conventional echocardiography and self-collected point-of-care data are combined, despite the paper’s title explicitly stating “point-of-care cardiac ultrasound,” has not been adequately addressed. Additionally, the concerns surrounding the benchmarking comparisons and implementation details of the proposed PEFT method remain outstanding. Furthermore, the EchoVQA training and validation data configuration and statistics used for stage 2 fine-tuning are not sufficiently explained, particularly given that the benchmark and training data are split from the same source. In light of these unresolved concerns, I have decided to maintain my original score.



Review #3

  • Please describe the contribution of the paper

    The paper presents a twofold contribution. Firstly, it introduces EchoVQA, a dataset of images and VQA pairs combining public and new images collected by the team. All the information was validated by medical experts. Secondly, authors present a method to address the VQA problem which combines simple yet effective techniques such as parameter optimisation, visual adaptation prompts, instance level prompts and fine-tuning using a medical vocabulary. Results show that the method can consistently perform both on benchmarks and in the newlye presented dataset.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper is easy to follow and impacts the reader from the abstract. All justifications are properly made, and it is clear that authors have thoroughly investigated this field. The work integrates effectively public and private datasets, which resonates to the community that has worked with the former. Authors also clearly understand which are the pains in the medical sector (i.e. sonographers/clinicians not being available at certain times) and all the data acquisition and labelling pipeline has been validated by experts. Every method presented in section 1 is there for a reason, as they have compared and benchmarked against all of them. The sunburst diagram presented in Fig1 is noteworthy, and it is a great example of good practices in visualisation.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The weaknesses I found in the paper are minor, yet I think it is important for me to point them out. In Fig 1, authors could have simply used “im” for images and that could have helped them arrange better the text across the circle. The depiction of step 2 in Fig 2 is a bit confusing; I would have liked to see more detail on how the GPT agent pipelines these steps. Also, I would have liked to see some percentages on how much data is needed from step 1 to help the agent complete the task in 2.In Fig 3 step b, you should have depicted how the question looks like after going through the tokenizer coming into this part of the pipeline. In Table 1, it is unclear why the baseline GPT-4V doesn’t have parameters (I assume because it is a fully closed model) and why it cannot be used in PathVQA and your EchoVQA. References could be standardised better (there are some references that have multiple links and on some you don’t show the conference acronym) and I would have liked to see more references to work previously presented in MICCAI (you only have one workshop paper) to show that you are continuing a trend that interests the people that regularly publish in this venue.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper is very interesting and relevant, and presents a major contribution in the field. The reproducibility aspect is a bit of a concern as authors don’t specify if both the data and the method will be released, but the paper itself is very explanatory and will motivate insightful discussions in this field.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    I stand by my previous decision and I think the paper is interesting and publishable.



Author Feedback

We thank the reviewers for their constructive feedback, and R1/R3 for recognizing EchoVQA as addressing a critical gap. We clarify the dataset pipeline, benchmark scope, and evaluation protocol below.

Dataset scope, combining sources, image vs. video, A4C (R1#1, R2#1). EchoVQA spans 14,299 images and 74,819 QA pairs across EchoNet-Dynamic, CAMUS, Lumify, and Clarius, stratified into view-quality categories. The data sources are complementary: the 12-subject PoC subset (7F/5M, age range 23.52–40.57, 3 White / 9 Asian) supplies deployment-like acquisition variation and guidance signal across the non-diagnostic-to-diagnostic spectrum, largely absent from public data, while the public partition adds demographic/pathology breadth from hospital settings. EchoVQA is image-level rather than video-level because acquisition guidance is real-time and frame-specific. Per-image QA is more fine-grained: a clip may begin with LV cutoff and end off-axis, requiring different guidance (“slide medially” vs. “rotate counterclockwise”), while a single video-level instruction would be ambiguous. Image-level guidance therefore provides richer supervision and composes with downstream video LVEF models that assume a diagnostic A4C clip. A4C is the canonical PoC view for LVEF — the most clinically actionable PoC measurement — and one of the hardest for novices to acquire [1,16]; multi-view extension is future work.

Expert coverage, IRB, validity (R1#2, R1#3, R1#6). All 74,819 QA pairs were reviewed by 4 experts (Sec. 2.3); given scale, we prioritized full-coverage correctness review over inter-rater agreement on a subset. Guidance was authored or revised by experts following standard acquisition practice [2,16]. A novice-operator user study on guidance efficacy is conducted as separate system-level validation, but was outside this submission. PoC scans collected under IRB (Sec. 2.1). Data/code will be released upon acceptance (R1, R3).

QA pipeline (R2#2-#3,R3). Per Sec. 2.2/Fig. 2: experts (i) categorize each image and (ii) author category-specific template QAs, e.g., for A4C_High_Qual: “Can I use this for EF measurement? Yes”, and for A4C_Low_Qual: “Can I estimate EF? No, the view quality is insufficient.”. GPT-4o turns these into image-specific QA capturing within-category variability (LV cutoff, foreshortening, low contrast). Conditioned on category label, templates, and image (not assigning categories) GPT-4o generates pairs like “Q:Can EF be measured? A:No, the LV apex is cut off,” which experts validate for image-grounded correctness.For replication,GPT-4o is not essential;stronger successor MLLMs could reduce expert corrections.

Method, baselines, compute (R2#4, R2#5, R1#4–5). Stage 1 (Sec. 3.2) aligns vision–LLM on medical image-caption corpora; Stage 2 fine-tunes per VQA dataset, including EchoVQA splits. Thus EchoVQA gains come from EchoVQA fine-tuning plus EchoVQA-adapted text prompts and TTO (R2#4). Comparisons follow prior per-dataset fine-tuning and strict exact-match/recall metrics (LLaVA-Med, LLaVA-Tri [ICLR 2025], PeFoMed). On echo-specific baselines (R1#5): we agree this would further strengthen the evaluation. Existing echo-specific models such as EchoCLIP and EchoPrime are primarily contrastive/retrieval-style, making direct comparison non-trivial. Still, including them as complementary baselines would indeed have been beneficial for completeness. MedGemma and Lingshu (2025) fold benchmark training splits into generalist multi-stage tuning, so they are not apples-to-apples. Even under Lingshu’s MedEvalKit (LLM-as-judge, looser than our recall), our method exceeds Lingshu-7B and MedGemma-4B-IT on aggregate accuracy (Lingshu Table 6). We report trainable parameters as the optimization-cost proxy and will clarify FLOPs/latency if permitted;TTO adds bounded per-query overhead scaling with vocabulary. We claim not per-component novelty but the medical-VQA-specific combination: visual adaptation prompts, text prompt learning, and TTO.



Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    Reviewers ask for a clearer justification of why conventional echocardiography and self-collected point-of-care data are combined, why the benchmark is image-based rather than video-based, and why the focus is limited to A4C views. They also ask for a much clearer explanation of the QA generation pipeline, including the relationship between expert templates, category labels, GPT-based generation, and expert validation.

    The number of subjects appears small, and there is not enough evidence yet that the acquisition-guidance questions are clinically valid or useful in practice. Reviewers specifically ask for more detail on subject diversity, expert review coverage, and whether there is any inter-rater agreement or user-level validation.

    The method contribution seems largely compositional, and about whether the experiments fully leverage the proposed benchmark. Stronger positioning against more recent medical multimodal large language models, together with clearer reporting of computational cost and the role of test-time optimization, would strengthen the paper.

    These are central enough questions that rebuttal is appropriate before a final decision, and the paper could still move either way depending on how convincingly they are addressed.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    Reviewers agree that the clinical problem is important and that the benchmark addresses a meaningful gap in echocardiography-focused VQA. The inclusion of acquisition-guidance questions extends VQA beyond image interpretation toward practical operator assistance. The benchmark itself is the primary contribution of the work, whereas the methodological contribution is generally viewed as incremental and largely compositional.

    The main concerns relate to the limited size of the POCUS cohort, the absence of user studies or inter-rater agreement analyses for acquisition-guidance questions, and insufficient initial clarity regarding benchmark construction, QA generation, and the rationale for several design choices. The rebuttal provided reasonable clarification regarding dataset composition, expert validation, IRB approval, benchmark scope, and the QA generation pipeline, addressing many of these concerns.

    While limitations remain regarding dataset diversity, clinical validation, and comparisons with more recent medical multimodal models, these do not outweigh the value of the benchmark contribution.

    For the camera ready:

    • Clarify benchmark construction and QA generation workflow in greater detail.
    • Improve discussion of limitations arising from the small POCUS cohort.
    • Clearly describe expert review procedures and quality assurance measures.
    • Expand discussion of image-level versus video-level design choices and A4C-only scope.
    • Improve positioning relative to recent medical multimodal large language models and echocardiography-specific methods.
    • Report computational costs and inference overhead associated with test-time optimization. VERY IMPORTANT: All images, QA pairs, expert templates and QA generation prompts must be released under a permissive license and accessible without barriers, e.g. via huggingface or zenodo. Without this the paper would need to be rejected.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The paper introduces EchoVQA, a clinically motivated benchmark for conversational assistance in point-of-care cardiac ultrasound, together with a parameter-efficient VQA framework. Reviewers found the benchmark timely and clinically relevant, particularly the acquisition-guidance component. Concerns focused on cohort scale, benchmark construction details, and limited evidence for clinical validity and efficiency. The rebuttal clarified the dataset design, QA generation pipeline, and evaluation protocol. While some limitations remain, the overall benchmark contribution and experimental evidence place the paper above the acceptance bar.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    I recommend acceptance. Two of the three reviewers support acceptance after the rebuttal. The main contribution, the EchoVQA benchmark with acquisition-guidance questions for point-of-care echocardiography, fills a genuine and underserved gap, and the rebuttal clarified the points the Meta-AC raised. The remaining concerns, including the small 12-subject PoC cohort, the lack of a user study validating acquisition-guidance usefulness, the combination of conventional and PoC data, and the unreported TTO compute cost, are real but reflect requests for additional validation rather than evidence that the results are invalid. I therefore recommend acceptance, with the suggestion that the camera-ready temper generalization claims given the small PoC cohort, clarify the EchoVQA train and test split configuration, and frame the novice-operator user study and FLOPs/latency reporting as necessary follow-up work.



back to top