List of Papers Browse by Subject Areas Author List
Abstract
Medical imaging research is increasingly shifting from controlled benchmark evaluation toward real-world clinical deployment. In such settings, applying analytical methods extends beyond model design and requires dataset-aware workflow configuration and provenance tracking. Two requirements therefore become central: adaptability, the ability to configure workflows according to dataset-specific conditions and evolving analytical goals; and reproducibility, the guarantee that all transformations and decisions are explicitly recorded and re-executable. Here, we present an artifact-based agent framework that introduces a semantic layer to augment medical image processing. The framework formalizes intermediate and final outputs through an artifact contract, enabling structured interrogation of workflow state and goal-conditioned assembly of configurations from a modular rule library. Execution is delegated to a workflow executor to preserve deterministic computational graph construction and provenance tracking, while the agent operates locally to comply with most privacy constraints. We evaluate the framework on real-world clinical CT and MRI cohorts, demonstrating adaptive configuration synthesis, deterministic reproducibility across repeated executions, and artifact-grounded semantic querying. These results show that adaptive workflow configuration can be achieved without compromising reproducibility in heterogeneous clinical environments. Our code is available at https://github.com/MASILab/medimg-agent
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/2038_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
https://github.com/MASILab/medimg-agent
Link to the Dataset(s)
NLST dataset: https://www.cancerimagingarchive.net/collection/nlst/
BibTex
@InProceedings{ZuoLia_An_MICCAI2026,
author = { Zuo, Lianrui AND Liu, Yihao AND Rudravaram, Gaurav AND Ramadass, Karthik AND Krishnan, Aravind R. AND Phillips, Michael D. AND Bodien, Yelena G. AND Patel, Mayur B. AND Trujillo, Paula AND Forero Martinez, Yency AND Deppen, Stephen A. AND Grogan, Eric L. AND Maldonado, Fabien AND McGann, Kevin AND Holmes, Hudson M. AND Cutting, Laurie E. AND Huo, Yuankai AND Landman, Bennett A.},
title = { { An Artifact-Based Agent Framework for Adaptive and Reproducible Medical Image Processing } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16878},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The proposed framework of this paper provides an agent-based architecture for integrating a semantic planning layer on top of a deterministic medical image processing workflow engine called Snakemake. In order to achieve reproducible executions of workflows and allowing for new datasets to be processed, this framework is based on the “artifact contract” concept: outputs from workflows are produced as type-based tuples (i. e. , type + attributes + provenance), which the long-lived, multipurpose agent interprets semantically via formalized artifact contracts to reason over artifact contract-compliant files, while simultaneously recording provenance for auditing purposes. By taking advantage of the structure of the state of the agent, the agent can complete two types of tasks: (1) generate configuration C = (π, θ) as a means of selecting rules from a modular rule library based on set of artifacts and desired analytical outcome at time of execution of workflow, and (2) respond to natural-language questions about contract-compliant artifacts (i. e. , structured output files produced by executing workflows) while allowing workflows to execute deterministically. The framework has been validated for use in three clinical cohorts (NLST, LungCaTrial, and BrainICU) and across 9 different workflows covering distinct analytical goals (curation, lung lobe/nodule segmentation, cancer risk estimation, harmonization of MRI images, and brain segmentation), and through the use of an ablation where access to the artifact contract was removed, the contribution of the structured basis provided by the artifact contract to the agent’s functioning was documented.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
S1.The authors aim at addressing a real issue related to the clinical translation of imaging techniques; in particular the authors have conducted research and implemented pipelines on actual PACS derived data and know that there is a large difference between the benchmarks of controlled environments and the clinical dataset, which include format differences, nested archives, planning/localizer images, inconsistent naming and missing metadata. The majority of work presented at MICCAI focuses primarily on the architecture of the model; however, this paper is focused on the infrastructure level, where most of the practical failures actually occur. S2.The decision to separate the semantic design (LLM-based, probabilistic) of a system from its deterministic execution (workflow engine) is an intentional and principled approach to achieving both a degree of flexibility as well as scientific repeatability in the final application of the system. The artifact contract is the interface between the two levels of the system architecture and is a very clever method of abstraction, since it is (a) representing the state of the artifact that can be queried, (b) gathering provenance data about the artifact that can be used for auditing purposes and (c) constraining the LLM reasoning to only the structured fields of data associated with the artifact rather than the raw filenames from which the artifact was created. This will help to alleviate a true risk with the use of LLM-driven pipeline tools. S3.The decision to run fully on-premise using Ollama with a DeepSeek-R1 14B system presents a significant engineering commitment that illustrates recognition of the impact of regulatory restrictions (HIPAA/GDPR/institutional agreements) applicable to clinical imaging research/data. This is crucial to be able to use the results derived using the constructed system in other clinical areas and differentiates the work from cloud-dependent agentic systems. S4.The ablation that removed access to the artifact registry (Section 3, Table 2) provides a direct quantitative measure of the value of the artifact contract. By demonstrating that filtering query accuracy drops from 90-95% down to 10-15% when the artifact contract is removed, the authors prove the observed accuracy is a function of structured grounding rather than solely LLM capabilities. The ablation design allows for the isolation of the contribution of structured grounding when producing the observed accuracy levels. S5.The three cohorts include research CT (NLST), clinical CT (LungCaTrial) with significant heterogeneity, and research MRI (BrainICU) with moderate heterogeneity. There are 9 different workflows, covering distinct analytical goals such as data curation, segmentation, risk estimation, and harmonization. This wide range of goals supports the framework nature of the paper, while also creating an opportunity for authors to fall into a common pitfall of evaluating only one curated dataset. S6.The authors have committed to making the code and report available using Snakemake for deterministic execution, which is also a well-established and auditable tool in bioinformatics. There is 100% DAG equivalence across repeated executions under all circumstances.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
W1.Only one existing workflow adaptation tool was compared in the study. Reference works such as Snakemake and NextFlow were mentioned; however, there were no time comparisons or calibration of performance against any of these systems (including your domain expert’s manual workflow assembly) or a “proof of correct operation” using the comparison with any of these systems or as a baseline. Findings from developing correct workflows indicate that the authors should have compared their own algorithm and tooling against a small but viable source of aggregated evidence including i) traditional manual scripting in Snakemake, ii) a conventional LLM, not using an artifact contract, and not using raw file listings, and iii) a basis for comparing against a BIDS formatted Pipeline Manager, would have provided substantial evidence to establish that the Artifact Contract architecture provides the greatest benefit to the user or workflows. W2.A small sample of semantic query brands for testing the audience of the model for providing statistics. Evaluating only 20 questions per category across these datasets provides a small sample size, and lacking a description of an independent generation protocol raises concerns of confirmation bias. The authors need to establish the method used for creating the question set to show their independence and defend against confirmation bias, as the system to establish the benchmarks and created the questions were written by the authors, thus leading to the possibility of the authors subconsciously designing the questions to enable the system to correctly answer all questions, thereby providing support for the authors of the system. At a minimum, the authors of the paper must include a sufficient description of the question generation protocol to establish its independence from bias and ideally include questions authored by an independent party (a collaborating clinician or radiologist). W3.Insufficient timing, latency, and cost analyses will hinder any framework paper that possesses dynamism being able to count any definite human labor saved; without knowing how long it takes to build a workflow in the framework compared with an expert manually, what the agent latency (LLM inference time) will be, and how many tokens are required for each query, we cannot determine whether the framework provides any real increase in operational efficiency or simply a better-structured abstraction of an essentially identical performance. W4.The true ground for determining adaptability metrics consists of a single domain expert’s opinion. Each of the metrics used (Initial Rule Match (IRM), Plan Iteration (PL), and Final Output (FO)) will be made against an “expert provided ground truth. “ The professional profile, years of experience, and institutional independence of the expert are not delineated, nor is there a measure of inter-rater reliability. Multiple valid ways exist to build a medical imaging workflow (i. e. , several valid strategies will exist to enclose intensity normalization, all may work in their respective applications). Since a single-domain-expert ground truth may lead to confusion, two or more experts’ agreement would greatly increase confidence in those metrics. W5.The scalability of the rule library within this evaluation is never stated. The total number of rules within each of the analytical goals as shown in figure A/C; each will contain 11 – 20 rules for this project; there will likely be a few dozen overall; an actual clinical workflow may run in the hundreds of rules across many modalities, and retrieval/planning over a large rule library is known to degrade agent performance due to context window constraints and selection ambiguity. The paper does not discuss how the framework scales, whether the rule library would be modularized or hierarchically indexed, or how performance degrades as the rule count grows. W6.The assertion that the method is reproducible is factually correct, but somewhat exaggerated. 100% Database Access (DAG) equivalence across multiple executions is anticipated when Snakemake is given control of the execution and replicability. The fact that an agent can repeatedly create the same configuration C when the same inputs are supplied is the true novel contribution of this procedure; the agent achieves this by being programmed with a defined Non-Fuzzy Metric (NFM) and defined inputs. Therefore, the LLM’s output is based on its sampling temperature and prompt consistency and is not based on Snakemake. To clarify the above statement, it is worth noting that the DAG equivalence of results is dependent on the probabilistic planning behavior, in which all planning is predetermined. The authors do not specify the decoding temperature for DeepSeek-R1 (e. g. , T=0.0 for greedy decoding), leaving the source of the agent’s deterministic output generation ambiguous. W7.Limited variations in LLM. The document only contains references to two different local LLMs, both 14 billion LLMs: DeepSeek-R1 and Qwen. If smaller local LLMs (7 billion) or larger local LLMs (70 billion, proprietary versions of GPT-4/Claude) were to undergo testing, or if an acknowledgment of the trade-off between localized privacy and capability was indicated, this would be beneficial for clinical facilities in the selection of their infrastructure. W8.The density of the information contained in Figure 2 (system overview) is difficult to read, as it contains a complete user layer, agent layer, execution layer, rule library, memory type, skill set, and an example configuration type all within the same diagram, therefore, at the time of first review, the reader is unable to systematically read the diagram. The diagram could easily be split into two (one depicting the ‘architectural depiction’ and the other depicting the ‘workflow action type’) to provide a clearer understanding. Similarly, the example dialog in Table 1 uses “[omit to save space]” several times, which reduces the transparency of the interaction loop. W9.There is a lack of discussion of agent-side failure modes other than missing metadata. Section 3 (Adaptability) discusses that most failures result from missing or flawed DICOM metadata. This is useful knowledge, but the submission also should discuss agent-side failure modes such as hallucinating nonsensical rules, selecting conflicting rules, or failing to terminate planning. All of these failure modes can easily occur in a real-world deployment of the framework; therefore, mechanisms to address how the proposed generally-constrained design can manage them would be worthwhile to include. W10.The submission does not propose any methodological innovation for image analysis. It’s a framework/systems paper, not a new algorithm/model for a specific medical imaging application. Therefore, this may present some discomfort to reviewers in an MIC-oriented venue; however, this is actually a positive in terms of the Clinical Translation track. Authors should make it clear in the rebuttal that the intention of this submission is to be part of the Clinical Translation track, not a pure methodology track.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
Thank you for your submission. I appreciate that you are tackling a very relevant, underrepresented area at MICCAI. The artifact contract is also a neat representation of an accessible abstraction, as is your separation of semantic planning from deterministic execution. Here are some of my suggestions for enhancing your work: 1.In the ablation studies, provide a pure LLM agent (no artifact contract) as a baseline comparison to Tables 1 and 2, ideally alongside a manual Snakemake pipeline developed by a domain expert. Right now, your ablation studies in Table 2 are a good start, but represent against a degraded/less capable version of the tested system rather than against established workflows. 2.Provide some quantification (in hours, for example) as to how much time/effort clinicians will save from using your system. This would give credibility to translating your work into the clinical setting, perhaps through the use of a plot of assembly time versus workflow complexity or at an average assembly time per workflow. 3.Please disclose how the authors of the semantic query benchmark questions created them, and if possible, engage a clinician who will not be a co-author to draft some questions as well. 4.Clarify your statement about reproducibility: DAG equivalence is a feature of Snakemake workflows, not of your agent. Your contribution should therefore not be described as reproducible but rather as producing configuration stability across LLM sampling (which is important). Did you use temperature=0? 5.Talk about how the rule library can grow; discuss ways that the agent might produce false (“hallucinated”) rules or fail to stop running conflicts when choosing which rule to run. 6.Look at doing something like dividing Figure 2 into separate figures so that it is easier to view. Also, make changes to the example shown in Table 1’s conversation to shorten it up a bit. 7.When presenting the rebuttal position to the clinical translation track, be sure to point out that this paper is providing an infrastructure, as opposed to algorithms. Overall, the paper has a lot of well-thought-out engineering and provides a validation of a real clinical translation issue. The above comments would enhance the paper’s overall contribution significantly.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I suggest Weak Accept (4). The manuscript presents an essential problem in clinical translation that is not as relevant as it could be at MICCAI, outlines an architectural solution (artifact contract + constrained agent + deterministic executor) that is based on principles, and validates that solution on three real clinical cohorts with a meaningful ablation. The artifact contract abstraction is a true contribution that goes beyond the “LLM + tool use” framing; placing a constraint on the agent to reason through structured artifact fields that contain explicit provenance is a fundamental solution to solving the critical grounding and reproducibility challenges that have made the use of LLMs for clinical pipeline tools limited. The ablation contained evidence of a 70% accuracy improvement, indicating that the artifact contract is actually producing useful results rather than just wrapping the results of the LLM. I have several greater evaluative concerns than conceptual for this submission. First, the baselines utilized are far too limited to a severely degraded self-baseline. Second, the semantic query benchmark appears not only small, but has a strong possibility of being constructed with author assistance from an as-yet-unreported author source, thereby creating substantial issues with its reliability. Third, the claim surrounding reproducibility conflates Snakemake’s (Container) deterministic outcomes with the agent’s own outcome stability, thus exhibiting a reduction in overall quality when making general conclusions about agent reproducibility. Finally, the scalability of the rule library has not been addressed. The overall strength of the authors’ claim of the artifact contract serving as the enabler to the proposed solution would be greatly enhanced with direct comparisons to either manual pipeline assembly or an unconstrained LLM agent. Additionally, I believe measurements of (and possibly statistics for) how long each of the tasks took, or how much effort was required to complete the tasks, would be imperative for any paper framed around adaptability and clinical translation. All things considered, I think the paper has a strong enough contribution to the field, and the problem it addresses is of sufficient importance that I would recommend accepting it if the authors can adequately respond to the evaluation deficiencies in their rebuttal. I would not be against rejecting the paper, since there are very real issues with the evaluation; I would rather see it accepted than rejected, because the quality of clinical translation infrastructure papers submitted to MICCAI is not too high.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #2
- Please describe the contribution of the paper
The paper addresses the important question of translating research imaging workflows to clinical practice. It proposes an agentic framework providing (1) adaptability to heterogeneous clinical environment, and (2) reproducibility through an integration with snakemake and test-retest experiments.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The use of AI agents to address “technical” clinical heterogeneity (e.g., heterogeneous data organizations) is very thoughtful
- Thorough problem formulation, unusual for a software tool contribution.
- Open-source, local execution stack including Ollama and snakemake.
- Broad evaluation including reproducibility, adaptability, semantic query and ablation.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- Reproducibility experiments do not test DAG stability across multiple LLMs or LLM versions (e.g., DeepSeek versions)
- Limited discussion of related work.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed framework is very thoughtful, well described and evaluated overall. My main comment relates to the lack of depth of the reproducibility evaluation, as in my experience results tend to vary substantially across LLMs and LLM versions. Measuring the variability of the generated DAGs across LLMs and their versions would be very useful. In addition, the paper lacks a discussion of the related work on existing systems to deal with clinical heterogeneity.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
The authors propose a novel, artifact-contract-based agent framework designed to enhance the adaptability and reproducibility of neuroimaging workflows. The core innovation is the introduction of a semantic layer atop deterministic execution (via Snakemake). This layer utilizes an artifact contract to encode intermediate and final workflow states into queryable records with explicit provenance. The system features a multi-agent architecture for workflow assembly, goal planning, and semantic querying, while maintaining a human-in-the-loop approval process for generated Directed Acyclic Graphs (DAGs). Notably, the framework is designed for local deployment to satisfy clinical privacy constraints.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Novelty Use of Artifacts: Unlike many workflow managers that focus solely on the final output, this framework prioritizes the paper trail of intermediate results, making the entire processing pipeline transparent and queryable.
- It is interesting how it keeps the agents from unfounded generation
- The authors use ablation studies to highlight how it improves performance
- Formal Rigor: The authors provide a clear mathematical formulation of the framework, which adds theoretical weight to the proposed architecture.
- Adaptable Framework: A core strength of the framework is its ability to create workflow configurations dynamically based on specific analytical objectives and by using a modular rule library in response to user commands, heterogeneous datasets without manual reconfiguration.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
- Architectural Ambiguity: While the high-level theory is sound, the specific interactions between the workflow components (ex: the Knowledge, Semantic Querying, Workflow Planning Agent, Workflow Assembling Agent, and Snakemake) are underspecified.
- Snakemake appears to be the foundation of the framework, but how it functions or the reason for its selection is not specified.
- The paper lacks discussion of relevant neuroimaging workflow tools such as Nipoppy, Neurobagel or the BIDS app ecosystem, and provides no justification for the selection of Snakemake over other executors (e.g., Nextflow).
- Very vague language is used. In Section 2.2 Adaptation via Workflow Assembly, it is unclear what the “planning component” is or how the “inspection skill” evaluates new datasets or how a feasible goal is established.
- Is DeepSeek used for all the mentioned agents? Was there fine-tuning/training involved? This isn’t specified and little description is given for agents other than the Semantic Querying Agent.
- Deterministic Paradox: There is a fundamental tension between the deterministic claim and the use of LLMs (DeepSeek/Qwen) for semantic querying and planning. The authors do not sufficiently address how the LLM is configured to control non-determinism or whether LLM hallucinations or non-deterministic sampling could compromise the contract.
- While the presence of artifacts and rules are supposed to ground the LLMs in truth, it is not clear how that prevents hallucinations or a loss of reproducibility with introduction of subtle variations possibly from different computational environments
- Incomplete Implementation Details: Crucial information regarding computational requirements is missing and the handling of competing tools (e.g., different brain segmentation algorithms/pipelines) is missing. It is unclear how a user defines the underlying toolsets within the configuration.
- In brain MRI processing, multiple tools can perform the same task. It is unclear whether the framework restricts or supports specific tools, and at what level users are expected to specify them. It is also not specified whether the system relies on pre-written Snakemake wrappers or generates them dynamically. These details are not addressed in the paper.
- Due to the framework’s use of several LLMs and GPUs for the experiments, it would be helpful to specify what computational resources a potential user might need (GPU, memory, etc.)
- Nitpick: DeepSeek has been banned or is heavily regulated by some institutions. If the user isn’t legally allowed to use this model at work, is there a workaround?
- Minor comment: In the experiments, 2 out of the 3 datasets are not cited, so it is unclear if it is possible to obtain them in order to replicate the paper. Moreover, besides describing them as moderate/high heterogeneity, it is unclear of their composition or file layout.
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
- With the emphasis on reproducibility in the introduction, why was the reproducibility experiment limited to only examining the DAGs for 2 repetitions?
- The authors should clarify what constitutes “representative questions” for the querying agent and whether more rigid query structures (e.g., SQL-style) were considered to mitigate LLM miscommunication.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(2) Reject — should be rejected, independent of rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper introduces an interesting concept in the semantic layer for medical imaging provenance, addressing the urgent need for adaptable frameworks. The artifact contract is also a strong conceptual contribution. However, key components (the planning logic, tool-set integration, framework component interaction, and LLM non-determinism) are not described with enough detail to justify an “Accept” or to fully clarify how the proposed framework functions or why certain design choices were made. The limited experimental reproducibility repetitions (n=2) and the lack of clarity on dataset availability further weigh down the score.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Author Feedback
We sincerely thank all three reviewers for their comments and the AC’s recommendation of early acceptance. We are especially grateful for the consistent recognition of the artifact contract, the semantic/deterministic separation, and the clinical-translation focus. This response clarifies major points raised by the AC & reviewers.
1.Determinism and stability across LLMs. Meta-Reviewer, R1 (W6), R2, R3 We appreciate this point and would like to clarify our framing. Snakemake provides DAG-level determinism once a configuration C is fixed. For a fixed C assembled by the agent, it is automatically stored as an artifact and can be re-executed to produce the exact output. This is verified in Tables 1 & 2.Independent user interaction with the agent may produce different configuration C, however, it will always check pre-existing configuration artifact. We will clarify our definition of reproducibility in the camera-ready version.
2.Related work: BIDS, Nipoppy, Neurobagel. Meta-Reviewer, R1, R3 Thank you for the suggestion. We would like to clarify that BIDS, Nipoppy, and Neurobagel are complementary standards/tools rather than alternative agent architectures, i.e., each can be registered as a rule (or set of rules) and incorporated in our modular library S (e.g., a BIDS-conversion rule, a Neurobagel-query rule). Our major contribution lies in the semantic agent layer and artifact contract that sit above such tools, i.e., prompting with users to decide which tools to use in which dataset. We will add a related-work paragraph making the complementary relationship between our work and these existing tools more explicit.
3.Report timing. Meta-Reviewer, R1 (W3) This is a very valuable comment. Due to the page limit, we will include discuss this in the Discussion; more thorough time analysis will be included in our journal extension.
4.Other comments from R1 We sincerely thank R1 for the careful reading and valuable comments on strengths and weaknesses. Several points were already in the paper but can be stated more visibly, and we will tighten them in the camera-ready: stating decoding temperature explicitly (W6); positioning the paper as Clinical Translation (W10); elaborate on Figure 2 and Table 1 dialog (W8); and briefly noting agent-side failure modes and guardrails imposed by the contract (W9). The remaining comments (additional baselines, larger independently-authored query sets, rule-library scaling, broader LLM-size sweeps; W1, W2, W5, W7) are excellent directions, and we will include them in future work due to page limit.
5.Other comments from R2 We thank R2 for the positive evaluation. The cross-LLM reproducibility and related-work are addressed above, will be reflected in the camera-ready version.
6.Other comments from R3 We sincerely thank R3 and gently clarify a few points. We will incorporate clarifications in the camera-ready. 6a) DeepSeek-R1 the open-source model and the DeepSeek App are two different things: the former is a set of model weights and architecture released publicly, while the latter is a cloud-based service (e.g., ChatGPT). In our paper, we download the weights and run inference entirely on our local GPU (Sec. 2.2). Conceptually, it is not fundamentally different from downloading a pretrained nnU-Net and running inference locally. 6b) No fine-tuning was performed on any LLM, as such fine tuning is beyond clinical computational capabilities. The framework operates by prompting & artifact-contract grounding as inference mode, consistent with the single-A6000 footprint reported in the paper. 6c) Competing tools (e.g., alternative brain-segmentation pipelines) are each registered as a separate rule in S; the agent selects among them based on goal, artifact state, and user preference upon request.
We again thank the meta reviewer and all reviewers; the feedback will help us improve this and our future work.
Meta-Review
Meta-review #1
- Your recommendation
Provisional Accept
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper presents a novel AI Agent with semantic planning layers on top of a deterministic medical image processing workflow engine (Snakemake) to address the issues of adapting research workflows to clinical practice, where data heterogeneity and privacy constraints limit this transition. The paper reads easily and is well-organized. The problem formulation is clear and particularly remarkable for a software tool paper. In addition, the proposed framework has been designed to address clinical constraints: open-source models and local execution to meet privacy requirements. The authors provided a deep and broad evaluation of their framework across 3 datasets, including a real-world clinical dataset, along with relevant ablations. Overall, this paper presents a strong contribution that targets an unmet need in research-to-clinics transitions. I recommend its provisional acceptance, but would still like to suggest the authors address a few concerns raised by reviewers:
- R1 and R3 pointed out a lack of clarity in model presentation, with only minor justifications of implementation parameters, model choices, etc. Please add a bit more detail about these.
- For more insights on clinical usability, provide a few metrics on computation time, assembly time vs. workflow complexity, to better highlight the benefit of the proposed framework for clinical practice.
- All reviewers were concerned about the non-deterministic nature of the workflow. While the authors claim that adding this semantic planning layer on top of an LLM leads to determinism and reproducible results, they should also discuss stability across multiple LLMs or versions, and how the introduction of subtle variations, possibly from different computational environments, would affect reproducibility.
- The paper lacks discussion about how the proposed framework compares (theoretically) to relevant neuroimaging workflow tools such as Nipoppy, Neurobagel, or the BIDS app ecosystem. I strongly advise the authors to include a statement on related work and a literature review. In any case, as stated by R1, I believe this paper falls in the scope of “Clinical translation” rather than “MIC”. Since the aforementioned comments would be critical for a method paper, its positioning as a clinical translation paper encourages me to recommend acceptance.
