List of Papers Browse by Subject Areas Author List
Abstract
Large vision-language models (LVLMs) achieve strong performance on general-domain tasks. Supervised fine-tuning (SFT) is commonly used for domain adaptation. However, unlike traditional pretrain-finetune paradigms, the benefits of large-scale pretraining do not reliably transfer to LVLMs in medical image analysis. Using the proposed DePass-VL framework, we quantitatively analyze the failure of LVLMs. Together with attention-based qualitative analysis, we find that LVLMs struggle to effectively focus on task-relevant regions, indicating a lack of fine-grained vision-language semantic alignment. Although pretrained LVLMs exhibit strong 2D spatial alignment, this capability is not effectively activated during SFT. To address this gap, we propose Localization-Grounded Supervision (LGS), which introduces explicit localization signals to accelerate semantic alignment and improve both performance and interpretability. For 3D medical imaging tasks, pretrained LVLMs lack intrinsic 3D spatial alignment due to limited volumetric exposure. We therefore incorporate 3D anatomical localization data through multi-task learning to jointly build spatial and semantic alignment in 3D space. Experiments across anatomical regions, imaging modalities, model architectures, and parameter scales demonstrate that LGS is a simple yet effective paradigm for LVLM adaptation in medical image analysis. The code is available at: https://github.com/MSIIP/LGS
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/3411_paper.pdf
SharedIt Link: Not yet available
SpringerLink (DOI): Not yet available
Supplementary Material: Not Submitted
Link to the Code Repository
Link to the Dataset(s)
https://huggingface.co/datasets/shiym2000/LGS
BibTex
@InProceedings{ShiYim_LocalizationGrounded_MICCAI2026,
author = { Shi, Yiming AND Zhang, Hengyu AND Chen, Jiawei AND Yang, Shaoshuai AND Chen, Xi AND Zhu, Xun AND Wang, Kaiwen AND Li, Haolin AND Zhao, Xiuheng AND Zhang, Shuheng AND Li, Miao AND Wu, Ji},
title = { { Localization-Grounded Supervision: Revisiting Vanilla SFT of Large Vision-Language Models for Medical Image Analysis } },
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
volume = {LNCS 16879},
month = {September},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper studies why vanilla supervised fine-tuning of LVLMs often fails to fully transfer pretrained capabilities to medical image analysis, and proposes Localization-Grounded Supervision (LGS), which incorporates explicit localization signals to improve spatial grounding, semantic alignment, and downstream performance in both 2D and 3D medical imaging tasks.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The paper addresses an important question for medical LVLM adaptation, namely why vanilla SFT often does not fully translate pretrained capabilities into downstream medical performance. 2.The proposed method is simple and easy to integrate, and the overall motivation is clear. 3.The paper includes qualitative and quantitative analyses, including attention visualization and DePass-VL, to support its central claim about insufficient semantic alignment under vanilla SFT. 4.Experiments cover both 2D diagnosis and 3D report generation, as well as multiple architectures and parameter scales.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The proposed method is conceptually simple, and the methodological novelty is limited. The core idea is to augment the supervision target with localization signals, and extend this to 3D through an additional localization task with multi-task learning. While practical, this is more a supervision design than a substantially new modeling contribution. 2.Some of the reported gains, especially on the 2D diagnosis tasks, are extremely large. For example, the improvement on CrossMoDA2021 is surprisingly large for such a lightweight change, which raises concerns about benchmark difficulty, possible shortcut signals, or whether the vanilla SFT baseline is sufficiently strong. 3.The 2D diagnosis benchmarks are derived from segmentation datasets by treating slices with lesion masks as positive and slices without masks as negative, with a balanced 1:1 setting. This construction is convenient, but it may oversimplify the task and may not fully reflect realistic fine-grained medical diagnosis scenarios. 4.The paper makes broad claims about the limitations of vanilla SFT and attributes them mainly to insufficient semantic alignment. Although the attention analysis and DePass-VL results are suggestive, the current experiments may not be sufficient to support such general conclusions across medical LVLM settings.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper studies an important problem in medical LVLM adaptation and proposes a simple localization-grounded supervision strategy to improve spatial grounding during SFT. The motivation is clear, and I appreciate the effort to combine analysis with experiments across both 2D and 3D settings. While the method is conceptually simple and some of the reported gains raise questions about benchmark difficulty, I still find the overall message meaningful and the empirical results promising. Overall, I view the paper as slightly above the acceptance threshold.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
After reading the rebuttal, I would keep my recommendation as Accept. The authors provided useful clarifications on several of my main concerns. In particular, they clarified that the CrossMoDA2021 experiment is a derived 2D slice-level binary lesion-presence classification task rather than the original 3D segmentation challenge, and that vanilla SFT and LGS are compared under the same split, training setup, and prompt setting. They also clarified the LUNA16 dataset construction, where positive samples are selected from the middle slice of the nodule mask and negative samples are selected from adjacent slices outside the mask boundary, which partially alleviates my concern about overly coarse shortcut signals.
The rebuttal also improves the methodological clarity of LGS. The authors clarify that LGS does not introduce a new detection head or regression loss, but simply changes the autoregressive target sequence from answer-only prediction to answer plus textual localization coordinates. This makes the method’s simplicity and practical value clearer. I also appreciate that the authors acknowledge the need to reposition the contribution relative to prior grounded medical VLMs, moderate the broad claims about vanilla SFT, and present DePass-VL mainly as a region-level application/adaptation of DePass rather than an entirely new attribution method.
That said, some limitations remain. The method is still conceptually simple, the novelty is moderate, and several important experimental details, including split sizes, patient-level split protocol, variance across seeds, and more complete comparisons with related grounded medical VLM work, still need to be clearly included in the final version. Overall, however, the rebuttal addresses my main concerns sufficiently, and I continue to view the paper as a useful and practically meaningful contribution slightly above the acceptance threshold.
Review #2
- Please describe the contribution of the paper
The paper makes two related claims. First, through attention visualization and a VLM-adapted version of DePass (called DePass-VL), the authors argue that vanilla supervised fine-tuning of large vision-language models on medical tasks fails to produce strong fine-grained vision-language semantic alignment because attention is diffuse and the ratio of contribution from task-relevant image regions to non-relevant ones is close to 1.Second, they propose Localization-Grounded Supervision (LGS), in which the model is trained to predict bounding-box coordinates alongside the answer, to exploit the pretrained 2D spatial grounding of LVLMs. For 3D tasks, where pretrained LVLMs lack volumetric grounding, an auxiliary 3D anatomical localization task is added via multi-task learning. Experiments cover 2D disease diagnosis on LUNA16 and CrossMoDA2021, 3D report generation on RadGenome-Chest CT, and scaling across 2B/4B/8B variants of Qwen3-VL and InternVL3.5.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
The central empirical observation that vanilla SFT on small-lesion medical tasks produces diffuse attention and doesn’t push the LVLM to attend to the lesion, is an interesting and useful finding. Also quantifying it with a region-level attribution score is a nice contribution.
-
The idea of attaching bounding-box outputs to the answer to push grounding during SFT is simple and the kind of thing that’s easy for others to adopt. I like simple interventions, and the paper presents this one cleanly.
-
The scaling robustness in Figure 5 that shows the effect across two architecture families and three model sizes is a good presentation of the paper’s contribution, and the consistency of the gains across scales is a useful piece of evidence.
-
The attention visualization pairs in Figure 2 are informative. The diffuse-vs-concentrated contrast is visually persuasive, and the accompanying DePass-VL contribution ratio in Figure 3 implements what the pictures suggest.
-
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
-
Direct prior art on grounded medical report generation is not cited. MAIRA-2 (Bannur et al., “MAIRA-2: Grounded Radiology Report Generation”, Microsoft Research 2024) does essentially the same thing the paper proposes: train an LVLM to predict bounding boxes interleaved with report text as tokenized outputs, using standard cross-entropy on the combined sequence. No special grounding loss; no regression head; grounding is handled autoregressively. VividMed extends this with anatomical segmentation grounding. RadVLM (Deperrois et al. 2025) similarly includes grounding with bounding-box supervision as an instruction type. MedGround frames the “image–text–box triplet” as the key to closing the cognitive-perceptual gap. None of these are cited. The “localization coordinates as a complementary output format” is not a new idea in medical LVLMs. It has a two-year history. The paper’s contribution needs to be repositioned relative to that literature: is the insight that it works with standard Qwen3-VL/InternVL3.5 out of the box without custom detection heads (that is a reasonable, if narrower, contribution), or is it the attention analysis that motivates the choice? Reframing the work as such would be more novel.
-
The CrossMoDA2021 result is implausibly high. Table 1 reports Qwen3-VL (LGS) at ACC=0.9844 / F1=0.9846 on CrossMoDA2021.CrossMoDA2021 is a cross-modality domain adaptation challenge (ceT1 to hrT2) for vestibular schwannoma and cochlea segmentation; it is not easy. The paper converts it to 2D slice classification by presence of segmentation mask. A jump from 0.5938 (vanilla SFT) to 0.9844 with the same backbone and the only difference being that bounding-box coordinates are now in the output is not a small effect. It’s 40 accuracy points on an allegedly balanced 1:1 positive/negative split. The obvious hypotheses are (i) the test split shares patients with the training split (no patient-level split is mentioned), allowing anatomical shortcut learning; (ii) the test set is small and fortuitously easy; (iii) the slice-level “positive = slice has a mask” task is largely solvable by looking at the anatomy visible in the slice, and the bounding-box supervision effectively teaches the model “output a box whenever the relevant anatomy is on screen.” None of these are ruled out by the text. Please provide: cohort sizes, train/val/test split protocol (especially patient-level), class balance per split, and variances.
-
The LUNA16 benchmark construction is nonstandard and underspecified. LUNA16 is a nodule-detection challenge (888 scans, LIDC-IDRI); the paper converts it to 2D slice classification. Again, no split protocol, no patient-level separation is noted, no dataset size, no discussion of how slice-level labels are derived (a slice can contain a nodule partially or have a nodule a few slices away so how are borderline slices handled?). These details matter because the whole argument rests on Table 1’s numbers.
-
DePass-VL is a thin extension. The paper presents DePass-VL as a “regional contribution analysis framework tailored for LVLMs” and one of its two quantitative contributions. DePass-VL partitions input tokens into three groups (task-relevant image regions, non-relevant image regions, other) and runs the standard DePass decomposition. Given a bounding box, doing this is straightforward as it’s an application of DePass, not a methodological extension of it. I would tone down the “we propose DePass-VL” claim to at most, “we apply DePass to LVLMs under a region-level partition.”
-
The attention analysis conflates two different things. Section 2.1 focuses on self-attention weights from answer tokens to image tokens. But in modern decoder-only LVLMs, the signal that propagates to the output logit is not the attention weight alone rather it’s the attention-weighted value vectors run through downstream MLPs and residual streams. DePass itself exists precisely because attention weights are not faithful attributions. The paper acknowledges this at the start of Section 2.2 (“attention-based analysis captures interactions within MHSA but does not account for other components such as the MLP”), which is correct but then the narrative continues to lean on the attention visualisation in Figure 2 as the primary qualitative evidence, and Section 4.5 re-emphasises it. I would recommend making DePass-VL the main evidence and the attention maps the illustrative companion, not the other way round.
-
Table 1 reports point estimates. Figures 4 and 5 likewise. With the kind of tiny derived datasets being used (slices from LUNA16 / CrossMoDA2021), seed-to-seed variance on VLM SFT can be substantial. Please report at least three seeds and include standard deviations; without them the 4-5 accuracy point gains are not convincingly separated from noise, and the 40-point CrossMoDA jump, while too large to be noise, still wants verification that it isn’t a single-seed artefact.
-
The “RL-based methods don’t replace SFT” claim is over-strong from the evidence. The paper runs GRPO on both datasets with apparently one configuration and reports it underperforming SFT. Fine, but GRPO notoriously needs careful reward design and base-model initialisation. Concluding that “RL-based methods such as GRPO do not currently replace SFT as a practical training paradigm for LVLMs in medical image analysis” from two datapoints with a single hyperparameter setting is not warranted. This can be softened.
-
Vanilla SFT accuracies in Figure 5 look unlike seen in practice. The Qwen3-VL 2B/4B/8B vanilla SFT curves seem to plateau around 0.5 accuracy on a 1:1 balanced LUNA16 slice task i.e. essentially chance. A modern 8B LVLM should be able to memorise or at least noisily fit a small balanced classification task, even with suboptimal prompting. To explain these given numbers in the submission, the paper should explain what’s going wrong (learning-rate issue, answer-format mismatch, prompt-ending whitespace, LLM-head-frozen-vs-not, etc.); if they are seed-selected or early-training snapshots, the figure needs caveats. At face value, the vanilla-SFT baseline looks under-tuned.
-
Qwen3-VL (ref [3]) is trained on a very large multimodal corpus that almost certainly includes LIDC-IDRI and LUNA16 derivatives in some form. The paper acknowledges this concern in Sec4.2 (“potential overlap with pretraining data may bias evaluation”) but the only mitigation is “derive 2D from 3D segmentation data.” That doesn’t actually address the data overlap concern. If the pretraining corpus contained LUNA16 images with or without captions mentioning nodules, slice-level labels derived from the same volumes can still be overlapping. A held-out set, or at minimum a de-identified subset with noise applied, would be more convincing. In the 3D setting (RadGenome-Chest CT / CT-RATE), this concern increases as CT-RATE is heavily circulated in the VLM community.
-
Quantitative comparison to MAIRA-2 or similar grounded-VLM baselines is absent. The 3D report generation numbers on RadGenome-Chest CT (F1 0.263, IoU 0.673) are presented without reference to MAIRA-2, VividMed, RadVLM, or any of the other grounded-VLM papers. These are not all directly comparable (MAIRA-2 is chest X-ray, not 3D CT), but the paper’s main empirical claim is that LGS is the right paradigm for grounded medical VLM training, and that claim is not tested against the closest existing paradigm.
Minor citation issues. -Ref [9]: the citation should ideally point to both the challenge and the paper if the authors used the challenge framework.
-
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The authors claimed to release the source code and/or dataset upon acceptance of the submission.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The central observation (vanilla SFT attention is diffuse and adding bounding-box outputs concentrates it) is reasonable and strong, and the empirical scaling evidence is useful. But the paper overstates its novelty relative to MAIRA-2 and related grounded-VLM work, which it does not cite. One of its main results (CrossMoDA2021 at 98% accuracy) is not adequately defended against shortcut-learning concerns; dataset splits and protocol details are not enough; there are no error bars; and the GRPO narrative is not based on evidence seen in practice. With better positioning against prior art, a properly defended CrossMoDA result, patient-level splits, and some seed-averaged numbers, this would be a good paper. As submitted it’s below my bar for the venue.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
N/A
- [Post rebuttal] Please justify your final decision from above.
N/A
Review #3
- Please describe the contribution of the paper
This paper investigates why vanilla supervised fine-tuning (SFT) underperforms for LVLMs in medical imaging and propose 3 fold contribution in this direction: by extending DePass framework to LVLM which they call ‘DePass-VL’, introduce a technique Localization-Grounded Supervision (LGS) that uses localization signals to improve spatial alignment during SFT and lastly also show the improvement not just for 2D dataset but 3D dataset.
- Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The abstract and introduction dig into the core problem of pretrained models to understand why vanilla SFT fails and authors claim to introduce an interpretability with DePass-VL to understand this better.
The idea of incorporating explicit localization signals (LGS) is simple and practically useful - specially for small lesions which they show in Figure 2
The authors attempt to mitigate data leakage concerns which I really like and appreciate. They identified that large pretrained models may have seen common datasets like LUNA16 and for a fair comparison they went further by constructing datasets from 3D segmentation sources, which is a thoughtful design choice.
- Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
1.The paper lacks essential details about dataset construction. Specifically, the number of samples for training/validation/test splits. In Table 1 - even though the datasets used to derive the images is 3D segmentation dataset - with the metrics reported it seems like authors did a classification task - lesion or no-lesion. Task is not explicitly clarified in the paper. These omissions make it difficult to assess statistical significance, lower bound and reproducibility.
2.Figure 5 reports results for InternVL3.5, but these results are not included in Table 1.A unified presentation would improve clarity and allow easier comparison across models. It would benefit the authors to include these results in Table 1. 3.The comparison with GRPO is underexplained. In particular, the paper does not clearly describe the training setup (reward design, policy definition, hyperparameters, etc). And why use GRPO and not other methods. Overall, comparison with RL is ill-motivated.
4.Contribution of DePass-VL is not justified. To understand the novelty and requirement, it would benefit the paper if there is a description of what is lacking is DePass and how authors resolve it. Moreover, compare how this interpretability contribution helps understanding vanilla SFT better.
5.In case of LGS which is the most significant contribution of the paper, key details are missing from the text on how is it implemented. This include mainly how are localization signals incorporated into training (and loss formulation)
6.Minor comments a. Overall the writing feels a bit disintegrated in the paper. I like the effort put in for the experiments and the idea but I struggled to find key details and map them to the figures.
b. Section 4.5 largely reiterates observations already presented in Figures 2 and 3, and thus can be avoided.
c. Fig 3(b): x-axis says Vanilla SFT/LGS and I assume pink violin plot is SFT and blobs are LGS. If this is the case, the interpretation ‘value smaller than 1 indicates that LGS enhances the LVLM’s focus on task-relevant regions‘ is also valid for Vanilla SFT.
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?
N/A
- Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.
(4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The authors have good instincts and present an idea specifically LGS and qualitative analysis of why vanilla SFt on LVLM fails. However, despite these strengths, the paper suffers from a lack of methodological clarity, incomplete experimental details, and several presentation inconsistencies. In particular, key aspects of the proposed method and evaluation setup are insufficiently described as mentioned above, making it difficult to fully assess and reproduce the reported results. While the overall direction is compelling, these gaps have led me to recommend a weak accept.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.
Accept
- [Post rebuttal] Please justify your final decision from above.
I thank the authors for their response and for addressing several of my concerns.
Regarding dataset construction, the authors indicate that the relevant details will be added to the paper. While this is appreciated, it would have been more informative had at least a summary of the construction procedure been provided directly in the rebuttal. Furthermore, if the dataset is non-standard, additional baseline comparisons would be necessary to disentangle whether the reported improvements stem from the data itself or from the proposed algorithmic contributions. This remains an open concern.
Regarding GRPO, the authors state that they will reduce the emphasis on this component. I would like to clarify that the level of emphasis was not the issue I raised; my concern was directed at the rationale for its necessity in the first place, which has not been adequately addressed.
On the positive side, I appreciate that the authors have agreed to provide clarification on LGS and DePass-VL, and that they commit to including the InternVL3.5 results in the main table. These are meaningful improvements to the manuscript.
Taking all of the above into account, I will maintain my score at weak accept.
Author Feedback
We thank all reviewers for their constructive feedback. We are encouraged that the reviewers recognize the importance of revisiting vanilla SFT for medical LVLM adaptation, the practical value of LGS, and the usefulness of our analyses. We address the key concerns below.
1.Novelty and prior grounded medical VLMs (R1&R2) We thank R2 for pointing out related works such as MAIRA-2.We agree that predicting localization coordinates with text has been explored. Our contribution is not claiming coordinate prediction itself as new. Instead, we use localization output as a supervision interface to analyze and improve vanilla SFT for off-the-shelf LVLMs. The key contribution is the combination of simple supervision design, mechanistic analysis of vanilla SFT failure, and extension to 3D medical imaging. We will revise Introduction to clarify this positioning.
2.CrossMoDA result and baseline fairness (R1&R2) Our CrossMoDA experiment is not the original 3D cross-modality segmentation challenge, but a derived 2D slice-level binary lesion-presence classification task. Thus, the high accuracy should not be interpreted as solving the original CrossMoDA task. Comparisons in all experiments are controlled (no patient-level data leakage): vanilla SFT and LGS use the same base model, split, learning rate, batch size, optimizer, epochs, parameter-freezing strategy, and prompt setting. The only intended difference is that LGS adds localization coordinates to the answer target. The substantial improvement over baseline precisely demonstrates the simplicity and effectiveness of LGS. We will add training details in the revised version, and release the split for reproducibility.
3.Dataset construction and benchmark purpose (R1&R2&R3) We agree that dataset construction needs more detail. The 2D benchmarks are controlled fine-grained spatial diagnosis tasks, not full clinical diagnosis simulations. Take LUNA16 as the example, we select CT scans with a single lung nodule, use the middle slice of the nodule mask as the positive sample, and use the adjacent slice just outside the mask boundary as the negative sample. This creates lesion-near negatives and reduces coarse anatomical shortcuts. We will report train/validation sizes, patient/scan-level split, label rules, and more details in the revised version.
4.LGS implementation (R3) LGS does not add a detection head or regression loss. Training still uses standard autoregressive next-token cross-entropy. The only change is the target sequence: vanilla SFT predicts only the answer, while LGS predicts the answer plus textual coordinates, e.g., “Yes, [x_min, y_min, x_max, y_max].” For 3D tasks, report-generation samples are mixed with anatomical localization. We will add target examples and loss formulation in the revised version.
5.DePass-VL and attention analysis (R2&R3) We will revise to clarify that DePass-VL is the first adaptation of DePass to LVLMs with region-level token partitions: task-relevant image regions, non-relevant image regions, and other tokens. As correctly pointed out (R2), attention weights alone are not faithful attributions. Thus, DePass-based contribution analysis will be the main quantitative evidence, while attention maps will be presented as qualitative illustrations in the revised version.
6.Claim scope, GRPO, variance, and presentation (R1&R2&R3) We will moderate claims: insufficient fine-grained semantic alignment is an important factor in fine-grained medical tasks, not a only universal cause. We will also soften the GRPO conclusion and present it only as a preliminary comparison, with added reward and hyperparameter details. If permitted by the rebuttal and revision policy, we will report mean±std for the main results in Tab.1, Fig.4&5.Finally, we will add InternVL3.5 results in Fig.5 to Tab.1, improve Fig.3(b) caption, clarify that the plotted value is the per-sample relative ratio, and shorten the repetitive Section 4.5.
Meta-Review
Meta-review #1
- Your recommendation
Invite for Rebuttal
- Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes a simple LGS method to address the diffuse attention problem after the vanilla SFT on various tasks of MLLMs. However, there exist certain concerns on the novelty (e.g., comparison with prior work like MAIRA-A), unexpectedly high performance gains on CrossModaby LGS, and insufficient explanations on implementations and datasets. Therefore, the authors are encouraged to clarify these points for rebuttal.
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The rebuttal addresses the reviewers’ concerns and therefore I lean towards acceptance. The authors are encouraged to justify the method choice (e.g., GRPO) and provide in-depth discussions on expperiments in the final version.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
All reviewers highlight the contributions of the paper and suggest to accept the submission.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation.
The reviewers appreciate the motivation and extensive experimental results, and the initial concerns have been mostly addressed during the rebuttal. I recommend accepting this submission.
