Abstract

Chest X-ray (CXR) interpretation often requires longitudinal comparison to assess disease progression. Existing approaches typically rely on temporal feature fusion or inter-study discrepancy modeling, yet remain limited in capturing subtle progression semantics and overlook the inherently directional nature of disease trajectories. In this paper, we propose ProTrans, a novel vision-language pretraining framework that formulates disease progression as a directional semantic transition between paired CXR studies. ProTrans leverages radiology reports to anchor individual CXR representations within interpretable disease states, and introduces a learnable progression feature map to explicitly encode semantic shifts between states, aligned with report-derived progression descriptions. To enforce direction-aware perception, ProTrans incorporates a reversed temporal modeling process and imposes bidirectional reconstruction consistency across states and transitions, thereby disentangling directional semantics and promoting coherent trajectory modeling. Extensive experiments on longitudinal downstream tasks, including disease progression classification and progression captioning, demonstrate that ProTrans consistently outperforms existing methods, establishing a unified pretraining framework for longitudinal CXR understanding.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2026/paper/3762_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/RPIDIAL/ProTrans

Link to the Dataset(s)

N/A

BibTex

@InProceedings{HuZha_Learning_MICCAI2026,
        author = { Hu, Zhangfeng AND Yang, Zefan AND Wang, Ge AND Syeda-Mahmood, Tanveer AND Burade, Anushree AND Kalra, Mannudeep K. AND Yan, Pingkun},
        title = { { Learning Directional Semantic Transitions for Longitudinal Chest X-Ray Analysis } },
        booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026},
        year = {2026},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 16887},
        month = {September},
        page = {pending}
}


Reviews

Review #1

  • Please describe the contribution of the paper
    1. The paper proposes ProTrans framework that models disease progression as a directional semantic transition using a learnable progression feature map between paired CXR embeddings.
    2. Joint state and transition contrastive alignment that grounds both CXR states and their interval transitions in a shared semantic space using radiology reports
    3. Directional awareness is given via reversed temporal modeling. A backward sequence with a separate learnable reversed progression map is constructed, forcing the model to distinguish direction of change from mere change detection.
  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The problem that this paper is tackling is clinically important and underexplored.
    • Framing progression as a directional semantic transition is conceptually well motivated.
    • Evaluation is reasonably broad, covering both classification and captioning, with appropriate clinical metrics.
    • It is additionally good practice that overlap with MS-CXR-T is excluded from the pretraining set.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    1. Progression encoder design
      • Figure1 (b) indicates that temporal attention is applied over the reshaped tensor (B-N_I)x3xD, while spatial attention is handled separately over (B-3)xN_IxD. This means cross-time interaction is built around fixed patch indices across visits. For Paired CXRs, where patient positioning, inspiration, and projection can vary, I believe it is a nontrivial assumption.
    2. Captioning details
      • The manuscript only states that the learned progression representations are fed into a frozen BioMistral-7B. However it does not explain which visual representation are fed to the LLM, how they are projected into the language model input space, how many visual tokens were used, or what parameters are actually optimized. More implementation details are needed.
    3. Irreversible clinical findings
      • Not all findings are reversible. The framework assumes all disease processes are reversible (‘improve’ -> ‘worsen’) but there are cases such as fibrosis, calcification, scarring, and post-surgical changes. A calcified nodule cannot de-calcify. Training the model on clinically impossible transitions conflates irreversibility with directionality and prevents the model from learning to flag impossible trajectories. Irreversible findings should be supervised only through the reconstruction path, where directionality is enforced geometrically rather than through flipped semantic labels. At minimum, this assumption should be acknowledged and discussed as a limitation.
    4. End-to-end training
      • The overall objective jointly combines all losses from the start and my intuition is that training all losses simultaneously from the start may make it difficult to learn good progression representations, which is the core contribution since P would be encoding transitions between states that are themselves still being learned. I am curious whether a curriculum approach, establishing stable state representations first before introducing the transition and reconstruction objectives, would improve the quality of the learned progression features.
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    Do you plan to release the code? I’d particularly like to inspect the temporal attention implementation to better understand how cross-visit spatial correspondence is handled.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Questions 1.Was the visual encoder initialized from the pretrained weights or trained from scratch? Also why did the author train end-to-end instead of staged training? 2.What exact visual representation is passed to BioMistral-7B? 3.How are irreversible findings handled?

    The paper addresses a clinically important and underexplored problem, and the evaluation covers appropriate tasks and metrics. However, key technical and clinical concerns, particularly around the progression encoder’s spatial alignment assumption, unexamined reversibility assumption in label flipping, and some of the design and pipeline choices, weakens confidence in the current claims. I am wiling to revise my score based on the author’s rebuttal, especially if the questions above are addressed.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The rebuttal addressed some implementation concerns, including the captioning interface, visual encoder initialization, planned code release. However several limitations remain only partially resolved (spatial alignment, etc.)

    My main remaining concern is the handling of clinically invalid reversed supervision. The rebuttal states that invalid reversed labels were not used, but the manuscript does not clearly explain how this was determined. This issue cannot be resolved simply by relying on chest imagenome labels or by assuming that dataset-provided progression labels are clinically reversible. Clinically irreversible or non-regressing findings may require phrase-level filtering, e.g., calcification, scarring, fibrosis, surgical changes,etc. Edema or effusions can still be irreversible. The paper should acknowledge this and discuss this as a clinical limitation.

    Despite these concerns, the paper addresses an important and underexplored clinical problem, proposes a reasonable architecture, and includes relevant evaluations across progression classification and captioning. The paper is not fully convincing and the novelty is moderate, but I think it remains slightly above the acceptance threshold.



Review #2

  • Please describe the contribution of the paper

    This paper proposes ProTrans, a longitudinal chest X-ray pretraining framework that models disease progression as a directional semantic transition between prior and current studies, using joint state-transition contrastive alignment and bidirectional reconstruction. The idea of explicitly learning transitions rather than only fusing features or modeling differences is novel and well motivated, and the paper presents strong experimental results on both progression classification and progression captioning.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.The overall technical direction is strong and timely. Modeling longitudinal progression as a semantic transition, together with contrastive supervision and directional constraints, is a fresh idea that goes beyond standard temporal fusion. 2.The experimental section is generally thorough and convincing for a MICCAI submission. The paper evaluates on multiple downstream tasks, compares against several relevant baselines, and includes ablations that support the usefulness of the proposed pretraining framework.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1.The method combines too many loss terms and training constraints, which makes the source of improvement hard to interpret. Even with the ablation study, it remains difficult to disentangle how much comes from state alignment, transition alignment, bidirectional modeling, or reconstruction, so the mechanism behind the gains is still somewhat unclear. 2.Figure 1 does not communicate the method clearly enough. The overview contains many blocks and arrows but is not modularized well, so it is hard to quickly understand the data flow and identify which component is the main contribution. 3.The encoder choices are not fully justified for the chest X-ray setting. The paper uses ViT-B/16 and BioClinicalBERT, but does not explain why it does not adopt encoders that were specifically pretrained on CXR data, which could be a stronger and more natural choice for this problem. 4.The improvement on MS-CXR-T over Coca-CXR is not consistently strong at the per-category level. Although ProTrans achieves the best average accuracy in the text prototype setting, the gain over Coca-CXR is small overall, and ProTrans is worse on three of the five findings, which weakens the claim of broad superiority on this benchmark. 5.The similarity-map analysis in Figure 2 raises a robustness concern that is not discussed. Since the visualization appears to rely on position-sensitive image features, it is unclear how well the method would work when prior and current CXRs are not well aligned due to pose, cropping, or acquisition differences. 6.The paper could do more to discuss the practical limits of the transition representation. The method assumes that a learnable transition map can reliably capture disease evolution, but the manuscript does not examine failure cases where progression cues are weak, spatially shifted, or confounded by acquisition variation.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — marginally above the acceptance threshold, but would not mind if rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The main reason for the positive score is that the paper introduces a genuinely interesting formulation of longitudinal CXR analysis as directional semantic transition learning, and supports it with solid experiments across classification and captioning tasks. The main reservations are about interpretability and presentation: the training objective is quite complex, some design choices are not fully justified, and the empirical advantage over the strongest baseline is not uniformly strong across all categories.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The rebuttal adequately addresses my main concerns. In particular, the authors clarify that the progression map is intended as an explicit semantic transition representation rather than a simple soft feature fusion, and they provide a reasonable explanation for why semantic transition learning may reduce sensitivity to spatial misalignment compared with direct subtraction or low-level feature differencing. They also clarify the captioning implementation, the use of CXR-pretrained visual initialization, and the handling of clinically invalid reversed labels. While some concerns remain, especially regarding robustness to acquisition variation, the complexity of the training objective, and the limited per-category gains over the strongest baseline on MS-CXR-T, these issues do not outweigh the paper’s strengths. Overall, the paper presents a novel and clinically relevant formulation for longitudinal CXR progression modeling, supported by solid experiments and useful ablations. I therefore maintain a positive recommendation and support acceptance.



Review #3

  • Please describe the contribution of the paper

    The authors propose a vision-language model that aims to model disease progression between longitudinal chest X-ray scans. The progression is included in forward and backward direction. They augment their model by paired text-based reports.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The method section is diligently written

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The abstract and introduction contain lots of jargon and vague formulations that make them hard to follow. From my understanding, the manuscript lacks either 1) an argument why the progression map is actually used by the model and not just a “soft” addition or 2) an ablation showing its merit. Furthermore, the language model component should also be ablated.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not mention open access to source code or data, but provides a clear and detailed description of the algorithm to ensure reproducibility.

  • Based on your review and your understanding of the MICCAI Scientific Code of Ethics, do you believe this submission may involve a potential ethics concern or violation?

    N/A

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2026/en/REVIEWER-GUIDELINES.html
    • State contrastive alignment:
    • Consider changing “I2T” into “(I, T)” (positional arguments) to make the reverse “T2I” logits clearer.
    • What are the dimensions of $q^t$? What does indexing it by $k$ denote here?
    • Transition Contrastive Alignment: The claim “learned progression representations capture meaningful transition semantics” by “align[ing] them with corresponding textual descriptions of disease progression” is not sufficiently substantiated in the results, as far as I understand. Please tone it down or remove it.
    • Bidirectional Reconstruction: Consider citing a source for “InfoNCE”.
    • Ablation study: Did you ablate the learnable progression feature map altogether? This is not clear to me from the paragraph.
    • Table 3: What does “bidirection” denote here?
  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — marginally below the acceptance threshold, but would not mind if accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The abstract and introduction contain lots of jargon and vague formulations that make them hard to follow. In contrast, the method section is diligently written. From my understanding, the manuscript lacks either 1) an argument why the progression map is actually used by the model and not just a “soft” addition or 2) an ablation showing its merit. Furthermore, the language model component should also be ablated.

  • Reviewer confidence

    Not confident (1)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Reject

  • [Post rebuttal] Please justify your final decision from above.

    The argument proposed by the authors on why the progression map is actually used by the model is still not clear to me and by MICCAI guidelines the authors cannot add the ablation in the revision. I encourage the authors to address these concerns and resubmit.



Author Feedback

We thank the reviewers for their constructive comments and for recognizing the clinical importance (R2,3), novelty (R2,3), and comprehensive evaluation (R2,3) of our work. Upon acceptance, we will publicly release the code and model weights to facilitate reproducibility. We clarify concerns below. 1.Progression map and spatial alignment (R2,3,4): We agree that longitudinal CXRs are often not spatially aligned, which motivated the use of semantic transition. Unlike direct subtraction or low-level feature differencing, which can be easily confounded by acquisition variation, ProTrans learns disease progression at the semantic level. Specifically, we use state contrastive alignment to ground prior/current images into corresponding semantic states, and introduce a learnable progression map to explicitly model the shift between them. Similar to CLS token in ViT, this map interacts with prior/current visual maps via the progression encoder and is supervised by report-derived progression descriptions via transition contrastive alignment, encouraging it to capture semantically grounded progression cues. We clarify that the progression encoder first uses temporal attention to capture local cross-time cues, and then uses spatial attention to exchange information globally across all visual tokens. After pretraining, the learned progression map is directly used for downstream tasks. It is an explicit transition representation rather than a soft addition. To further disentangle its role, we will add a targeted ablation that removes the progression map and replaces it with direct prior-current feature fusion. This directly tests whether downstream performance comes from explicit transition learning rather than from generic paired-image fusion. Although ProTrans aims to reduce sensitivity to acquisition variation by learning semantic transitions rather than raw visual differences, we agree that this doesn’t fully mitigate robustness concerns. We will add failure-case analysis and explicit registration as future extensions. 2.Captioning implementation (R2): The learned progression map is projected into the LLM embedding dimension via a trainable linear layer and used as visual prompts for frozen BioMistral. Only the projection layer is optimized during downstream adaptation, isolating the contribution of the learned progression features. We will add more details. 3.Irreversible findings (R2): We agree that flipped semantic labels are inappropriate for irreversible findings. In our work, we didn’t apply reversed semantic supervision when the reversed label is clinically invalid. Our current supervision from Chest ImaGenome mainly includes findings such as edema, pleural effusion, etc., where improved/worsened transitions are clinically valid. Reconstruction further provides a label-free visual constraint. 4.Training strategy and components (R2,3): Good point on curriculum training. Our motivation for joint optimization is that state and transition learning are coupled: if the model is first trained only with state alignment, it may learn good single-study semantic features but ignore progression-relevant cues. Combining transition alignment encourages the state space to support directional disease evolution. Ablation study shows that removing state/transition alignment, reconstruction, or bidirectional modeling degrades performance, with transition alignment causing the largest drop. All components contribute to reliable progression learning. 5.Encoder choice (R2,3): We initialized the visual encoder (ViT-B/16) with CXR-pretrained weights. ProTrans is a backbone-agnostic progression pretraining framework, and can be combined with stronger backbones. 6.Others: q^t is a BxB one-hot alignment matrix, B is the batch size; element indexed by k denoting the k-th sample in the batch (R4). “w/o bidirection” means removing the reversed temporal modeling (R4). We will improve Fig.1 (R3) and abstract, introduction (R4), and add InfoNCE citation (R4).




Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • Please justify your decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper proposes ProTrans, a longitudinal CXR pretraining framework that models disease progression as a directional semantic transition using a learnable progression feature map between paired embeddings, supported by joint state-transition contrastive alignment and bidirectional reconstruction. Reviewers acknowledged that it is a clinically important and underexplored problem, a novel formulation of progression as directional semantic transition, and a broad evaluation covering classification and captioning. Weaknesses are also pointed out: (1) The progression encoder assumes patch‑wise spatial correspondence across visits, which is fragile given variation in patient positioning, inspiration, and acquisition (R2, R3). (2) The complex multi‑loss objective makes the source of improvement unclear; a direct ablation of the progression map itself is missing, and the contribution of the LLM component is not isolated (R3, R4). (3) Empirical gains over Coca‑CXR are not uniform across findings, and the abstract/introduction contains vague jargon (R3, R4).

    Issues to be discussed in rebuttal:

    • Spatial alignment assumption in the progression encoder under realistic CXR variation.
    • Handling of irreversible clinical findings and their impact on directionality learning.
    • Disentangling the contribution of the progression map via targeted ablation; clarifying whether the map is actively used or a “soft” addition.
    • Missing captioning details (visual tokens, projection, trainable parameters).
    • Justification for not using CXR‑pretrained encoders.
  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The paper introduces a novel and clinically relevant framework for modeling disease progression as directional semantic transitions in longitudinal chest X-rays. The rebuttal effectively clarified implementation details, encoder choices, and the role of the progression map, addressing the majority of critiques from Reviewers #2 and #3.They clarified that the progression map is an explicit semantic transition representation (not a “soft” addition), explained the captioning interface (projection layer into frozen BioMistral), confirmed CXR-pretrained ViT initialization, and acknowledged that irreversible findings were not subjected to reversed supervision because the Chest ImaGenome labels primarily involve reversible conditions (e.g., edema, effusion). They also proposed a targeted ablation removing the progression map, though this was a promise rather than an existing result. Reviewers #2 and #3 found the rebuttal adequate and changed their scores to accept, while Reviewer #4 remained unconvinced about whether the progression map is actually used and noted that the ablation cannot be added post-submission. Despite these limitations, the problem is underexplored and important, the method is well-motivated, and the evaluation is solid. While the strength outweighs the remaining issues, I believe the paper is above the acceptance threshold.



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    This paper proposes ProTrans, a longitudinal chest X-ray pretraining framework that models disease progression as directional semantic transitions between prior and current studies through joint state-transition contrastive alignment and bidirectional reconstruction. The work presents a conceptually interesting formulation for progression modeling, supported by evaluations across both progression classification and captioning tasks with clinically meaningful metrics. The main remaining concerns are that the progression encoder may still be sensitive to spatial misalignment across longitudinal CXRs, the contribution of the learnable progression map is not fully isolated due to the lack of a targeted ablation, and the handling of clinically invalid reversed supervision could be more clearly justified and discussed as a limitation. However, the rebuttal adequately clarified several implementation and design choices and two reviewers maintained positive recommendations after rebuttal. Overall, this work presents a technically reasonable and clinically meaningful formulation for longitudinal progression modeling that appears slightly above the acceptance threshold.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation.

    The paper addresses an important longitudinal CXR progression modeling problem with a novel directional semantic transition framework and solid experimental validation, despite some remaining limitations.



back to top