Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Medical image restoration (MedIR) aims to recover high-quality medical images from their low-quality counterparts. Recent advancements in MedIR have focused on All-in-One models capable of simultaneously addressing multiple different MedIR tasks. However, due to significant differences in both modality and degradation types, using a shared model for these diverse tasks requires careful consideration of two critical inter-task relationships: \textit{task interference}, which occurs when conflicting gradient update directions arise across tasks on the same parameter, and \textit{task imbalance}, which refers to uneven optimization caused by varying learning difficulties inherent to each task. To address these challenges, we propose a task-adaptive Transformer (TAT), a novel framework that dynamically adapts to different tasks through two key innovations. First, a task-adaptive weight generation strategy is introduced to mitigate \textit{task interference} by generating task-specific weight parameters for each task, thereby eliminating potential gradient conflicts on shared weight parameters. Second, a task-adaptive loss balancing strategy is introduced to dynamically adjust loss weights based on task-specific learning difficulties, preventing task domination or undertraining. Extensive experiments demonstrate that our proposed TAT achieves state-of-the-art performance in three MedIR tasks—PET synthesis, CT denoising, and MRI super-resolution—both in task-specific and All-in-One settings. Code is available at \href{https://github.com/Yaziwel/TAT}{this https URL}.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2025/paper/1155_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

N/A

Link to the Dataset(s)

N/A

BibTex

@InProceedings{YanZhi_TAT_MICCAI2025,
        author = { Yang, Zhiwen and Zhang, Jiaju and Yi, Yang and Liang, Jian and Wei, Bingzheng and Xu, Yan},
        title = { { TAT: Task-Adaptive Transformer for All-in-One Medical Image Restoration } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2025},
        year = {2025},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15975},
        month = {September},
        page = {572 -- 582}
}

Reviews

Review #1

  • Please describe the contribution of the paper
    1. The proposed weight generation and loss balancing strategies effectively mitigate inter-task conflicts and imbalance, advancing All-in-One MedIR.
    2. TAT outperforms SOTA methods, including recent All-in-One approaches like AMIR and MambaIR, while maintaining competitive performance in task-specific settings.
  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The ablation studies validate the necessity of each component (e.g., stop-gradient operation, depth-wise convolutions). ​
    2. The All-in-One framework is clinically relevant, as it consolidates multiple restoration tasks into a single model, reducing deployment complexity.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    1. The experiments are confined to three tasks (PET, CT, MRI). Recent works (e.g., DenoMamba [11], MambaIR [23]) suggest that modern architectures (e.g., state-space models) may offer broader applicability across modalities. The paper should discuss how TAT compares to or integrates with such trends. ​2. Computational Cost: While depth-wise convolutions are efficient, the total model size (41.62M parameters) is larger than some alternatives (e.g., MambaIR). A deeper analysis of runtime/memory trade-offs would strengthen the paper.
    2. The loss balancing strategy is empirically effective but lacks theoretical grounding compared to uncertainty-based multi-task learning [20].
    3. This kind of mixed training cannot truly achieve the all-in-one effect in essence, because it is impossible for you to exhaust all recovery tasks.
    4. From the results in Table 1 and Table 2, we can see that the results of the proposed method using dedicated training and mixed training are almost the same, which is quite worrying, and the authenticity of the results needs further verification.
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — could be rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    While the proposed method achieves some degree of restoration across multiple different tasks, it essentially lacks practical applicability.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    N/A

  • [Post rebuttal] Please justify your final decision from above.

    N/A



Review #2

  • Please describe the contribution of the paper

    This work introduces a task-adaptive transformer for all-in-one image restoration, addressing the task interference with task-adaptive weight generation and task imbalance via task-adaptive loss balancing strategy.

    The “task-adaptive weight generation” injects task-specific representations into convolution weights via lightweight MLPs, while the “task-adaptive loss balancing” is based on a learnable function of reconstruction error statistics.

    Experimental results show that the proposed model achieves strong performance across PET synthesis, CT denoising, and MRI super-resolution, both as individual tasks and in a unified setting.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The motivation is clear.
    • The paper is well-written and easy to follow.
    • Weight adaptation via task representation is lightweight and avoids full parameter duplication.
    • The idea of sample-level loss weighting is reasonable, especially when each task (e.g., CT denoising) can be further divided into multiple levels.
    • The t-SNE visualization is a nice touch to support task separation.
    • Extensive experimental results demonstrate the model’s superiority over other all-in-one models as well as task-specific models.
  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.
    • Will TAT model achieve even stronger performance if it is trained in a task-specific fashion?
    • PSNR and RMSE are largely two equivalent metrics. Also, they might not be the best choice for evaluating medical image reconstruction algorithms and correlate less with human readers [1-3]. This work can be further enhanced using metrics more closely related to perceptual or diagnostic quality.
    • The model is trained on patches, which typically introduces block artifacts in full-resolution inference and requires postprocessing [4]. What postprocessing, if any, was used when applying the trained TAT model for full-resolution inference?
    • While the sample-level loss weighting makes sense to me, its empirical gains (Table 3) appear modest, compared to other configurations. I am also curious whether “task imbalance” is really an issue we should care about. Note that a harder task naturally requires the model to “pay more attention to” through more effective gradient updates, which is reflected in its greater loss function value and might not need explicit reweighting in some cases.

    [1] Image quality in CT: From physical measurements to model observers.

    [2] Applicability evaluation of full-reference image quality assessment methods for computed tomography images.

    [3] Comparison of objective image quality metrics to expert radiologists’ scoring of diagnostic quality of MR images.

    [4] DiffLight: Integrating Content and Detail for Low-light Image Enhancement.

    [5] PromptIR: Prompting for All-in-One Blind Image Restoration.

    [6] Prompted Contextual Transformer for Incomplete-View CT Reconstruction.

    [7] All-In-One Medical Image Restoration via Task-Adaptive Routing.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html
    • It will benefit the readers if the paper includes a discussion on related works on all-in-one medical image restoration [5-7]. For example, Ref. [6] introduces task-specific prompts to enable universal restoration in CT images and also involves a task-specific loss weighting to avoid dominance of specific tasks. For another example, what are the advantages of injecting conditions as weights of network layers (as in this work) over modulating features (as in [5-7]), both of which can operate on the network features?
    • While the proposed TAT model demonstrates impressive performance, its clinical applicability raises important considerations. In real-world workflows, PET, CT, and MRI are acquired independently with distinct hardware and protocols, and image postprocessing is typically modality-specific. Given this context: How do the authors envision the deployment of an all-in-one model like TAT in real clinical workflows? Are there practical scenarios where TAT offers tangible benefits over modular pipelines? A more grounded discussion on clinical deployment would enhance the paper’s impact and relevance.

    [1] Image quality in CT: From physical measurements to model observers.

    [2] Applicability evaluation of full-reference image quality assessment methods for computed tomography images.

    [3] Comparison of objective image quality metrics to expert radiologists’ scoring of diagnostic quality of MR images.

    [4] DiffLight: Integrating Content and Detail for Low-light Image Enhancement.

    [5] PromptIR: Prompting for All-in-One Blind Image Restoration.

    [6] Prompted Contextual Transformer for Incomplete-View CT Reconstruction.

    [7] All-In-One Medical Image Restoration via Task-Adaptive Routing.

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — could be accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The manuscript is well-written and technically solid. However, minor revisions are needed for broader discussion of related works, architectural novelty, and clinical relevance, as noted in the “Weaknesses” and “Additional Comments”.

  • Reviewer confidence

    Very confident (4)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The rebuttal addresses most of the concerns, and the authors promise that the related discussion will be included in the revised manuscript. Therefore, I recommend accepting this paper.



Review #3

  • Please describe the contribution of the paper

    This paper aims to address the issues of limited generalization, low efficiency, and data scarcity in all-in-one medical image restoration tasks. The proposed framework builds upon the AMIR architecture, introducing a task-adaptive weight generation strategy to mitigate task interference and a task-adaptive loss balancing approach to address task imbalance. The all-in-one model achieves superior performance on PET synthesis, CT denoising, and MRI super-resolution tasks, surpassing both task-specific and other all-in-one methods.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper structure is clear, and the methodological descriptions and analyses are explicit and comprehensible.

    The proposed method cleverly employs a stop gradient approach to alleviate potential gradient conflicts in multi-task weight sharing.

    Dynamic loss weighting effectively balances performance across different tasks within the same model.

    The authors present extensive ablation studies on PET, MRI, and CT modalities, convincingly demonstrating the effectiveness and rationale of their proposed method.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The proposed model’s innovation is somewhat limited, exhibiting significant overlap with the AMIR framework. However, this is understandable, as the current work specifically addresses previously identified issues of task interference and imbalance by introducing the WATB and TREN modules into the original AMIR architecture.

    While the comparison methods include task-specific and all-in-one approaches employing CNN, GAN, Transformer, and Mamba-based techniques, diffusion model-based methods are notably absent. Given the recent state-of-the-art performance achieved by diffusion models in tasks such as PET synthesis, their inclusion as comparative methods would have strengthened the study.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The authors claimed to release the source code and/or dataset upon acceptance of the submission.

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html

    Authors are advised to verify the correctness of the PET image dimensions reported in Section 3.1 (Dataset).

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    All-in-one medical image restoration is a crucial topic. The proposed method effectively addresses the shortcomings of previous approaches. The experimental results are robust and thoroughly presented.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    I uphold my initial recommendation for acceptance.



Author Feedback

We sincerely thank the reviewers for their constructive comments. Our responses are as follows: Q1(R1): Comparison and integration with modern architectures A1: #Comparison: Modern architectures like Transformers and SSMs serve as general backbones and work well across various individual tasks. In contrast, TAT focuses more on all-in-one settings that use a single model to handle multiple different tasks. #Integration: TAT can be integrated into and benefit from these modern architectures. In fact, TAT is built on the Transformer-based Restormer by introducing two plug-and-play strategies: weight generation and task balancing. These two strategies are architecture-agnostic and can be easily incorporated into other modern architectures like SSMs, potentially enhancing their performance in all-in-one settings. We will include these discussions in our revision

Q2(R1): Computational cost A2: The number of params(M), runtime(s), and FLOPs(G) on a 128×128 image are: ARGAN(31.14M/0.32s/8.48G), DenoMamba(112.62M/0.44s/110.24G), MambaIR(31.50M/0.39s/34.35G), AirNet(7.61M/0.82s/230.48G), AMIR(23.54M/0.54s/31.76G) and TAT(41.62M/0.38s/38.00G). TAT performs the best with a balanced computational cost

Q3(R1): Lack of theoretical grounding for task balancing A3: Our sample‑level task‑balancing can be viewed as a special case of the original method in [20] if we treat the restoration of each image as an independent task. In [20], a parameter σ is assigned to estimate uncertainty of each task; likewise, our approach assigns a σ to estimate uncertainty of every image‑based task. The key distinction is that we derive σ from the observed loss values, whereas [20] treats σ itself as a learnable parameter. We will include this discussion in our revision and plan to develop a formal proof in future work

Q4(R1&R2): Limited practical applicability A4: The current method is applicable to restoration tasks in multimodal PET/CT and PET/MRI imaging, where multiple tasks coexist. We acknowledge that the mixed training paradigm has limitations and further exploration is needed to achieve a truly all‑in‑one solution for all restoration tasks

Q5(R1&R2): All-in-one vs. task-specific models A5: In general, task-specific models tend to outperform all‑in‑one models because they are specialized for individual tasks, whereas all‑in‑one models suffer from task interference (as demonstrated by MambaIR and DenoMamba in Tab 1&2). However, an all‑in‑one model such as TAT can match or even exceed the performance of task‑specific ones because it effectively captures inter‑task relationships and benefits from both larger training datasets and shared knowledge across tasks. Similar findings have been observed in AMIR and AirNet

Q6(R1): Reproducibility A6: Code and data will be publicly available

Q7(R2): More evaluation A7: We follow [2] to measure the VIF on CT image. TAT(0.5514) outperforms the second-best AMIR(0.5495). We will properly cite [1-3]

Q8(R2): Window artifact A8: We follow AMIR and MambaIR to directly take the full-resolution image as input and thus there are no window artifacts

Q9(R2): Reasonableness of task balancing A9: We agree that vanilla loss tends to prioritize tasks with large losses. But uncertainty-based task balancing assumes that larger losses correspond to higher uncertainty—which can lead to misleading update directions—and thus assigns them lower weights, and vice versa. The performance of task balancing is significant (p<0.05) in Tab 3

Q10(R2): Discussion on related works A10: We will cite and discuss [5-7] follow your advice. Modulating on weight and features are two different approaches but Instruct-IPT (by Tian et al.) proves that weight modulation is more effective than feature modulation for handling very distinct tasks

Q11(R3): Comparison with diffusion-based model A11: As no experiment is allowed, we will discuss diffusion-based models like DDPM-PET and uPETe in our revision

Q12(R3): PET image shape A12: It should be 400×192



Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • If your recommendation is “Provisional Reject”, then summarize the factors that went into this decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. You do not need to provide a justification for a recommendation of “Provisional Accept” or “Invite for Rebuttal”.

    N/A

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    N/A



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    N/A



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    This work has two positive reviewers and a negative reviewer. By checking the comments and rebuttals, I think that this work has the merits, and the rebuttal also addressed most of concerns of reviewers. Hence, I recommend to accept this work.



back to top