List of Papers Browse by Subject Areas Author List
Abstract
Reducing scan time in Positron Emission Tomography (PET) imaging while maintaining high-quality images is crucial for minimizing patient discomfort and radiation exposure. Due to the limited size of datasets and distribution discrepancy across scanners in medical imaging, fine-tuning in a parameter-efficient and effective manner is on the rise. Motivated by the potential of Parameter Efficient Fine-Tuning (PEFT), we aim to address these issues by effectively leveraging PEFT to improve limited data and GPU resource issues in multi-scanner setups. In this paper, we introduce PETITE, Parameter Efficient Fine-Tuning for MultI-scanner PET to PET REconstruction, which represents the optimal PEFT combination when independently applying encoder-decoder components to each model architecture. To the best of our knowledge, this study is the first to systematically explore the efficacy of diverse PEFT techniques in medical imaging reconstruction tasks via prevalent encoder-decoder models. This investigation, in particular, brings intriguing insights into PETITE as we show further improvements by treating the encoder and decoder separately and mixing different PEFT methods, namely, Mix-PEFT. Using multi-scanner PET datasets comprised of five different scanners, we extensively test the cross-scanner PET scan time reduction performances (i.e., a model pre-trained on one scanner is fine-tuned on a different scanner) of 21 feasible Mix-PEFT combinations to derive optimal PETITE. We show that training with less than 1% parameters using PETITE performs on par with full fine-tuning (i.e., 100% parameter). Code is available at: https://github.com/MICV-yonsei/PETITE
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/0560_paper.pdf
SharedIt Link: https://rdcu.be/dV5Er
SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72104-5_50
Supplementary Material: https://papers.miccai.org/miccai-2024/supp/0560_supp.pdf
Link to the Code Repository
https://github.com/MICV-yonsei/PETITE
Link to the Dataset(s)
BibTex
@InProceedings{Kim_Parameter_MICCAI2024,
author = { Kim, Yumin and Choi, Gayoon and Hwang, Seong Jae},
title = { { Parameter Efficient Fine Tuning for Multi-scanner PET to PET Reconstruction } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15007},
month = {October},
page = {518 -- 528}
}
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes PETITE, a parameter-efficient fine-tuning approach for multi-scanner PET reconstruction to reduce scan time while maintaining image quality. PETITE use information from majority scanners to improve performance on minorities with limited data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper addresses an important problem of reducing PET scan time while maintaining high quality images, which has practical clinical value. It is interesting to apply PEFT on PET reconstruction tasks. Comprehensive experiments on multi-scanner PET datasets demonstrate the efficacy of the proposed PETITE approach, showing it can achieve similar results as full fine-tuning with less than 1% parameters.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The authors did not compare their methods with enough PET-specific reconstruction solutions.
- Only one dataset cannot prove the universality of method.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Please see Point 6.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, this paper presents solid novelty to use foundation models and prompt techniques for PET reconstruction, which is an interesting application. However, some more experiments are expected to further prove the effectiveness of the proposed methods. The rate will be adjusted after the rebuttal.
- Reviewer confidence
Very confident (4)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Reject — could be rejected, dependent on rebuttal (3)
- [Post rebuttal] Please justify your decision
The lack of experiments is my major concern for this submission. Based on the MICCAI rebuttal clarification: “Note that the lack of certain experiments or missing comparisons can be valid concerns listed under the paper’s weaknesses and may have informed the reviewers’ scores.” I would keep my rating unchanged.
Review #2
- Please describe the contribution of the paper
This paper introduces a mixture of parameter-efficient finetuning techniques to handle PET reconstruction under limited data. The proposed method approaches the full-fintuning method. Please see the detailed comments below.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This paper conducts extensive experiments on parameter-efficient finetuning approaches in PET reconstruction.
- This paper is easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The sentence seems to be repeated here in the SSF section. ‘SSF module consists of the scale factor γ, dot product with x, and the shift factor β, added to x. SSF module consists of the scale factor γ with the dot product ⊙ and the shift factor β.
- The authors should carefully check the Tables. For example, in Table 2 (a), the second best results on PSNR is not highlighted with underline, and two numbers are underlined regarding NRMSE.
- The authors validated a series of choices of different PEFT approaches in both encoders and decoders in the main draft and supplementary. However, further concluding analysis are needed for the proposed mixture of PEFT techiniques. E.g. which PEFT method is more suitable for ViT/CNN, encoder/decoder and why?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
Please refer to the strength and weakness.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, I think this paper makes several experimental contributions, while the paper needs further revise.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Weak Accept — could be accepted, dependent on rebuttal (4)
- [Post rebuttal] Please justify your decision
I would keep my initial score
Review #3
- Please describe the contribution of the paper
In this paper, the authors propose PETITE, aiming to address the challenge of reducing scan time in PET imaging while maintaining high-quality images. They leverage Parameter-Efficient Fine-Tuning (PEFT) techniques to improve limited data and GPU resource issues in multi-scanner setups. PETITE utilizes fewer than 1% of the parameters and systematically explores the efficacy of diverse PEFT techniques in medical imaging reconstruction tasks using prevalent encoder-decoder-type deep models. Their investigation, including the treatment of encoder and decoder separately and the mixing of different PEFT methods (Mix-PEFT), yields intriguing insights and demonstrates significant improvements.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
A) This paper offers a new insight into multi-scanner PET reconstruction by applying currently popular Parameter-Efficient Fine-Tuning techniques (which are mostly applied in large models), avoiding the redundant and time-consuming training process on new scanner datasets.
B) The motivation of the paper is acceptable, addressing the problem of reconstruction model training on multi-scanner data, which aligns well with clinical practice.
C) The authors extensively explored various combinations of PEFT based on two reconstruction backbones through experiments, demonstrating the effectiveness of fine-tuning techniques in the reconstruction model. They also identified the best combination of PEFT through this process.
D) The paper provides a clear overview of the methodology and is easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
A) Although transferring fine-tuning techniques from large models to PET reconstruction tasks is indeed a notable achievement, this paper only explores combinations of existing tuning methods. Maybe tuning techniques tailored for medical imaging models can be developed in the future.
B) There are some questions I would like to ask in the experimental section of the paper. a) My biggest concern is that how do the authors construct low-count PET from the public ADNI data? Maybe the authors can supplement this in the supplementary materials or later on the git-hub. b) I’m curious about the details of how the authors use 30 samples for pre-training and 10 samples for fine-tuning. The current description of data preprocessing is confusing.
C) The presentation of the experiments requires improvement. The experimental tables lack information about multiple scanners. Additionally, if the authors pre-train using one scanner and fine-tune on data from other scanners, this setting should be described in detail.
D) The abbreviation “PETITE” may cause confusion. It might be helpful to consider using a more straightforward alternative to improve clarity for readers.
E) Fig. 4 seems inconsistent with the experimental results table. Although the quantitative results indicate that the full-FT method outperforms the parameter-efficient fine-tuning in terms of PSNR and NMSE values, the error map analysis shows smaller errors for the proposed method. It would also be beneficial to showcase the results of other combinations of fine-tuning modules.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission has provided an anonymized link to the source code, dataset, or any other dependencies.
- Do you have any additional comments regarding the paper’s reproducibility?
The code or instruction of low-count PET data construction from ADNI dataset can be shared for better reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
My comments are mainly based on the “weaknesses” part: A) To enhance the effectiveness and innovation of the proposed method, consider exploring novel fine-tuning techniques that are suitable for the frameworks of medical imaging.
B) In the “Datasets and Image Preprocessing” section, the methodology for constructing low-count PET from the ADNI data lacks clarity. I hope the authors can answer my questions in “weaknesses” part and supplement this in the supplementary materials or later on the git-hub. . C) The work investigates only two reconstruction backbones: 3D CVT-GAN and UNETR. To enhance the generalizability of the discussed fine-tuning techniques, experiments on a broader range of network architectures would be beneficial.
D) The abbreviation “PETITE” appears to be confusing. Consider using a simpler alternative.
E) For better alignment between Fig. 4 and the experimental results table, consider providing a more comprehensive analysis. Additionally, showcasing results from other combinations of finetuning modules could offer a more rounded evaluation.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The authors’ motivation for proposing PETITE to address the challenge of reducing scan time in PET imaging while maintaining high-quality images in a multi-scanner scenario seems reasonable. Their initial attempt to transfer fine-tuning techniques from large models to PET reconstruction tasks is commendable and holds practical significance in clinical practice. There are shortcomings in the experimental presentation. It would be great if the authors can address my concerns and reply to my questions.
Overall, this is a good paper, and I would like to give it a rating of “5 - Accept”. - Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Accept — should be accepted, independent of rebuttal (5)
- [Post rebuttal] Please justify your decision
Considering the submitted rebuttal and other papers in my stack, i would like to insist my score.
Author Feedback
We thank all the reviewers for their thoughtful feedback & constructive criticisms. We address all the comments below:
R1) Tuning methods specific to medical imaging Thank you for your feedback. We realized it was crucial to first empirically validate the efficacy of the existing PEFT to multi-scanner as rigorously as possible via exhaustive experiments involving numerous En-De and PEFT combinations. As suggested, we plan to develop PEFT methods tailored for medical imaging.
R1) How to get low-count PET? Each ADNI PET is a sequence of six 5-minute frame scans (i.e., 0-5, …, 25-30). The low-count (i.e., short-time) is the first 5-minute scan (0-5) only. We will clarify this and release the data preprocessing code.
R1) Sample split details We apologize for any confusion caused by the brief explanation. The sample split is Pretrain (Source): Train 30 / Valid 15 and PEFT (Target): Train 10 / Valid 15. Data setup details (e.g., split, 3-fold CV, etc.) will be clarified.
R1) Details on multi-scanner and data splits We see the need for details which we clarify below.
- We use datasets from 5 scanners: S1, S2, S3, S4, S5
- S1 as Source: Pretrain on S1 and finetune on the other 4 scanners.
- Repeat for each scanner (S2, S3, S4, S5) as Source and finetune on the others. Thus, each source scanner finetunes 4 times, resulting in 20 possible finetune results across 5 scanners. Due to the page limit, we had to show the average results instead of showing all combinations as 5x4 tables. We will clarify this further.
R1) Abbreviation PETITE refers to the best PEFT setup for PET recon among various Mix-PEFTs for En-De. We will consider a simpler alternative.
R1) Full-FT in Table 1 is not similar to Fig. 4 We appreciate your point. Fig. 4 shows an example from a specific slice, sample, and scanner, which may not best align with Table 1, which shows the average of all scanners. We will include a better-aligned sample. Due to the sheer number of PEFT combinations, we regretfully could not include all visual results. Our submitted code will be updated to provide all error map generations.
R1&R3) Availability of PET-specific models While more models would always be better, we hope the reviewers see the rationale behind our choice. To gain experimentally sound insights on this first work, we prioritized exhaustive and rigorous experiments on two models that best met the following criteria: 1) robustness & high performance 2) reproducibility (open-source) 3) compatibility with ViT-based PEFT Based on these criteria, we chose (A) 3D CVT-GAN, the only SOTA PET recon model with reproducible code, and (B) UNETR, one of the most widely used 3D transformer-based models with UNET-like robustness. As the first step, we believe these two models effectively span the range of possible PET recon solutions, although we agree more models could better fill the gap. As suggested, we plan to explore various PET-specific models in future work.
R3) “Only one dataset cannot prove the universality of method” We do agree that universality is indeed important, but proving it is a long-term pursuit, especially in a field with rapid methodological advances. Therefore, like other initial attempts, we aimed to validate the method in a controlled environment, providing insights through exhaustive experiments within the PET modality. Nonetheless, we fully agree that the universality suggested is a crucial direction for future work, and we will pursue it.
R4) Details on PEFT suitability We apologize for the brief explanation due to space limitations. Based on the tables (See Table 1; Suppl. Tables 5 and 6), VPT and LoRA are most suitable for ViT in both En-De. Using them together creates a synergistic effect, as both operate within the critical attention component of ViT. Additionally, SSF is best for CNNs. Tuning SSF in ViT with other PEFT methods results in a performance drop, as it hinders ViT’s complex attention mechanism. We will clarify this further.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
This paper proposed to leverage Parameter-Efficient Fine-Tuning (PEFT) to deal with the challenges of limited data and GPU resources in reducing scan time for PET imaging. This is an interesting and timely attempt in this field, ac acknowledged by all reviewers. My feeling to the experimental validation is mixed. The proposed method was validated on two backbone models to show its effectiveness, but was not widely compared with other PET image synthesis methods. However, as the authors pointed out, the current experimental results on the two distinct backbones have verified the rationale of the proposed strategy. Evidently, it has the potential to be extended to other PET imaging synthesis backbones effectively. Considering this is a timely effort to borrow techniques from the foundation models to help PET reconstruction, I tend to accept and believe it could be of interest to researchers in the similar field.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
This paper proposed to leverage Parameter-Efficient Fine-Tuning (PEFT) to deal with the challenges of limited data and GPU resources in reducing scan time for PET imaging. This is an interesting and timely attempt in this field, ac acknowledged by all reviewers. My feeling to the experimental validation is mixed. The proposed method was validated on two backbone models to show its effectiveness, but was not widely compared with other PET image synthesis methods. However, as the authors pointed out, the current experimental results on the two distinct backbones have verified the rationale of the proposed strategy. Evidently, it has the potential to be extended to other PET imaging synthesis backbones effectively. Considering this is a timely effort to borrow techniques from the foundation models to help PET reconstruction, I tend to accept and believe it could be of interest to researchers in the similar field.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
I agree with the reviewers that the paper has made novel contributions, and I recommend accept provided that the authors will make the revisions as promised.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
I agree with the reviewers that the paper has made novel contributions, and I recommend accept provided that the authors will make the revisions as promised.