Paper Info Reviews Author Feedback Meta-Review Back to top

List of Papers Browse by Subject Areas Author List

Abstract

Knowledge transfer from a source to a target domain is vital for whole slide image classification, given the limited dataset size due to high annotation costs. However, domain shift and task discrepancy between datasets can impede this process. To address these issues, we propose a Target-Aware Knowledge Transfer framework using a teacher-student paradigm, enabling a teacher model to learn common knowledge from both domains by actively incorporating unlabelled target images into the teacher model training. The teacher bag features are subsequently adapted to supervise the student model training on the target domain. Despite incorporating the target features during training, the teacher model tends to neglect them under inherent domain shift and task discrepancy. To alleviate this, we introduce a target-aware feature alignment module to establish a transferable latent relationship between the source and target features by solving an optimal transport problem. Experimental results show that models employing knowledge transfer outperform those trained from scratch, and our method achieves state-of-the-art performance among other knowledge transfer methods on various datasets, including TCGA-RCC, TCGA-NSCLC, and Camelyon16. Codes are released at https://github.com/BearCleverProud/TAKT.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/2121_paper.pdf

SharedIt Link: https://rdcu.be/dY6i0

SpringerLink (DOI): https://doi.org/10.1007/978-3-031-72083-3_47

Supplementary Material: https://papers.miccai.org/miccai-2024/supp/2121_supp.pdf

Link to the Code Repository

https://github.com/BearCleverProud/TAKT

Link to the Dataset(s)

N/A

BibTex

@InProceedings{Xio_TAKT_MICCAI2024,
        author = { Xiong, Conghao and Lin, Yi and Chen, Hao and Zheng, Hao and Wei, Dong and Zheng, Yefeng and Sung, Joseph J. Y. and King, Irwin},
        title = { { TAKT: Target-Aware Knowledge Transfer for Whole Slide Image Classification } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15004},
        month = {October},
        page = {503 -- 513}
}

Reviews

Review #1

Please describe the contribution of the paper

The paper proposes a target-aware knowledge transfer framework for whole slide image classification. The framework alleviates the impact of a domain shift between the teacher and student domain while mitigating bias towards the source domain.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper appears to have a good structure and adequate experiments. The experiment results also show the advantages of their methods over other existing work.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The figure remains a little bit confusing. As I understand it, first, train a teacher model using target information/centroid, etc., and second, train the student-teacher framework. However, the figure looks like the teacher and student models are trained simultaneously.

In this paper, the sole purpose of training a teacher model is to benefit a student model. Therefore, the teacher model is not reused for other domains; the application of those teacher models is limited to only one target domain.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The authors claimed to release the source code and/or dataset upon acceptance of the submission.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

Please define your teacher model and student model. I assume your teacher and student model are an MHA layer with 8 heads?

Please define:

A(.), pt, and ps in the figure
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Reject — could be rejected, dependent on rebuttal (3)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Their work proposed a method to tranfer knowledge from one dataset to another in WSI classification, which is an important topic. The experiments are adequate. However, the application is limited in trained a teacher and a student solely benefit on target dataset.
Reviewer confidence

Confident but not absolutely certain (3)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

Weak Accept — could be accepted, dependent on rebuttal (4)
[Post rebuttal] Please justify your decision

Regarding the experiments: NSCLC → Camelyon16 RCC → Camelyon16 RCC → NSCLC NSCLC → RCC It is still unclear to me whether we need to train four pairs of teacher-student models, or two teachers (NSCLC and RCC) plus four students.

However, I believe the paper still has some merit, so I rate it as Weak Accept.

Review #2

Please describe the contribution of the paper

The paper introduces a novel Target-Aware Knowledge Transfer framework for whole slide image classification that employs a teacher-student paradigm to overcome challenges associated with domain shift and task discrepancy. This framework is designed to enhance the transfer of knowledge between different datasets, which is vital given the limited number of samples often available due to high annotation costs. The teacher model is trained using both labeled source domain images and unlabeled target domain images, which helps it to acquire common knowledge applicable across domains. To ensure effective knowledge transfer, a target-aware feature alignment module is introduced that uses optimal transport theory to align features between the source and target domains.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The research direction of this paper is significant, as many clinical WSI classification datasets are small, necessitating the introduction of diverse datasets for knowledge transfer and enhancement. The problem description and logic of the paper are clearly articulated.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The paper significantly lacks innovation and does not provide a detailed summary of existing methods or how its approach differs from them.
-1) The method described in this paper is very similar to that in reference [1], which first uses similarity-based retrieval and augmentation to address domain shift issues between the target and source domains, followed by establishing latent connections through Optimal Transport. The paper does not mention a close comparison with reference [1]. Please specify how the method in this paper differs from that.

-2) As reviewed by me and stated in reference [1], there has been substantial related work in knowledge transfer for WSI classification. This paper does not provide a detailed summary of these existing methods or clarify how its approach differs from them.
1. Regarding the effectiveness of adding source domain data after retrieval, given that the tasks in the target and source domains might be different, how can one ensure that the data added post-retrieval does not disrupt the source domain? If the optimization directions of the two tasks differ under different task optimizations, what is the rationale behind the effectiveness of simple feature retrieval?
2. What contributions does this paper make regarding adaptation to different tasks? Task adaptation is more challenging than domain adaptation, yet the main contributions of this paper seem to focus on adapting to different domains.
3. Clustering is not stable; is re-clustering performed every epoch or fixed? Moreover, how can it be ensured that each clustering result is effective?
4. In terms of performance, please compare this method against closely related existing methods, particularly reference [1], which must be included in the comparison.
[1] Cui, Y., Liu, Z., Chen, Y., Lu, Y., Yu, X., Liu, X. S., … & Chan, A. (2024). Retrieval-Augmented Multiple Instance Learning. Advances in Neural Information Processing Systems, 36.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The authors claimed to release the source code and/or dataset upon acceptance of the submission.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

Please carefully address the weaknesses.
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Reject — could be rejected, dependent on rebuttal (3)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Please carefully address the weaknesses.
Reviewer confidence

Very confident (4)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

Reject — should be rejected, independent of rebuttal (2)
[Post rebuttal] Please justify your decision

I carefully reviewed the author’s rebuttal, but unfortunately, my final assessment remains a rejection. My primary concern continues to be the novelty of the work. As I mentioned in my initial review, the method described in this paper is very similar to that in reference [1], which initially employs similarity-based retrieval and augmentation to address domain shift issues between the target and source domains, followed by establishing latent connections through Optimal Transport. The rebuttal from the authors mentioned only minor differences or applications in different scenarios, but the innovation in technical methodology is evidently insufficient. Additionally, the transfer for entirely different tasks lacks more convincing explanations or experiments. The authors’ response did not fully resolve my doubts.

[1] Cui, Y., Liu, Z., Chen, Y., Lu, Y., Yu, X., Liu, X. S., … & Chan, A. (2024). Retrieval-Augmented Multiple Instance Learning. Advances in Neural Information Processing Systems, 36.

Review #3

Please describe the contribution of the paper

The author focused on the knowledge transfer method on whole slide image classification between the different domains.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper is well-written and well-structured. The tecnical novelty (Target-Aware Data Augmentation and Target-Aware Feature Alignment Module) seems to be clear.
The experiment results demonstrated the effectiveness of their proposed method.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Some explanation is not sufficient to understarnd for readers. In 2.3, the author mentioned “The bag features from the teacher model only using F_S and C_T are denoted as h_S and h_T, respectively.” I’m not sure how does the author divide the output of M_t() into h_S and h_T. Augmnented WSI bags consits of source feature and target centroid and it seems to be difficult to devide the bag feature into h_S and h_T. Please should be clear.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The authors claimed to release the source code and/or dataset upon acceptance of the submission.
Do you have any additional comments regarding the paper’s reproducibility?

N/A
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

The indices which the author used are a bit confusing. s->student, t->tearcher, S->source, T->target. I recommend the author to review the indices in the paper. In Fig. 1(a), which column indicates the source or the target domain?
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Weak Accept — could be accepted, dependent on rebuttal (4)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The technical novelty is clear and the experiment demonstrated the effectiveness, so I set the WA.
Reviewer confidence

Somewhat confident (2)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

Weak Accept — could be accepted, dependent on rebuttal (4)
[Post rebuttal] Please justify your decision

My concern was clear so I set the rating as WA.

Review #4

Please describe the contribution of the paper

The paper introduces a framework for enabling knowledge transfer in Whole Slide image classification between different domains. With a very clear structure, the general concept and its math are introduced. The evaluation is extensive and completed by an ablation study. The proposed method proves significantly superior to current knowledge transfer methods for Whole Slide Image Classification.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper’s main strength is its novel approach, which is supported by an excellent structure and strong evaluation. Using a teacher-student approach, a teacher model is trained with some features form the target domain and then provided to the student model during the training on the target dataset. Using a “Target-Aware Feature Alignment” the importance of such features from the target domain during the teacher models training was increased to provide meaningful improvements.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

There are no major weaknesses (if the code is realy provided at acceptance).
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

The authors claimed to release the source code and/or dataset upon acceptance of the submission.
Do you have any additional comments regarding the paper’s reproducibility?

The paper is completely based on public datasets. It is claimed that code will be provided upon acceptance.

It is not clear how the used datasets are labeled. Please consider adding a short description for each dataset.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html

Power Temperature Scaling is missing its abbreviation at the end of section 2.1.

Figure 1(b) is a bit confusing. “Append” looks like adding the Centroids tile from the target domain directly to the tile of the source domain. Maybe consider a different depiction technique. This is not that important, as it is described quite clearly in the text, however, it was confusing at first sight. Training settings in Section 3.2 are unclear: “The student model in our method is initialised with the teacher model. The model is trained up to 200 epochs and no less than 50 epochs.”. Which model (teacher/student) is meant in the second sentence
Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making

Strong Accept — must be accepted due to excellence (6)
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The quality and clarity of the work are excellent. The evaluation and results of the presented framework are great (maybe even too great, which can only be determined by independent researchers using it). If the provided code holds up to the standard of this paper, it will be a great addition to the scientific community.
Reviewer confidence

Very confident (4)
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

Strong Accept — must be accepted due to excellence (6)
[Post rebuttal] Please justify your decision

As I stated before, the work is excellent (if the code is provided and the results of the work are actual reproducible)

Author Feedback

We express our sincere gratitude to the reviewers for their valuable time and effort in evaluating our paper. We greatly appreciate the insightful and overall positive feedback. We are particularly pleased to note that the reviewers have recognized our manuscript’s clarity (R1, R3, R4, R5), significance of the topic (R3, R4), clear technical novelty (R1, R5), and efficacy of our method (R1, R3, R5). Below, we provide our responses to the questions.

Subscripts and differentiation of augmented and original features (R1) We consistently use the notation ‘s’ and ‘t’ for the student and teacher models, and ‘S’ and ‘T’ in calligraphic font for the source and target domain related concepts, respectively. To differentiate the augmented and original features, we record the IDs of augmented features during augmentation. This allows us to index and distinguish between them and feed them into M_t() separately to obtain h_S and h_T. More details will be provided in our code.

Figure and definition problems (R1, R3, R5) We acknowledge the need to improve the clarity of our figures and definitions. We will revise the figures to enhance readability. Also, we will ensure that definitions, such as those for the teacher (can be used for any WSI-related tasks) and student models (both are the CLAM model), the adaptation module A(), and the teacher and the student logits p_t and p_s, are precise. Furthermore, we will clearly show the training sequence of the two models.

Related works (R4) Due to the page limit of MICCAI, the related work section is omitted. However, we will integrate more related works into the introduction in the final version. The paper mentioned by the reviewer addresses a different problem, as it focuses on the model’s inability to adapt well to out-of-distribution domains. Instead of reusing the teacher model, our method aims to transfer knowledge from the teacher model trained on the source domain to enhance the student model on the target domain. Additionally, our usage of OT differs. While they use OT to retrieve the most similar bag (all patch features), our method uses OT to compute the distance between the teacher and student bag features (only one feature) as an additional loss. For results of RAM-MIL, we are strictly prohibited from including additional results. We will clarify the differences in the final version and compare with it.

Task adaptation (R4) As mentioned in our paper, Camelyon and TCGA datasets have distinct tasks, cancer detection and subtyping. To address domain shift and task discrepancy, we propose TADA and TAFA to incorporate and align the target data during teacher training. Moreover, we use the MHA adaptor to mitigate the issue of task discrepancy, as different tasks require different features.

Efficacy of adding target features (R4) The probability of augmenting each feature is low (30% in our experiments), which limits the number of target features in the source domain. Moreover, it is less likely that the optimization direction will change. If the source WSI is cancerous, the retrieved target features are expected to include tumor patches due to their similarity. Similarly, when the source domain WSI is normal, the retrieval of a tumor patch is less likely to occur. Therefore, this operation only introduces diversity and does not significantly disrupt the distribution.

Clustering (R4) Clustering aggregates numerous features from the target domain, with each centroid representing a group of features. The interpretability of these centroids is not necessary in a conventional sense; instead, they serve as a statistical summary of the features, enriching feature diversity. As long as the features are clustered, the effectiveness is achieved. Additionally, clustering is pre-computed once for each dataset. We believe that these responses and revisions can address the concerns of the reviewers and improve the quality of our manuscript. Thank you again for your valuable feedback.

Meta-Review

Meta-review #1

After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

Accept
Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

The paper is well written and offers a neat way of improving transfer learning. The performance gains are reasonable. One reviewer rejects the paper on the grounds of insufficient novelty but 2 reviewers comment on novel approach and 3/4 reviewers are in favour of acceptance
What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

The paper is well written and offers a neat way of improving transfer learning. The performance gains are reasonable. One reviewer rejects the paper on the grounds of insufficient novelty but 2 reviewers comment on novel approach and 3/4 reviewers are in favour of acceptance

Meta-review #2

After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

Accept
Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

The paper proposes an approach to improve knowledge transfer between tasks using a student-teacher paradigm with additional feature alignment.

This paper has received rather diverging reviews. The majority sees this paper above the acceptance threshold (weak / strong accept), given the application, the performance, and the evaluation. One reviewer is critical about the difference to existing approaches following a similar idea of combining student-teacher training and optimal transport.

Considering the rebuttal and the overall opinion, the approach seems to be have a sufficiently large contribution compared to related work (method / application). It therefore seems to me that benefits of discussing the results and proposed approach outweigh this. I would strongly encourage the authors to put their paper better into perspective especially with the work that R4 has mentioned.
What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

The paper proposes an approach to improve knowledge transfer between tasks using a student-teacher paradigm with additional feature alignment.

This paper has received rather diverging reviews. The majority sees this paper above the acceptance threshold (weak / strong accept), given the application, the performance, and the evaluation. One reviewer is critical about the difference to existing approaches following a similar idea of combining student-teacher training and optimal transport.

Considering the rebuttal and the overall opinion, the approach seems to be have a sufficiently large contribution compared to related work (method / application). It therefore seems to me that benefits of discussing the results and proposed approach outweigh this. I would strongly encourage the authors to put their paper better into perspective especially with the work that R4 has mentioned.

back to top

TAKT: Target-Aware Knowledge Transfer for Whole Slide Image Classification

Author(s):