Paper Info Reviews Author Feedback Meta-Review Back to top
List of Papers Browse by Subject Areas Author List

Abstract

Medical image retrieval is a valuable field for supporting clinical decision-making, yet current methods primarily support 2D images and require fully annotated queries, limiting clinical flexibility. To address this, we propose RadiomicsRetrieval, a 3D content-based retrieval framework bridging handcrafted radiomics descriptors with deep learning-based embeddings at the tumor level. Unlike existing 2D approaches, RadiomicsRetrieval fully exploits volumetric data to leverage richer spatial context in medical images. We employ a promptable segmentation model (e.g., SAM) to derive tumor-specific image embeddings, which are aligned with radiomics features extracted from the same tumor via contrastive learning. These representations are further enriched by anatomical positional embedding (APE). As a result, RadiomicsRetrieval enables flexible querying based on shape, location, or partial feature sets. Extensive experiments on both lung CT and brain MRI public datasets demonstrate that radiomics features significantly enhance retrieval specificity, while APE provides global anatomical context essential for location-based searches. Notably, our framework requires only minimal user prompts (e.g., a single point), minimizing segmentation overhead and supporting diverse clinical scenarios. The capability to query using either image embeddings or selected radiomics attributes highlights its adaptability, potentially benefiting diagnosis, treatment planning, and research on large-scale medical imaging repositories. Our code is available at https://github.com/nainye/RadiomicsRetrieval.

Links to Paper and Supplementary Materials

Main Paper (Open Access Version): https://papers.miccai.org/miccai-2025/paper/1682_paper.pdf

SharedIt Link: Not yet available

SpringerLink (DOI): Not yet available

Supplementary Material: Not Submitted

Link to the Code Repository

https://github.com/nainye/RadiomicsRetrieval

Link to the Dataset(s)

BraTS-GLI & BraTS-MEN: www.synapse.org/Synapse:syn51156910/wiki/622351 NSCLC-Radiomics: www.cancerimagingarchive.net/collection/nsclc-radiomics NSCLC-Radiomics-Interobserver1: www.cancerimagingarchive.net/collection/nsclc-radiomics-interobserver1 RIDER-LungCT-Seg: www.cancerimagingarchive.net/collection/rider-lung-ct NSCLC Radiogenomics: www.cancerimagingarchive.net/collection/nsclc-radiogenomics LUNG-PET-CT-Dx: www.cancerimagingarchive.net/collection/lung-pet-ct-dx

BibTex

@InProceedings{NaIny_RadiomicsRetrieval_MICCAI2025,
        author = { Na, Inye and Rue, Nejung and Chung, Jiwon and Park, Hyunjin},
        title = { { RadiomicsRetrieval: A Customizable Framework for Medical Image Retrieval Using Radiomics Features } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2025},
        year = {2025},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15960},
        month = {September},
        page = {559 -- 569}
}

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper introduces a content-based image retrieval framework that integrates handcrafted radiomics descriptors and deep learning-based embeddings at the tumor level in 3D medical images

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The strength of this paper lies in its central idea of using tumor-level features and embeddings to identify similarities between 3D medical images.”

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    This paper has the following weaknesses:

    1. Lack of detail on handcrafted features The paper does not specify which handcrafted features were used, making it difficult to assess the feasibility and reproducibility of the proposed approach.

    2. Insufficient explanation of radiomics features While radiomics features appear central to the methodology, the paper fails to define or describe them clearly.

    3. Unclear use of SAM models It is not evident how SAM models are incorporated into the framework. Figure 2(A) does not clarify their role either.

    4. Ambiguity around tumor-specific embeddings The concept of tumor-specific embeddings is not technically defined. It is unclear whether these are based on ROIs, and if so, how they are extracted or used.

    5. Lack of justification for FPE and unclear explanation of APE The rationale behind using the FPE model is not discussed, and the APE model is also not explained in enough detail to understand its importance.

    6. Unexplained terms and tools The paper includes terms such as TransTab, ANTsPy, and ICBM T1 without adequate explanation or context.

    7. Unclear methodology in Table 2 In Table 2, it is unclear how the contrastive unimodal approach was implemented or evaluated.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (3) Weak Reject — could be rejected, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Overall, the paper’s weaknesses are more significant than its strengths

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Reject

  • [Post rebuttal] Please justify your final decision from above.

    There are still several unclear aspects. For instance, it is not well explained what qualifies GLCM as a radiomic feature in contrast to SIFT. Additionally, the unimodal contrastive approach is not sufficiently detailed—specifically, the construction of negative pairs and how intra-similarities among images are handled, including anchor-to-positive and anchor-to-negative relationships.

    Given these and other unresolved issues, my previous review stands.



Review #2

  • Please describe the contribution of the paper

    The paper introduces RadiomicsRetrieval, a novel content-based image retrieval (CBIR) framework that bridges handcrafted radiomics features with deep learning-based embeddings (at the tumor level) using contrastive learning. Proposed framework supports full or partial radiomics feature-based queries, and image-based retrieval from point prompts, showcasing its value for real-world clinical practice.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1. The authors proposed an original and by itself interesting idea of combining radiomics features with learnable embeddings to improve CBIR.

    2. The authors also created a strong end product, flexible CBIR framework, which allows retrieval based on radiomics features, anatomical location (via APE), or minimal image prompts, enabling diverse use cases like shape-only or region-specific searches.

    3. The paper contains robust evaluation. Performance is analyzed across multiple query types and datasets (brain MRI and lung CT), with quantitative and qualitative evaluations clearly showing good performance scores (localization accuracy and radiomics features correlation).

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    1. Lack of formal definitions and reproducibility. The paper lacks formal definitions for key components of the proposed framework, particularly the contrastive training objective, which is central to the method. The architecture is described primarily through high-level schematics without accompanying mathematical detail or algorithmic descriptions. (The absence of publicly available code prevents verification of implementation details.) This lack of clarity may lead to misunderstandings, including potential discrepancies between training (contrastive alignment) and inference (retrieval) objectives. The authors should dedicate sufficient space to rigorously formalize at least the contrastive learning component. And it would be also beneficial to know, e.g., (a) how APE is downsamples and processed (b) why E_prompt has only one path to D_multi in Fig. 2A and then two paths in Fig. 2B, (c) how FPE replaces APE when E_radiomics wasn’t trained with FPE features, and other misc architectural questions?

    2. Inconsistency between claimed query flexibility and actual requirements. The paper claims support for flexible querying, including the ability to retrieve based solely on radiomics features (Claim 2). However, the radiomics embedding path (R) depends on Anatomical Positional Embeddings (APE) and is never trained without them (Fig. 3), which in turn require access to the original image. This implies that image data is always necessary at inference time, contradicting the claim of image-free radiomics-based queries. This inconsistency should be clarified or revised. Alternatively, generalization on removing APE should be ablated.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission does not provide sufficient information for reproducibility.

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html

    Use ``text’’ for nice quotation in LaTeX.

    Why is <Image,FPE+APE> missing?

    The dual-path design and reliance on pretrained modules (SAM, APE, TransTab) increases architectural complexity, which may limit adoption or reproduction without code and pretrained models. The utility of this system depends heavily on accessible pretrained components and a working demo/codebase. As of now, the GitHub repository is unavailable.

    While partial feature training is mentioned, there is no systematic study on how specific feature types (e.g., shape vs. texture) contribute to retrieval quality. Also, see weakness 2 for removing APE ablation study.

    I could clearly see a future extensions to include support for natural language queries (e.g., “find large, round tumors in the frontal lobe” -> which hypothetically should be well translated into R embeddings).

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (4) Weak Accept — could be accepted, dependent on rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Both major weaknesses impact the paper’s credibility and practical reproducibility, while I find the core idea strong and potentially impactful. These concerns must be addressed before the work can be recommended for clear acceptance.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    The authors have provided clear responses to both my major concerns.

    For the lack of formal definitions (R2-W1), they clarified architectural and methodological components with specific answers and committed to clarifying these aspects in the camera-ready version.

    For the query flexibility issue (R2-W2), they explained that APE is randomly used during training in the radiomics path (Fig. 3 to be corrected accordingly), enabling image-free inference. Thus, now aligning with one the paper’s primary claims.

    While some ambiguity remains until the public release of the code, the rebuttal strengthens the credibility of the methodology.

    I find the core idea of the paper original and the results strong, with promising potential for practical (clinical) impact. And therefore recommend this paper for acceptance.

    As final suggestions to the authors:

    1. Use an anonymized GitHub account (or a platform like 4open.science) to upload the code alongside the paper submission.

    2. Slightly shift the paper’s focus from showcasing an “end product” toward emphasizing methodological contributions: clearly explain the method, provide intuitive motivations, and carefully ablate key components.

    3. Also, clearly state which version of the model (e.g., with or without APE, FPE) would be used in practice, since a potential clinical deployment is intended.



Review #3

  • Please describe the contribution of the paper

    In the paper, the authors report a new tool for clinical image retrival that allows to both identifiy relevant images using either a search image or an radiomics feature. A novelty is the identification based on local selections. The code for the proposed method is going to be published allowing its application by others, and the used methods are evaluated using multiple datasets.

  • Please list the major strengths of the paper: you should highlight a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    A major strengths of the paper is for the usage of local identifications approaches which is achieved by an interesting combination of traditional computer vision features and deep learning.

  • Please list the major weaknesses of the paper. Please provide details: for instance, if you state that a formulation, way of using data, demonstration of clinical feasibility, or application is not novel, then you must provide specific references to prior work.

    The paper is interesting with only minor weaknesses. Some weaknesses are:

    1. There are no measures for uncertainty in the reported results of the experiments
    2. The proposed approach is not a novel method in itself but rather a complex combination of existing approaches
    3. I would have loved to see some user-based evaluation
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.

    The submission has provided an anonymized link to the source code, dataset, or any other dependencies.

  • Optional: If you have any additional comments to share with the authors, please provide them here. Please also refer to our Reviewer’s guide on what makes a good review and pay specific attention to the different assessment criteria for the different paper categories: https://conferences.miccai.org/2025/en/REVIEWER-GUIDELINES.html

    N/A

  • Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making.

    (5) Accept — should be accepted, independent of rebuttal

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The proposed approach is intersting and taggles an important problem. I haven’t spotted a significant flaw in the submission.

    The proposed approach does not inovate a single method but reports a complex approach which solves a relevant task in the field.

  • Reviewer confidence

    Confident but not absolutely certain (3)

  • [Post rebuttal] After reading the authors’ rebuttal, please state your final opinion of the paper.

    Accept

  • [Post rebuttal] Please justify your final decision from above.

    For me, the reported approch is highly relevant to the field and the description, which I understand is the major concern of R3, is mainly due to the lack of space.

    Since all my concerns were addressed, I suggest accepting the paper.



Author Feedback

We sincerely thank the reviewers for their insightful comments and positive assessment. Below, we address key concerns.

[R2-W1, R3-W5] Formal definitions and reproducibility We clarify critical architectural points:

  • (a) In the image path, anatomical positional embedding (APE) is downsampled from (3,128,128,128) to (384,8,8,8) via convolution layers (Fig. 2B).
  • (b) Fig. 2A provides a simplified overview, while Fig. 2B details the combination of FPE and APE in E_prompt, maintaining the standard SAM architecture.
  • (c) Fourier-based positional encoding (FPE), originally employed in SAM, encodes positional information solely based on locations within the image. Conversely, APE captures robust global anatomical location context. The image patch inherently contains tumor features along with positional cues from the background. However, radiomics features solely describe tumor-internal attributes without positional context. Thus, the radiomics path incorporates APE to explicitly encode anatomical positional information. FPE is excluded from the radiomics path because it encodes positional details strictly tied to image coordinates, irrelevant without image input. Without APE (as in the <Image, Radiomics, FPE>), radiomics embeddings consist only of tumor-internal features, omitting global positional context during multimodal contrastive learning (as shown in Table 1). In summary, APE is integrated into both the image and radiomics paths to provide consistent anatomical context. We will clarify these points and further formalize the contrastive learning component to enhance reproducibility. Details will be verifiable in the public code upon acceptance.

[R2-W2] Inconsistency in claimed query flexibility In the radiomics path, APE usage was randomized (used/not used) during training to ensure flexible retrieval. This flexibility is naturally supported by TransTab, which allows variable input features. Thus, our model indeed supports genuine image-free queries, as confirmed by Table 2 and Fig. 4. Fig. 3B will be revised to clarify that APE usage is randomized and optional.

[R3-W1, R3-W2] Details on radiomics features Standard radiomics features (shape, histogram, GLCM, GLSZM features) from PyRadiomics (via enableAllFeatures setting, 72 features) were utilized. Detailed descriptions and extraction methods will be provided in the public code.

[R3-W3, R3-W4] SAM and tumor-specific embeddings Fig. 2A indicates SAM components in the image path (green modules). Pre-trained SAM-Med3D provides tumor-level image embeddings through minimal prompts. Tumor ROI masks are used only during training as ground truth for L_seg, ensuring that embeddings represent the tumor indicated by the point prompt. Thus, the term “tumor-specific embeddings” refers to tumor-level rather than image-level embeddings in the manuscript. We will clarify these points.

[R3-W6] Terms/tools clarification (TransTab, ANTsPy, ICBM T1) TransTab (E_radiomics) encodes radiomics features and APE values into embeddings. The ICBM T1 atlas provides anatomical masks, including specific brain lobes. ANTsPy’s non-linear registration aligns these atlas masks onto each MRI, facilitating accurate location-based retrieval evaluation. We will clarify these.

[R3-W7] Methodology in Table 2 The unimodal contrastive approach involves training solely on the image path, defining positive pairs as different cropped views from the same tumor (Fig. 3C). Table 2 was assessed by comparing the correlation between radiomics features extracted from query and retrieved tumors. We will clarify this.

Future Work (R1-W1, R1-W3, R2-C4, R2-C5) We agree with the importance of uncertainty quantification, user-based evaluation, systematic radiomics feature analysis, and natural language query expansions. These will be addressed as future directions in Conclusion. We commit to releasing detailed public code for preprocessing, training, and inference to ensure reproducibility.



Meta-Review

Meta-review #1

  • Your recommendation

    Invite for Rebuttal

  • If your recommendation is “Provisional Reject”, then summarize the factors that went into this decision. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. You do not need to provide a justification for a recommendation of “Provisional Accept” or “Invite for Rebuttal”.

    The reviewers overall appreciate the idea of combining radiomics with deep learning methods for image retrieval. However, the reviewers have also raised concerns regarding clarity of methods and details, as well as limitation in the experimental evaluations. Please address reviewers’ concerns carefully.

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    N/A



Meta-review #2

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Reject

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    This paper introduces a fine-grained 3d image retrieval method based on tumour-level features and images. All reviewers agree that the technique is effective; nonetheless, it relies on a combination of well-established and standard methods. Furthermore, even with the rebuttal, it is unclear how the proposed methods work and improve upon prior art.



Meta-review #3

  • After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.

    Accept

  • Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’

    The paper presents a medical image retrieval framework with an “end product” demonstration. It should be considered an interesting application study with clear clinical relevance. To further strengthen the work, an end user evaluation and the release of code are recommended.



back to top