List of Papers Browse by Subject Areas Author List
Abstract
We propose SpeChrOmics, a characterization framework for generating potential biomarkers of pathologies from hyperspectral images of body tissue. We test our model using a novel clinical application – hyperspectral imaging for the diagnosis of latent tuberculosis infection (LTBI). This is a neglected disease state predominantly prevalent in sub-Saharan Africa. Our model identified water, deoxyhemoglobin, and pheomelanin as potential chromophore biomarkers for LTBI with mean cross validation accuracy of 96%. Our framework can potentially be used for pathology characterization in other medical applications.
Links to Paper and Supplementary Materials
Main Paper (Open Access Version): https://papers.miccai.org/miccai-2024/paper/1551_paper.pdf
SharedIt Link: pending
SpringerLink (DOI): pending
Supplementary Material: N/A
Link to the Code Repository
N/A
Link to the Dataset(s)
N/A
BibTex
@InProceedings{Ola_SpeChrOmics_MICCAI2024,
author = { Oladokun, Ajibola S. and Malila, Bessie and Campello, Victor M. and Shey, Muki and Mutsvangwa, Tinashe E. M.},
title = { { SpeChrOmics: A Biomarker Characterization Framework for Medical Hyperspectral Imaging } },
booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
year = {2024},
publisher = {Springer Nature Switzerland},
volume = {LNCS 15003},
month = {October},
page = {pending}
}
Reviews
Review #1
- Please describe the contribution of the paper
The paper proposes a method to identify biomarkers of diseases, mainly latent tuberculosis infection ( LTBI), from tissue images. The authors extract two types of features from hyperspectral imaging, mainly deep features based on deep learning models and deterministic features based on PyRadiomics to diagnose LTBI and they achieve high accuracy. The authors believe this method can be used for diagnosing other diseases as well.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main strengths of the paper include: 1- Applying the proposed framework to a clinical application for latent tuberculosis infection (LTBI), addressing a neglected disease condition and providing diagnostic biomarkers for LTBI using hyperspectral imaging. 2- Integrating the traditional biomarker extractor PyRadiomics and deep learning models in extracting spatial features from tissue indurations, showcasing the superiority of deep radiomics features over deterministic radiomics features.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The main weaknesses of the paper include: 1- The study acknowledges the limitation of a small dataset size (37 human subjects of African descent), which can impact the generalizability and robustness of the results. 2- The paper does not provide a detailed comparison with existing methods or frameworks for biomarker extraction from hyperspectral images. 3- The technical contributions are very limited. 4- While the paper discusses the potential applications of SpeChrOmics in various medical conditions, there is a lack of in-depth discussion on the practical implementation challenges, regulatory considerations, and integration into existing clinical workflows, which are essential for translating the framework into routine clinical practice. 5- The paper may benefit from providing more detailed technical information on the spectral feature extraction using supplementary materials, such as spatial feature extraction, and feature selection & ranking processes within the paper framework.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
The authors used a private dataset that is not available to reproduce the proposed framework.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
The proposed SpeChrOmics demonstrates an approach to biomarker extraction from hyperspectral images, integrating spectral and spatial feature extraction techniques effectively. The incorporation of deep learning models for spatial feature extraction showcases the potential for advanced image analysis in medical imaging. However,
- The contributions and the novelty of the paper are very limited. There are no scientific contributions.
- The use of PyRadiomics is not novel to extract spatial features of the medical images. As well, as the use of pre-trained deep learning models for analyzing the hyperspectral images.
- The authors should prove that the DL models used are not overfitted, especially with the small set of the used datasets.
- The authors should demonstrate how they chose the wavelength they mentioned in the paper. They mention that “We generated an estimated RGB image from each hypercube by selecting the 750 nm, 560 nm, and 410 nm band images of each hypercube as the red, green, and blue channel images, respectively. These choices of wavelengths were empirically determined by Specim [23] and are an attempt to mimic the peak wavelengths of the color filters used in RGB cameras [24].” But, for me, the choice should not be empirical. The authors should take a look at this reference “Rogers, M., Blanc-Talon, J., Urschler, M. et al. Wavelength and texture feature selection for hyperspectral imaging: a systematic literature review. Food Measure 17, 6039–6064 (2023). https://doi.org/10.1007/s11694-023-02044-x”
- The authors should use one of the off-shelf interpretability methods (GradCAM) to show the features extracted by deep learning models since the authors mentioned in the paper “These features are the potential optical imaging-based biomarkers for TST and IGRA from in vivo induration samples. The best-ranked deterministic radiomics features natively possess explainability, unlike deep radiomics features. “ It’s important to address potential limitations or challenges associated with the proposed methodology, such as computational complexity or potential biases introduced by the dataset or algorithm design.
- The ablation study helps to dissect the contribution of each proposed component to the overall performance of your method. Consider providing more detailed discussions on the insights gained from the ablation study and how they inform future research directions. So the authors should apply an ablation study to see the effect of traditional features and DL features and the integration between them.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Reject — could be rejected, dependent on rebuttal (3)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper missed the following items: 1- Addressing the small dataset size limitation and providing more detailed technical descriptions of the methodology would enhance the robustness and reproducibility of the framework. Consider expanding the discussion on the comparison with existing methods to highlight the uniqueness and advantages of SpeChrOmics 2- While the study shows promise in LTBI diagnosis, further clinical validation studies with larger and diverse patient cohorts are essential to assess the real-world applicability and performance of SpeChrOmics in clinical settings. Consider discussing the practical implementation challenges and regulatory considerations for integrating the framework into routine clinical practice.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The authors proposed a novel framework for LTBI diagnosis, radiomics features and deep learning features were used to validate the performance when different image channels were used for prediction. Machine learning techniques were used to select features and construct the final prediction model. Promissing cross validation performance were reported on an institutional dataset.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This study focuses on a neglected disease, different imaging methods or image channels were evaluted for their predictive ability for diagnosis.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Please check comments in 10.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not mention open access to source code or data but provides a clear and detailed description of the algorithm to ensure reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
NA
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
There two major issues with the submission. 1. The number of patients are too limited, although the authors performed cross validation, the results might still of high uncertainty, and the performance evaluation metric might not be able to refelect it, especailly give the positive and negtive sample are imbalanced. 2. The pretrained models are trained with RGB images, I’m not sure if it’s reasonable to use it for HSI image, since data in each channel represent different things. Addiationally, it would be better to mention the number of selected features, it’s important for radiomics based study especailly for small dataset experiment.
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Weak Accept — could be accepted, dependent on rebuttal (4)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The manuscript is well organized and in good writting, although there are some limitations of the current study.
- Reviewer confidence
Somewhat confident (2)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
Applies multispectral analysis and a method of feature selection over a library of chromophores (molecules) to search for biomarkers.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
A fundamentally interesting, novel method which opens up new options for medical diagnostics research.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
A central technical section of the Methods lost me (or I got lost - don’t know who is responsible).
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Please be aware that providing code and data is a plus, but not a requirement for acceptance.
The submission does not provide sufficient information for reproducibility.
- Do you have any additional comments regarding the paper’s reproducibility?
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review. Pay specific attention to the different assessment criteria for the different paper categories (MIC, CAI, Clinical Translation of Methodology, Health Equity): https://conferences.miccai.org/2024/en/REVIEWER-GUIDELINES.html
A wonderful idea and research program. Pg 2 line 3: Contradictory statements around “not obvious … dimensions.”: Quantitation methods ignore spatial dimensions anyway. Also, the new method focuses on spatial structure. Do you mean “not immediately obvious to a human observer”? Pg 2, Paragraph 2, line 4: “LTBI is a neglected…”: This sentence should perhaps go first to make the sentences follow one another logically. Pg 2 “subdermal”: how deep? Pg 3 “q-channel array”: perhaps specify that these chromophores are a prechosen library. Also, can it be heavily over-complete, or does this break the method (see eg SINDy methods for sparsifying libraries of functionals). pg 3: Does the unmixing function L converge? is the solution unique, and is the problem well-posed? pg 3-4: Are the chromophore spatial maps assumed continuous? What is the impact of skin pores, puncture from TST, hair follicles? Pg 3-4: Also, since you average over all patients when searching for features, are these elements blurred out, and does this matter? Pg 3-4: Is there any spatial component at all to feature selection, since it averages over all patients, who each have different spatial characteristics (absent a well-defined positioning method for the imager). Pg 4: nu: should be lambda by prior notation. Pg 4: gamma-channel: please explain this a bit. pg 4: what does 1 channel consist of? Is it a linear combination of chromophores? And does it have any spatial component? Fig 1: too small to read on printed out paper Fig 1 to half of pg 5: I got lost through here. Maybe run it past a local colleague for clarity (but it totally could be me). pg 5: “in equation …. metric scores”: font “we generated an estimated…”: why not use wider bandpass filters for each color channel? And did the estimated RGB match the actual? “Thus, balanced…”: I don’t see why this follows (ie, why “thus”). “46%”: Does this mean almost perfectly random, no correlation? Is this expected? pg 5 bottom: The library of chromophores all appear “normal”. Are there candidates associated with IGRA which you could have used? Pg 5 bottom: what is the impact of skin melanin? It seems this would be a strong confounder. Fig 3: How distinct are the various subgroups as seen in each of these features? Is there lots of overlap, or little? Is there a way to visualize the distributions and their overlap? Discussion: what is the impact of timing on indurations? Is it slow enough that this is a non-issue?
- Rate the paper on a scale of 1-6, 6 being the strongest (6-4: accept; 3-1: reject). Please use the entire range of the distribution. Spreading the score helps create a distribution for decision-making
Accept — should be accepted, independent of rebuttal (5)
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It describes a creative, novel and potentially useful approach to biomarker searches.
- Reviewer confidence
Confident but not absolutely certain (3)
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Accept — should be accepted, independent of rebuttal (5)
- [Post rebuttal] Please justify your decision
The authors’ response is detailed and thoughtful. Hopefully it addresses the concerns of Reviewers 1 and 4. The conscientiousness of the response indicates to me that the authors will likely take full advantage of the reviews to improve the paper. The authors’ response affirmed my high opinion of the paper.
By the way, Reviewer # 1 had insightful and valuable comments. I respectfully disagree with the request for info re regulatory issues, translation etc is not appropriate for this paper, which focuses on an exploratory technique that has utility at a much earlier phase of research.
Author Feedback
We thank reviewers [R1, R3, R4] for their positive feedback. They appreciate that our approach: provides diagnostic biomarkers for LTBI, a neglected disease condition, using HSI [R1], is a “fundamentally interesting, novel method which opens up new options for medical diagnostics research” [R3], and is a “novel framework for LTBI diagnosis” [R4]. Below we address the reviewers’ major concerns.
Concerns on limited sample size [R1, R4] – We acknowledge concerns on the limited dataset size. To the best of our knowledge, the dataset in [Gu et al., 2018, DOI:10.1007/978-3-030-01201-4_29] is the only publicly available HSI dataset of skin pathology (skin cancer). However, its limited number of wavelengths (16 vs 204 in ours) and narrow wavelength range (460 – 630 nm vs 450 – 1003 nm in ours) prevent its use as a validation dataset for our framework. Estimation of chromophores such as water and fat require wavelengths beyond 630 nm. The pre-trained DL models used for deep radiomics feature extraction were frozen and not fine-tuned to our HSI data. Our SVM classification-based approach to feature selection and ranking is where a risk of overfitting may exist. We mitigated this risk as well as the risk of result uncertainty raised by R4 by utilising balanced accuracy metric (rather than standard accuracy) over five cross-validation (CV) folds such that the validation set per round is unique. Best features were ranked and selected based on average balanced validation accuracies across the five rounds. Each round’s SVM model had the same initial condition, and no knowledge transfer occurred between rounds, reducing overfitting risk fivefold compared to a single round of CV.
Scientific Contributions [R1] – While PyRadiomics and pre-trained DL models have been used for spatial feature extraction in other medical imaging modalities, we are the first, to the best of our knowledge, to layer PyRadiomics, and pre-trained models trained on a recent large-scale multi-modality medical imaging dataset (RadImageNet), on top of traditional HSI spectral feature extraction to generate a comprehensive array of features that encapsulate both spectral (chromophore) and spatial characteristics of hypercubes. Showing the potential of multimodal domain knowledge from RadImageNet’s CT, MRI, and US images of 11 anatomical regions in generating promising HSI biomarkers is noteworthy. We pioneer the extraction and analysis of chromophore maps for TST-based LTBI diagnosis.
Comparison with existing methods for biomarker extraction from HSI [R1] – To the best of our knowledge, our framework is the first to generate HSI-based biomarkers that encapsulate both chromophore concentrations and robust quantitative spatial tissue relationship. Current methods focus mainly on chromophore map generation and qualitative analysis of the maps.
Integrating SpeChrOmics into clinical workflows [R1] – This paper focuses on assessing the predictive value and HSI biomarker-generating capabilities of SpeChrOmics. We believe this paper serves as a foundation for later works which can validate the SpeChrOmics on a larger and more diverse dataset, which will generate more evidence of efficacy and safety, in addition to the ones presented in this study, in support of downstream regulatory measures for integration into routine clinical practices.
Suitability of RadImageNet for HSI [R4] – Though RadImageNet images have 3 channels, they are different from conventional/natural RGB images. They are grayscale images captured from 2D slices of CT, MRI, and US, which are repeated into 3 channels per image. We believe that spatial tissue characteristics captured in RadImageNet are significantly similar to the spatial characteristics of chromophore maps. Thus, we believe RadImageNet is a suitable pretraining dataset for spatial feature extraction in HSI.
Meta-Review
Meta-review #1
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
this paper is an interesting one. Authors responded well to the critiques. First of all, the use of radionics with pre-trained deep learning model is maybe not the first one but the way authors show combination is the first one. Also, application is important, it is an ancient disease with resisting to all human attempt, and sample size answer is not bad, sometimes it is hard to find the dataset with enough samples to convince the readers but authors use the methods with recognizing the limitations.
I think the paper and responses deserve a weak accept, the overall recommendation by the reviewers is not reject either.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
this paper is an interesting one. Authors responded well to the critiques. First of all, the use of radionics with pre-trained deep learning model is maybe not the first one but the way authors show combination is the first one. Also, application is important, it is an ancient disease with resisting to all human attempt, and sample size answer is not bad, sometimes it is hard to find the dataset with enough samples to convince the readers but authors use the methods with recognizing the limitations.
I think the paper and responses deserve a weak accept, the overall recommendation by the reviewers is not reject either.
Meta-review #2
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Reject
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
The paper proposes a method to identify biomarkers of diseases, mainly latent tuberculosis infection ( LTBI), from tissue images. Though the rebuttal submitted by the authors addressed some of the concerns, the major concerns are related to technical novelty and generalizability of the results given the dataset is small. Due to this, I recommend reject.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
The paper proposes a method to identify biomarkers of diseases, mainly latent tuberculosis infection ( LTBI), from tissue images. Though the rebuttal submitted by the authors addressed some of the concerns, the major concerns are related to technical novelty and generalizability of the results given the dataset is small. Due to this, I recommend reject.
Meta-review #3
- After you have reviewed the rebuttal and updated reviews, please provide your recommendation based on all reviews and the authors’ rebuttal.
Accept
- Please justify your recommendation. You may optionally write justifications for ‘accepts’, but are expected to write a justification for ‘rejects’
This paper received an accept, weak accept, and weak reject score. The main criticism concerned the limited novelty, small dataset size, and missing technical details. The primary and secondary ACs disagreed in their final recommendation.
This paper addresses a neglected disease condition where data scarcity is a fundamental problem. This meta reviewer felt that the paper makes a valuable contribution despite its limitations. It is important to highlight under-represented areas of research which naturally struggle with the availability of large datasets, established evaluation frameworks, and baseline comparisons. Here, the paper makes a good starting point for further research and is well in scope for the health equity track, addressing a medical image computing problem in limited-resource settings with sound methodology.
The authors should highlight limitations in their discussion of the final version.
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
This paper received an accept, weak accept, and weak reject score. The main criticism concerned the limited novelty, small dataset size, and missing technical details. The primary and secondary ACs disagreed in their final recommendation.
This paper addresses a neglected disease condition where data scarcity is a fundamental problem. This meta reviewer felt that the paper makes a valuable contribution despite its limitations. It is important to highlight under-represented areas of research which naturally struggle with the availability of large datasets, established evaluation frameworks, and baseline comparisons. Here, the paper makes a good starting point for further research and is well in scope for the health equity track, addressing a medical image computing problem in limited-resource settings with sound methodology.
The authors should highlight limitations in their discussion of the final version.