Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/viacheslavdanilov/hsi_analysis

This repository is dedicated to the segmentation of hyperspectral images during experimental animal surgery, where a variety of tasks were performed to process and analyze the hyperspectral data collected at the Institute of Image Guided Surgery in Strasbourg.
https://github.com/viacheslavdanilov/hsi_analysis

clustering deep-learning hyperspectral-imaging machine-learning object-detection segmentation

Last synced: 8 days ago
JSON representation

This repository is dedicated to the segmentation of hyperspectral images during experimental animal surgery, where a variety of tasks were performed to process and analyze the hyperspectral data collected at the Institute of Image Guided Surgery in Strasbourg.

Awesome Lists containing this project

README 

        
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.10444212.svg)](https://zenodo.org/doi/10.5281/zenodo.10444212)

[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.10444269.svg)](https://zenodo.org/doi/10.5281/zenodo.10444269)

[![DOI](http://img.shields.io/badge/DOI-10.1016/j.compbiomed.2024.108849-B31B1B)](https://doi.org/10.1016/j.compbiomed.2024.108849)



# Harnessing machine learning for laser ablation assessment in hyperspectral imaging





## 📖 Contents

- [Introduction](#introduction)

- [Data](#data)

- [Methods](#methods)

- [Results](#results)

- [Conclusion](#conclusion)

- [Requirements](#requirements)

- [Installation](#installation)

- [Data Access](#data-access)

- [How to Cite](#how-to-cite)





## 🎯 Introduction

Our study aims to advance the application of dimensionality reduction, object detection, and segmentation in Hyperspectral Imaging (HSI), specifically for tissue ablation monitoring. We evaluate different modalities for ablation detection and segmentation in [hyperspectral images](https://en.wikipedia.org/wiki/Hyperspectral_imaging), focusing on thermal effects induced by laser ablation treatment _in vivo_.





## 📂 Data



During our experimental study, we utilized a [TIVITA hyperspectral camera](https://diaspective-vision.com/en/produkt/tivita-2-0/#produkt) with a spectral range of 500-995 nm to capture hypercubes of 640x480x100 voxels, encompassing 100 bands, alongside regular RGB images. The acquisition process, synchronized to mitigate breathing motion, involved placing polyurethane markers around the target area for spatial reference. To minimize extraneous light, the camera was positioned vertically at a 40 cm distance from the surgical field. A 20 W Halogen lamp was used as the light source. Two distinct imaging modes, reflectance-based and absorbance-based (Table 1 and Table 2), were employed to provide comprehensive insight into sample properties. A dataset comprising 233 hyperspectral cubes from 20 experiments, spanning pre-laparotomy, temperature escalation, and post-ablation phases, was collected, offering a robust foundation for analysis. Temperature thresholds recorded during the experiments delineate the thermal effects produced, as illustrated in accompanying figures.



Table 1. Example of hyperspectral images taken at different wavelengths.



|                                                     Absorbance                                                      |                                                         HSV                                                         |                                                     Reflectance                                                     |

|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|

|  |  |  |



 



        
Table 2. Ablated area at different temperatures.


    


        T 

        Absorbance

        HSV

        Reflectance

    

    

        35

        Absorbance (start)

        HSV (start)

        Reflectance (start)

    

    

        60

        Absorbance (mid)

        HSV (mid)

        Reflectance (mid)

    

    

        110

        Absorbance (end)

        HSV (end)

        Reflectance (end)

    





## 🔬 Methods

The workflow proposed in this study, presented in Figure 1, consists of several key steps to extract valuable information from hyperspectral data in the laser-mediated thermal treatment scenario. First, dimensionality reduction is applied to reduce the high-dimensional data to a manageable size. Next, a supervised learning technique based on neural networks is used to detect the ablation area, i.e., the region of the image where tissue has been treated by laser irradiation. Finally, an unsupervised learning technique based on clustering is used to segment the spectral signature of the ablation area, allowing the identification of specific tissue types or classes of thermal damage. The proposed workflow provides a comprehensive approach to the analysis of hyperspectral data and has the potential to improve the accuracy and efficiency of diseased tissue analysis in the thermal treatment scenario.





  Proposed workflow






    Figure 1. Proposed workflow for hyperspectral image processing and analysis. The workflow consists of three main components: dimensionality reduction, ablation area detection using supervised learning, and spectral signature segmentation based on unsupervised learning.






## 📈 Results

The segmentation of the ablation area in hyperspectral images was meticulously examined through various clustering algorithms (Figure 2). While DBSCAN, OPTICS, and affinity propagation resulted in oversimplification, k-means, BIRCH, agglomerative clustering, spectral clustering, and GMM showcased superior performance, albeit requiring manual cluster input. Notably, Mean Shift emerged as a standout performer, offering high-quality segmentation without manual cluster definition, thanks to its adaptability, autonomous cluster center determination, and robustness to noise. Our analysis revealed significant variation in cluster numbers across reflectance and absorbance modalities, influenced by tissue-specific spectral characteristics and temperature-dependent variations, underscoring the necessity for adaptable segmentation approaches tailored to spectral complexities.





  Segmentation results






    Figure 2. Comparison of ablation segmentation performed with different unsupervised algorithms. The top row represents the input data for clustering algorithms.






## 🏁 Conclusion

This study introduces a robust workflow for analyzing ablation detection and segmentation in hyperspectral images from laser treatment on in-vivo tissues. Leveraging PCA, t-SNE, and Faster R-CNN, we enhance hyperspectral data analysis, facilitating accurate ablation identification and localization. Mean Shift stands out for automated, high-quality segmentation. Our findings guide future research in refining techniques and extending applications to diverse medical scenarios, improving analysis and decision-making in laser cancer therapy and beyond.





## 💻 Requirements

- Operating System

  - [x] macOS

  - [x] Linux

  - [x] Windows (limited testing carried out)

- Python 3.8.x

- Required core packages: [dev.txt](https://github.com/ViacheslavDanilov/hsi_analysis/blob/main/requirements/dev.txt)





## ⚙ Installation

**Step 1:** Download and install Miniconda

``` bash

wget https://repo.anaconda.com/miniconda/Miniconda3-py38_22.11.1-1-Linux-x86_64.sh

chmod +x Miniconda3-latest-Linux-x86_64.sh

./Miniconda3-latest-Linux-x86_64.sh

```



**Step 2:** Install FFmpeg and verify that the installation is correct



- Linux

``` bash

sudo apt update

sudo apt upgrade

sudo apt install ffmpeg

ffmpeg -version

```



- macOS

``` bash

brew update

brew upgrade

brew install ffmpeg

ffmpeg

```



**Step 3:** Clone the repository, create a conda environment, and install the requirements for the repository

``` bash

git clone https://github.com/ViacheslavDanilov/hsi_analysis.git

cd hsi_analysis

chmod +x create_env.sh

source create_env.sh

```





## 🔐 Data Access

All essential components of the study, including the curated source code, dataset, and trained models, are publicly available:

- **Source code:** https://github.com/ViacheslavDanilov/hsi_analysis

- **Dataset:** https://doi.org/10.5281/zenodo.10444212

- **Models:** https://doi.org/10.5281/zenodo.10444269





## 🖊️ How to Cite

Please cite [our paper](https://www.sciencedirect.com/science/article/pii/S001048252400934X) if you found our data, methods, or results helpful for your research:



> Danilov V.V., De Landro M., Felli E., Barberio M., Diana M., Saccomandi P. (**2024**). _Advancing laser ablation assessment in hyperspectral imaging through machine learning_. **Computers in Biology and Medicine**. DOI: https://doi.org/10.1016/j.compbiomed.2024.108849