Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/Zonmgin-Zhang/ASI-Seg
[IROS2024] ASI-Seg: Audio-Driven Surgical Instrument Segmentation with Surgeon Intention Understanding
https://github.com/Zonmgin-Zhang/ASI-Seg
Last synced: about 1 month ago
JSON representation
[IROS2024] ASI-Seg: Audio-Driven Surgical Instrument Segmentation with Surgeon Intention Understanding
- Host: GitHub
- URL: https://github.com/Zonmgin-Zhang/ASI-Seg
- Owner: Zonmgin-Zhang
- Created: 2024-07-05T05:33:13.000Z (6 months ago)
- Default Branch: main
- Last Pushed: 2024-07-08T09:29:53.000Z (6 months ago)
- Last Synced: 2024-09-16T08:16:53.662Z (4 months ago)
- Language: Python
- Homepage:
- Size: 668 KB
- Stars: 4
- Watchers: 2
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- Awesome-Segment-Anything - [code
README
# ASI-Seg: Audio-Driven Surgical Instrument Segmentation with Surgeon Intention Understanding
## Overview
## Installation
The code requires `python>=3.8`, as well as `pytorch>=1.7` and `torchvision>=0.8`. For this project, `python=3.8`, `pytorch=1.11.0`, and `torchvision=0.12.0` are used; run the following command in the root directory of this project:
```
cd ./ASI
pip install -r requirements.txt
```## Dataset
The datasets we used in our experiments are [endoivs 2018](https://cataracts2018.grand-challenge.org/data/) and [endoivs 2017](https://endovissub2017-kidneyboundarydetection.grand-challenge.org/Data/).
For EndoVis2017, we use [robot-surgery-segmentation](https://github.com/ternaus/robot-surgery-segmentation) as our pre-processing strategies and cross-validation splits.
For EndoVis2018, we use [ISINet](https://github.com/BCV-Uniandes/ISINet) as the instrument type segmentation annotation.
## Checkpoints
In ASI-Seg, we used `vit_h` for SAM (Segmentation Anything Model), and CLIP (Contrastive Language-Image Pre-Training).
Please see the download link of the checkpoint of SAM in the `vit_h` version [here](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth).
Please run the following command in the root directory of this project to download CLIP:
```
cd ./ASI
git clone https://github.com/openai/CLIP.git
```## Train
Run the following command in the root directory:
```
cd ./ASI
python train.py
```
## InferenceRun the following command in the root directory:
```
cd ./ASI
python inference.py
```## Citation
If you use our code or paper in your work, please cite our paper.
```
@inproceedings{chen2024iros,
title={{ASI-Seg: Audio-Driven Surgical Instrument Segmentation with Surgeon Intention Understanding}},
author={Zhen Chen, Zongming Zhang, Wenwu Guo, Xingjian Luo, Long Bai, Jinlin Wu,
Hongliang Ren, Hongbin Liu},
booktitle={IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
year={2024}
}
```