https://github.com/TissueImageAnalytics/cerberus
One Model is All You Need: Multi-Task Learning Enables Simultaneous Histology Image Segmentation and Classification
https://github.com/TissueImageAnalytics/cerberus
computational-pathology deep-learning multi-task-learning python pytorch
Last synced: 16 days ago
JSON representation
One Model is All You Need: Multi-Task Learning Enables Simultaneous Histology Image Segmentation and Classification
- Host: GitHub
- URL: https://github.com/TissueImageAnalytics/cerberus
- Owner: TissueImageAnalytics
- License: gpl-3.0
- Created: 2022-07-27T17:07:19.000Z (almost 3 years ago)
- Default Branch: master
- Last Pushed: 2024-11-27T19:43:47.000Z (6 months ago)
- Last Synced: 2025-03-29T20:01:47.377Z (about 2 months ago)
- Topics: computational-pathology, deep-learning, multi-task-learning, python, pytorch
- Language: Python
- Homepage:
- Size: 217 KB
- Stars: 83
- Watchers: 5
- Forks: 12
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-pathology - Cerberus - Multi-task learning enables simultaneous histology image segmentation and classification. (Software / Model)
README
[](https://www.gnu.org/licenses/gpl-3.0)
# One Model is All You Need: Multi-Task Learning Enables Simultaneous Histology Image Segmentation and Classification
This repository contains code for using Cerberus, our multi-task model outlined in our [Medical Image Analysis paper](https://doi.org/10.1016/j.media.2022.102685).
Scroll down to the bottom to find instructions on downloading our [pretrained weights](#download-weights) and [WSI-level results](#download-tcga-results).
## Set Up Environment
```
# create base conda environment
conda env create -f environment.yml
# activate environment
conda activate cerberus
# install PyTorch with pip
pip install torch==1.10.1+cu102 torchvision==0.11.2+cu102 -f https://download.pytorch.org/whl/cu102/torch_stable.html
```
## Repository Structure
Below we outline the contents of the directories in the repository.
- `infer`: Inference scripts
- `loader`: Data loading and post processing scripts
- `misc`: Miscellaneous scripts and functions
- `models`: Scripts relating to model definition and hyperparameters
- `run_utils`: Model engine and callbacks
The purpose of the main scripts in the repository:
- `run_infer_tile.py`: Run inference on image tiles
- `run_infer_wsi.py`: Run inference on whole-slide images
## Inference
### Tiles
To process large image tiles, run:
```
python run_infer_tile.py --gpu= --batch_size= --model= --input_dir= --output_dir=
```
For convenience, we have also included a bash script, where you can populate command line arguments. To make this script executable, run `chmod +x run_tile.sh`. Then use the command `./run_tile.sh`.
### WSIs
To process whole-slide images, run:
```
python run_infer_wsi.py --gpu= --batch_size= --model= --input_dir= mask_dir= --output_dir=
```
Similar to the tile mode, we have included an example bash script (`run_wsi.sh`) that can be used to run the command, without having to always re-enter the arguments.
For both tile and WSI inference, the model path should point to a directory containing the settings file and the weights (`.tar` file). You will see from the above command that there is a `mask_dir` argument. In this repo, we assume that tissue masks have been automatically generated. You should include masks - otherwise it will lead to significantly longer processing times.
## Download Weights
In this repository, we enable the download of:
- Cerberus model for simultaneous:
- Gland instance segmentation
- Gland semantic segmentation (classification)
- Nuclear instance segmentation
- Nuclear semantic segmentation (classification)
- Lumen instance segmentation
- Tissue type patch classification
- Pretrained ResNet weights (torchvision compatible) for transfer learning
- Pretrained weights obtained from training each fold using:
- ImageNet weights and MTL
- ImageNet weights and MTL (with patch classification)
Download all of the above weights by visiting [this page](https://warwick.ac.uk/fac/cross_fac/tia/software/cerberus/).
Note, the pretrained weights are designed for weight initialisation - not for model inference.
All weights are under a non-commercial license. See the [License section](#license) for more details.
## Download TCGA Results
Download results from processing 599 CRC WSIs using Cerberus at [this page](https://warwick.ac.uk/fac/cross_fac/tia/software/cerberus/).
## License
Code is under a GPL-3.0 license. See the [LICENSE](https://github.com/TissueImageAnalytics/cerberus/blob/master/LICENSE) file for further details.
Model weights are licensed under [Attribution-NonCommercial-ShareAlike 4.0 International](http://creativecommons.org/licenses/by-nc-sa/4.0/). Please consider the implications of using the weights under this license.
## Cite this repository
```
@article{graham2022one,
title={One model is all you need: multi-task learning enables simultaneous histology image segmentation and classification},
author={Graham, Simon and Vu, Quoc Dang and Jahanifar, Mostafa and Raza, Shan E Ahmed and Minhas, Fayyaz and Snead, David and Rajpoot, Nasir},
journal={Medical Image Analysis},
pages={102685},
year={2022},
publisher={Elsevier}
}
```