Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/borda/pyimsegm
Image segmentation - general superpixel segmentation & center detection & region growing
https://github.com/borda/pyimsegm
biomedical-imaging center-detection ellipse-fitting graph-cut graph-cuts image-analysis image-annotation image-processing image-segmentation ipynb medical-imaging microscopy-images object-detection region-growing segmentation-pipeline shape-models superpixel-segmentation superpixels
Last synced: about 2 months ago
JSON representation
Image segmentation - general superpixel segmentation & center detection & region growing
- Host: GitHub
- URL: https://github.com/borda/pyimsegm
- Owner: Borda
- License: bsd-3-clause
- Archived: true
- Created: 2017-07-10T02:17:15.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2022-01-04T12:26:08.000Z (almost 3 years ago)
- Last Synced: 2024-09-27T23:41:07.077Z (about 2 months ago)
- Topics: biomedical-imaging, center-detection, ellipse-fitting, graph-cut, graph-cuts, image-analysis, image-annotation, image-processing, image-segmentation, ipynb, medical-imaging, microscopy-images, object-detection, region-growing, segmentation-pipeline, shape-models, superpixel-segmentation, superpixels
- Language: Python
- Homepage: https://borda.github.io/pyImSegm
- Size: 21.8 MB
- Stars: 373
- Watchers: 14
- Forks: 73
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Contributing: .github/CONTRIBUTING.md
- License: LICENSE
- Code of conduct: .github/CODE_OF_CONDUCT.md
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
# Image segmentation toolbox
[](https://github.com/Borda/pyImSegm/actions?query=workflow%3A%22CI+testing%22)
[](https://codecov.io/gh/Borda/pyImSegm)
[](https://www.codacy.com/app/Borda/pyImSegm?utm_source=github.com&utm_medium=referral&utm_content=Borda/pyImSegm&utm_campaign=Badge_Grade)
[](https://circleci.com/gh/Borda/pyImSegm)
[](https://www.codefactor.io/repository/github/borda/pyimsegm)
[](https://lgtm.com/projects/g/Borda/pyImSegm/context:python)
[](https://pyimsegm.readthedocs.io/en/latest/?badge=latest)
[](https://gitter.im/pyImSegm/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
[](https://results.pre-commit.ci/latest/github/Borda/pyImSegm/master)
______________________________________________________________________
## Superpixel segmentation with GraphCut regularisation
Image segmentation is widely used as an initial phase of many image processing tasks in computer vision and image analysis. Many recent segmentation methods use superpixels because they reduce the size of the segmentation problem by order of magnitude. Also, features on superpixels are much more robust than features on pixels only. We use spatial regularisation on superpixels to make segmented regions more compact. The segmentation pipeline comprises (i) computation of superpixels; (ii) extraction of descriptors such as colour and texture; (iii) soft classification, using a standard classifier for supervised learning, or the Gaussian Mixture Model for unsupervised learning; (iv) final segmentation using Graph Cut. We use this segmentation pipeline on real-world applications in medical imaging (see [sample images](data-images/).
We also show that [unsupervised segmentation](notebooks/segment-2d_slic-fts-clust-gc.ipynb) is sufficient for some situations,
and provides similar results to those obtained using [trained segmentation](notebooks/segment-2d_slic-fts-classif-gc.ipynb).
**Sample ipython notebooks:**
- [Supervised segmentation](notebooks/segment-2d_slic-fts-classif-gc.ipynb) requires training annotation
- [Unsupervised segmentation](notebooks/segment-2d_slic-fts-clust-gc.ipynb) just asks for expected number of classes
- **partially annotated images** with missing annotation is marked by a negative number
**Illustration**
Reference: _Borovec J., Svihlik J., Kybic J., Habart D. (2017). **Supervised and unsupervised segmentation using superpixels, model estimation, and Graph Cut.** In: Journal of Electronic Imaging._
## Object centre detection and Ellipse approximation
An image processing pipeline to detect and localize Drosophila egg chambers that consists of the following steps: (i) superpixel-based image segmentation into relevant tissue classes (see above); (ii) detection of egg center candidates using label histograms and ray features; (iii) clustering of center candidates and; (iv) area-based maximum likelihood ellipse model fitting.
See our [Poster](http://cmp.felk.cvut.cz/~borovji3/documents/poster-MLMI2017.compressed.pdf) related to this work.
**Sample ipython notebooks:**
- [Center detection](notebooks/egg-center_candidates-clustering.ipynb) consists of center candidate training and prediction, and candidate clustering.
- [Ellipse fitting](notebooks/egg-detect_ellipse-fitting.ipynb) with given estimated center structure segmentation.
**Illustration**
Reference: _Borovec J., Kybic J., Nava R. (2017) **Detection and Localization of Drosophila Egg Chambers in Microscopy Images.** In: Machine Learning in Medical Imaging._
## Superpixel Region Growing with Shape prior
Region growing is a classical image segmentation method based on hierarchical region aggregation using local similarity rules. Our proposed approach differs from standard region growing in three essential aspects. First, it works on the level of superpixels instead of pixels, which leads to a substantial speedup. Second, our method uses learned statistical shape properties which encourage growing leading to plausible shapes. In particular, we use ray features to describe the object boundary. Third, our method can segment multiple objects and ensure that the segmentations do not overlap. The problem is represented as energy minimisation and is solved either greedily, or iteratively using GraphCuts.
**Sample ipython notebooks:**
- [General GraphCut](notebooks/egg_segment_graphcut.ipynb) from given centers and initial structure segmentation.
- [Shape modeling](notebooks/RG2Sp_shape-models.ipynb) estimation from training examples.
- [Region growing](notebooks/RG2Sp_region-growing.ipynb) from given centers and initial structure segmentation with shape models.
**Illustration**
Reference: _Borovec J., Kybic J., Sugimoto, A. (2017). **Region growing using superpixels with learned shape prior.** In: Journal of Electronic Imaging._
______________________________________________________________________
## Installation and configuration
**Configure local environment**
Create your own local environment, for more see the [User Guide](https://pip.pypa.io/en/latest/user_guide.html), and install dependencies requirements.txt contains list of packages and can be installed as
```bash
@duda:~$ cd pyImSegm
@duda:~/pyImSegm$ virtualenv env
@duda:~/pyImSegm$ source env/bin/activate
(env)@duda:~/pyImSegm$ pip install -r requirements.txt
(env)@duda:~/pyImSegm$ python ...
```
and in the end terminating...
```bash
(env)@duda:~/pyImSegm$ deactivate
```
**Compilation**
We have implemented `cython` version of some functions, especially computing descriptors, which require to compile them before using them
```bash
python setup.py build_ext --inplace
```
If loading of compiled descriptors in `cython` fails, it is automatically swapped to use `numpy` which gives the same results, but it is significantly slower.
**Installation**
The package can be installed via pip
```bash
pip install git+https://github.com/Borda/pyImSegm.git
```
or using `setuptools` from a local folder
```bash
python setup.py install
```
______________________________________________________________________
## Experiments
Short description of our three sets of experiments that together compose single image processing pipeline in this order:
1. **Semantic (un/semi)supervised segmentation**
1. **Center detection and ellipse fitting**
1. **Region growing with the learned shape prior**
### Annotation tools
We introduce some useful tools for work with image annotation and segmentation.
- **Quantization:** in case you have some smooth colour labelling in your images you can remove them with following quantisation script.
```bash
python handling_annotations/run_image_color_quantization.py \
-imgs "./data-images/drosophila_ovary_slice/segm_rgb/*.png" \
-m position -thr 0.01 --nb_workers 2
```
- **Paint labels:** concerting image labels into colour space and other way around.
```bash
python handling_annotations/run_image_convert_label_color.py \
-imgs "./data-images/drosophila_ovary_slice/segm/*.png" \
-out ./data-images/drosophila_ovary_slice/segm_rgb
```
- **Visualisation:** having input image and its segmentation we can use simple visualisation which overlap the segmentation over input image.
```bash
python handling_annotations/run_overlap_images_segms.py \
-imgs "./data-images/drosophila_ovary_slice/image/*.jpg" \
-segs ./data-images/drosophila_ovary_slice/segm \
-out ./results/overlap_ovary_segment
```
- **In-painting** selected labels in segmentation.
```bash
python handling_annotations/run_segm_annot_inpaint.py \
-imgs "./data-images/drosophila_ovary_slice/segm/*.png" \
--label 4
```
- **Replace labels:** change labels in input segmentation into another set of labels in 1:1 schema.
```bash
python handling_annotations/run_segm_annot_relabel.py \
-out ./results/relabel_center_levels \
--label_old 2 3 --label_new 1 1
```
### Semantic (un/semi)supervised segmentation
We utilise (un)supervised segmentation according to given training examples or some expectations.
- Evaluate superpixels (with given SLIC parameters) quality against given segmentation. It helps to find out the best SLIC configuration.
```bash
python experiments_segmentation/run_eval_superpixels.py \
-imgs "./data-images/drosophila_ovary_slice/image/*.jpg" \
-segm "./data-images/drosophila_ovary_slice/annot_eggs/*.png" \
--img_type 2d_split \
--slic_size 20 --slic_regul 0.25 --slico
```
- Perform **Un-Supervised** segmentation in images given in CSV
```bash
python experiments_segmentation/run_segm_slic_model_graphcut.py \
-l ./data-images/langerhans_islets/list_lang-isl_imgs-annot.csv -i "" \
-cfg experiments_segmentation/sample_config.yml \
-o ./results -n langIsl --nb_classes 3 --visual --nb_workers 2
```
OR specified on particular path:
```bash
python experiments_segmentation/run_segm_slic_model_graphcut.py \
-l "" -i "./data-images/langerhans_islets/image/*.jpg" \
-cfg ./experiments_segmentation/sample_config.yml \
-o ./results -n langIsl --nb_classes 3 --visual --nb_workers 2
```
- Perform **Supervised** segmentation with afterwards evaluation.
```bash
python experiments_segmentation/run_segm_slic_classif_graphcut.py \
-l ./data-images/drosophila_ovary_slice/list_imgs-annot-struct.csv \
-i "./data-images/drosophila_ovary_slice/image/*.jpg" \
--path_config ./experiments_segmentation/sample_config.yml \
-o ./results -n Ovary --img_type 2d_split --visual --nb_workers 2
```
- Perform **Semi-Supervised** is using the the supervised pipeline with not fully annotated images.
- For both experiment you can evaluate segmentation results.
```bash
python experiments_segmentation/run_compute-stat_annot-segm.py \
-a "./data-images/drosophila_ovary_slice/annot_struct/*.png" \
-s "./results/experiment_segm-supervise_ovary/*.png" \
-i "./data-images/drosophila_ovary_slice/image/*.jpg" \
-o ./results/evaluation --visual
```
The previous two (un)segmentation accept [configuration file](experiments_segmentation/sample_config.yml) (YAML) by parameter `-cfg` with some extra parameters which was not passed in arguments, for instance:
```yaml
slic_size: 35
slic_regul: 0.2
features:
color_hsv: ['mean', 'std', 'eng']
classif: 'SVM'
nb_classif_search: 150
gc_edge_type: 'model'
gc_regul: 3.0
run_LOO: false
run_LPO: true
cross_val: 0.1
```
### Center detection and ellipse fitting
In general, the input is a formatted list (CSV file) of input images and annotations. Another option is set by `-list none` and then the list is paired with given paths to images and annotations.
**Experiment sequence is the following:**
1. We can create the annotation completely manually or use the following script which uses annotation of individual objects and create the zones automatically.
```bash
python experiments_ovary_centres/run_create_annotation.py
```
1. With zone annotation, we train a classifier for centre candidate prediction. The annotation can be a CSV file with annotated centres as points, and the zone of positive examples is set uniformly as the circular neighbourhood around these points. Another way (preferable) is to use an annotated image with marked zones for positive, negative and neutral examples.
```bash
python experiments_ovary_centres/run_center_candidate_training.py -list none \
-segs "./data-images/drosophila_ovary_slice/segm/*.png" \
-imgs "./data-images/drosophila_ovary_slice/image/*.jpg" \
-centers "./data-images/drosophila_ovary_slice/center_levels/*.png" \
-out ./results -n ovary
```
1. Having trained classifier we perform center prediction composed from two steps: i) center candidate clustering and ii) candidate clustering.
```bash
python experiments_ovary_centres/run_center_prediction.py -list none \
-segs "./data-images/drosophila_ovary_slice/segm/*.png" \
-imgs "./data-images/drosophila_ovary_slice/image/*.jpg" \
-centers ./results/detect-centers-train_ovary/classifier_RandForest.pkl \
-out ./results -n ovary
```
1. Assuming you have an expert annotation you can compute static such as missed eggs.
```bash
python experiments_ovary_centres/run_center_evaluation.py
```
1. This is just cut out clustering in case you want to use different parameters.
```bash
python experiments_ovary_centres/run_center_clustering.py \
-segs "./data-images/drosophila_ovary_slice/segm/*.png" \
-imgs "./data-images/drosophila_ovary_slice/image/*.jpg" \
-centers "./results/detect-centers-train_ovary/candidates/*.csv" \
-out ./results
```
1. Matching the ellipses to the user annotation.
```bash
python experiments_ovary_detect/run_ellipse_annot_match.py \
-info "~/Medical-drosophila/all_ovary_image_info_for_prague.txt" \
-ells "~/Medical-drosophila/RESULTS/3_ellipse_ransac_crit_params/*.csv" \
-out ~/Medical-drosophila/RESULTS
```
1. Cut eggs by stages and norm to mean size.
```bash
python experiments_ovary_detect/run_ellipse_cut_scale.py \
-info ~/Medical-drosophila/RESULTS/info_ovary_images_ellipses.csv \
-imgs "~/Medical-drosophila/RESULTS/0_input_images_png/*.png" \
-out ~/Medical-drosophila/RESULTS/images_cut_ellipse_stages
```
1. Rotate (swap) extracted eggs according the larger mount of mass.
```bash
python experiments_ovary_detect/run_egg_swap_orientation.py \
-imgs "~/Medical-drosophila/RESULTS/atlas_datasets/ovary_images/stage_3/*.png" \
-out ~/Medical-drosophila/RESULTS/atlas_datasets/ovary_images/stage_3
```
### Region growing with a shape prior
In case you do not have estimated object centres, you can use [plugins](ij_macros) for landmarks import/export for [Fiji](http://fiji.sc/).
**Note:** install the multi-snake package which is used in multi-method segmentation experiment.
```bash
pip install --user git+https://github.com/Borda/morph-snakes.git
```
**Experiment sequence is the following:**
1. Estimating the shape model from set training images containing a single egg annotation.
```bash
python experiments_ovary_detect/run_RG2Sp_estim_shape-models.py \
-annot "~/Medical-drosophila/egg_segmentation/mask_2d_slice_complete_ind_egg/*.png" \
-out ./data-images -nb 15
```
1. Run several segmentation techniques on each image.
```bash
python experiments_ovary_detect/run_ovary_egg-segmentation.py \
-list ./data-images/drosophila_ovary_slice/list_imgs-segm-center-points.csv \
-out ./results -n ovary_image --nb_workers 1 \
-m ellipse_moments \
ellipse_ransac_mmt \
ellipse_ransac_crit \
GC_pixels-large \
GC_pixels-shape \
GC_slic-large \
GC_slic-shape \
rg2sp_greedy-mixture \
rg2sp_GC-mixture \
watershed_morph
```
1. Evaluate your segmentation ./results to expert annotation.
```bash
python experiments_ovary_detect/run_ovary_segm_evaluation.py --visual
```
1. In the end, cut individual segmented objects comes as minimal bounding box.
```bash
python experiments_ovary_detect/run_cut_segmented_objects.py \
-annot "./data-images/drosophila_ovary_slice/annot_eggs/*.png" \
-img "./data-images/drosophila_ovary_slice/segm/*.png" \
-out ./results/cut_images --padding 50
```
1. Finally, performing visualisation of segmentation results together with expert annotation.
```bash
python experiments_ovary_detect/run_export_user-annot-segm.py
```
______________________________________________________________________
## References
For complete references see [BibTex](docs/references.bib).
1. Borovec J., Svihlik J., Kybic J., Habart D. (2017). **Supervised and unsupervised segmentation using superpixels, model estimation, and Graph Cut.** SPIE Journal of Electronic Imaging 26(6), 061610. [DOI: 10.1117/1.JEI.26.6.061610](http://doi.org/10.1117/1.JEI.26.6.061610).
1. Borovec J., Kybic J., Nava R. (2017) **Detection and Localization of Drosophila Egg Chambers in Microscopy Images.** In: Wang Q., Shi Y., Suk HI., Suzuki K. (eds) Machine Learning in Medical Imaging. MLMI 2017. LNCS, vol 10541. Springer, Cham. [DOI: 10.1007/978-3-319-67389-9_3](http://doi.org/10.1007/978-3-319-67389-9_3).
1. Borovec J., Kybic J., Sugimoto, A. (2017). **Region growing using superpixels with learned shape prior.** SPIE Journal of Electronic Imaging 26(6), 061611. [DOI: 10.1117/1.JEI.26.6.061611](http://doi.org/10.1117/1.JEI.26.6.061611).