https://github.com/bnbsking/cosmodules

Classification, Object detection, Segmentation MOduleS. Such as automatic format conversion, visualization, metrics with plotting and exporting suspected data.
https://github.com/bnbsking/cosmodules

classification conversion-tool metrics-exporter object-detection segmentation visualization-tools

Last synced: 12 days ago
JSON representation

Classification, Object detection, Segmentation MOduleS. Such as automatic format conversion, visualization, metrics with plotting and exporting suspected data.

• Host: GitHub
• URL: https://github.com/bnbsking/cosmodules
• Owner: bnbsking
• License: mit
• Created: 2022-02-11T07:44:09.000Z (almost 3 years ago)
• Default Branch: main
• Last Pushed: 2024-12-08T10:13:21.000Z (17 days ago)
• Last Synced: 2024-12-08T10:25:39.008Z (17 days ago)
• Topics: classification, conversion-tool, metrics-exporter, object-detection, segmentation, visualization-tools
• Language: Jupyter Notebook
• Homepage:
• Size: 85 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# COSMOduleS: Classification, Object detection, Segmentation MOduleS



  


  


  Github: source 

  



## **Introduction**

This repo provides comprehensive preprocessing and post-processing tools for common **Computer Vision** tasks.

| Tasks | Subtasks | Defined
Format | Visualization | Format
Conversion | Output
Analysis | Label
Merging | Active
Learning |

| - | - | - | - | - | - | - | - |

| Classification | binary1
 binary-bg2
 multi-class1
 multiclass-bg2
 multi-binary3
 | [single_label](https://github.com/bnbsking/COSMOduleS/blob/main/example/classification/data/single_label.json)1
 [single_label_bg](https://github.com/bnbsking/COSMOduleS/blob/main/example/classification/data/single_label_background.json)2
 [multi_label](https://github.com/bnbsking/COSMOduleS/blob/main/example/classification/data/multi_label.json)3 | - | - | [metrics
plotting
export](https://github.com/bnbsking/COSMOduleS/blob/main/example/classification/s3_output_analysis.ipynb) | [ALL](https://github.com/bnbsking/COSMOduleS/blob/main/example/classification/s4_label_merging.ipynb) | [Entropy](https://github.com/bnbsking/COSMOduleS/blob/main/example/classification/s5_active_learning.ipynb) |

| Detection      | - | [coco](https://github.com/bnbsking/COSMOduleS/blob/main/example/detection/data/coco)
 [voc](https://github.com/bnbsking/COSMOduleS/blob/main/example/detection/data/voc)
 [yolo](https://github.com/bnbsking/COSMOduleS/blob/main/example/detection/data/yolo)
 [**GENERAL**](https://github.com/bnbsking/COSMOduleS/blob/main/example/detection/data/general.json)
 | [ALL](https://github.com/bnbsking/COSMOduleS/blob/main/example/detection/s1_visualization_gt_and_pd.ipynb) | [between ANY
two types](https://github.com/bnbsking/COSMOduleS/blob/main/example/detection/s2_format_conversion.ipynb) | [metrics
plotting
export](https://github.com/bnbsking/COSMOduleS/blob/main/example/detection/s3_output_analysis.ipynb) | [V](https://github.com/bnbsking/COSMOduleS/blob/main/example/detection/s4_label_merging.ipynb) | [horizontal
flip](https://github.com/bnbsking/COSMOduleS/blob/main/example/detection/s5_active_learning.ipynb) |

| Segmentation   | instance1
 semantic2
 | [coco](https://github.com/bnbsking/COSMOduleS/blob/main/example/segmentation/data/coco)1+2
 [**GENERAL**](https://github.com/bnbsking/COSMOduleS/blob/main/example/segmentation/data/general)1+2 | [ALL](https://github.com/bnbsking/COSMOduleS/blob/main/example/segmentation/s1_visualization_gt_and_pd.ipynb) | [coco2general](https://github.com/bnbsking/COSMOduleS/blob/main/example/segmentation/s2_format_conversion.ipynb) | [metrics
plotting
export](https://github.com/bnbsking/COSMOduleS/blob/main/example/segmentation/s3_output_analysis.ipynb) | - | [instance
semantic
](https://github.com/bnbsking/COSMOduleS/blob/main/example/segmentation/s5_active_learning.ipynb) |

+ "bg" means background. If there is background class, it must be class 0 in this repo.

+ Adding prediction results after the defined format can use the visualization and output analysis. All the formats with predictions are in `example/*/prediction`, e.g. [here](https://github.com/bnbsking/COSMOduleS/blob/main/example/detection/prediction/general.json).

## **Motivation**

+ **[Classification]** Complicated tasks

| task                                          | label idx min | compute class-0 metrics | threshold optimization | data format |  

| -                                             | -             | -                       | -                      | -           |

| binary classification                         | 0             | V                       | V                      | [single_label](https://github.com/bnbsking/COSMOduleS/blob/main/example/classification/data/single_label.json)       |

| binary classification (cls-0 background)      | 1             |                         | V                      | [single_label_background](https://github.com/bnbsking/COSMOduleS/blob/main/example/classification/data/single_label_background.json) |

| multi-class classification                    | 0             | V                       |                        | single_label |

| multi-class classification (cls-0 background) | 1             |                         | V                      | single_label_background |

| multi-label classification (cls-0 background) | 0             |                         | V                      | [multi_label](https://github.com/bnbsking/COSMOduleS/blob/main/example/classification/data/multi_label.json)   |

+ **[Classification]** threshold optimization

    + `multi-class classification (cls-0 background)` checks whether prob-cls-0 < threshold, if yes, the pd-cls is pd[1:].argmax()

    + `multi-class classification (cls-0 background)` and `multi-label classification (cls-0 background)` take the mean of all optimized threshold for each foreground class

+ **[Object Detection]** Develop a [**GENERAL**](https://github.com/bnbsking/COSMOduleS/blob/main/example/detection/data/general.json) format to be the most convenient.

The formats can be summarized as following:

| format | extension | files     | type  | box                      | disadvantage |

| -      | -         | -         | -     | -                        | -            |

| coco   | .json     | 1         | int   | (xmin, ymin, w, h)       | get label of an image |

| yolo   | .txt      | len(imgs) | float | (cx, cy, w/2, h/2)       | visualization, compute metrics, etc. |

| voc    | .xml      | len(imgs) | int   | (xmin, ymin, xmax, ymax) | get class list |

| general| .json | 1 | int | (xmin, ymin, xmax, ymax) | **NO** |

+ **[Segmentation]** Develop a [**GENERAL**](https://github.com/bnbsking/COSMOduleS/blob/main/example/segmentation/data/general) format to be the most convenient.

| Includes         | Content | Advantage |

| -                | -       | -         |

| general.json     | Includes every imgs: path, contour, filled and boxes with class | Searching | 

| gt_contour_*.npy | (H, W) with {0, 1, ..., num_classes} int | Plotting |

| gt_filled_*.npy  | (num_classes, H, W) with 0 or 1 int values | Compute IOU for Metrics |

| *.jpg            | Raw data | - |

+ Segmentation prediction format: `(num_classes, H, W) with 0~1 float values (probability)`. e.g. [here](https://github.com/bnbsking/COSMOduleS/blob/main/example/segmentation/prediction/instance/pd_filled_img1.npy)

## **Installation**

```bash

pip install cosmodules

```

or

```bash

git clone https://github.com/bnbsking/COSMOduleS.git

pip install -e .

```

## **Quick Start - Classification**

+ Output Analysis:

    + Please conform your data format as either of one

        + [single_label](https://github.com/bnbsking/COSMOduleS/blob/main/example/classification/data/single_label.json)

        + [multilabel](https://github.com/bnbsking/COSMOduleS/blob/main/example/classification/data/multi_label.json)

        + [single_label_background](https://github.com/bnbsking/COSMOduleS/blob/main/example/classification/data/single_label_background.json)

    + The analysis pipeline is at [here](https://github.com/bnbsking/COSMOduleS/blob/main/cosmodules/classification/output_analysis.yaml)

    + See more in the [example](https://github.com/bnbsking/COSMOduleS/blob/main/example/classification/s3_output_analysis.ipynb)

```python

from cosmodules.classification import ClassificationAnalysis

ClassificationAnalysis(

    ant_path = "example/classification/data/single_label.json",

    save_folder = "example/classification/output/single_label",

)

```

+ Label Merging:

```python

from cosmodules.classification import ClassificationLabelMerging

ClassificationLabelMerging(

    cfg_path_list = [

        "example/classification/data/single_label.json",

        "example/classification/data_another_labeler/single_label.json",

    ],

    save_path = f"example/classification/output/label_merging/single_label.json"

)

```

+ Active Learning (see more in the [example](https://github.com/bnbsking/COSMOduleS/blob/main/example/classification/s5_active_learning.ipynb)):

```python

from cosmodules.classification import ClassificationActiveLearning

ClassificationActiveLearning(

    pred_path = "example/classification/prediction/single_label.json",

    save_path = "example/classification/output/active_learning/single_label.json",

    loss_name = "entropy"

)

```

## **Quick Start - Object detection**

+ Format Conversion (see more in the [example](https://github.com/bnbsking/COSMOduleS/blob/main/example/detection/s2_format_conversion.ipynb))

```python

from cosmodules.detection import coco2any

coco2any(

    tgt_foramt = "voc",

    img_folder = "example/detection/data/coco",

    ant_path = "example/detection/data/coco/coco.json",

    save_folder = "example/detection/output/visualization_gt_conversion/coco2voc"

)

```

or 

```python

from cosmodules.detection import coco2general

coco2general(

    img_folder = "example/detection/data/coco",

    ant_path = "example/detection/data/coco/coco.json",

    save_path = "example/detection/output/visualization_gt_conversion/coco2general/general.json"

)

```

+ Visualization (see more in the [example](https://github.com/bnbsking/COSMOduleS/blob/main/example/detection/s1_visualization_gt_and_pd.ipynb))

```python

from cosmodules.detection import show_coco

show_coco(

    img_name = "pic0.jpg",

    img_folder = "example/detection/data/coco",

    ant_path = "example/detection/data/coco/coco.json"

)

```

or

```python

from cosmodules.detection import show_general

show_general(

    img_name = "pic0.jpg",

    ant_path = "example/detection/data/general.json",

)  # when the anntotation includes predictions it will be shown!

```

+ Output Analysis

    + Please use the above `Format conversion` to change data format as [general](https://github.com/bnbsking/COSMOduleS/blob/main/example/detection/data/general.json)

    + The analysis pipeline is at [here](https://github.com/bnbsking/COSMOduleS/blob/main/cosmodules/detection/output_analysis.yaml)

```python

from cosmodules.detection import DetectionAnalysis

DetectionAnalysis(

    ant_path = "example/detection/data/general.json",

    save_folder = "example/detection/output/metrics"

)

```

+ Label Merging:

```python

from cosmodules.detection import DetectionLabelMerging

DetectionLabelMerging(

    cfg_path_list = [

        "example/detection/data/general.json",

        "example/detection/data_another_labeler/general.json",

    ],

    save_path = "example/detection/output/label_merging/general.json",

    ties_handling = "union"

)

```

+ Active Learning:

```python

from cosmodules.detection import DetectionActiveLearningByHFlip

DetectionActiveLearningByHFlip(

    pred_path_1 = f"{ROOT}/example/detection/prediction/general.json",

    pred_path_2 = f"{ROOT}/example/detection/prediction/general_horizontal_flip.json",

    save_path = f"{ROOT}/example/detection/output/active_learning/general.json"

)

```

## **Quick Start - Segmentation**

+ Format Conversion (see more in the [example](https://github.com/bnbsking/COSMOduleS/blob/main/example/segmentation/s2_format_conversion.ipynb))

```python

from cosmodules.segmentation import coco2general

coco2general(

    img_folder = "example/segmentation/data/coco",

    ant_path = "example/segmentation/data/coco/coco.json",

    save_folder = f"example/segmentation/output/visualization_gt_conversion/coco2general"

)

```

+ Visualization (see more in the [example](https://github.com/bnbsking/COSMOduleS/blob/main/example/segmentation/s1_visualization_gt_and_pd.ipynb))

```python

from cosmodules.segmentation import show_coco

show_coco(

    img_name = "img1.jpg",

    img_folder = "example/segmentation/data/coco",

    ant_path = "example/segmentation/data/coco/coco.json"

)   # when the anntotation includes predictions it will be shown!

```

or

```python

from cosmodules.segmentation import show_general

show_general(

    img_name = "img1.jpg",

    ant_path = "example/segmentation/data/general/general.json"

)

```

+ Output Analysis

    + Please use the above `Format conversion` to change data format as [general](https://github.com/bnbsking/COSMOduleS/blob/main/example/segmentation/data/general)

    + The analysis pipeline is at [here](https://github.com/bnbsking/COSMOduleS/blob/main/cosmodules/segmentation/output_analysis_instance.yaml)

```python

from cosmodules.segmentation import SegmentationAnalysis

SegmentationAnalysis(

    ant_path = "example/segmentation/prediction/instance/general.json",

    save_folder = "example/segmentation/output/metrics/instance",

    task = "instance",

)

```

or

```Python

from cosmodules.segmentation import SegmentationAnalysis

SegmentationAnalysis(

    ant_path = "example/segmentation/prediction/semantic/general.json",

    save_folder = "example/segmentation/output/metrics/semantic",

    task = "semantic"

)

```

+ Active Learning:

```python

from cosmodules.segmentation import (

    InstanceSegmentationActiveLearningByHFlip,

    SemanticSegmentationActiveLearning

)

InstanceSegmentationActiveLearningByHFlip(

    pred_path_1 = "example/segmentation/prediction/instance/general.json",

    pred_path_2 = "example/segmentation/prediction/instance_horizontal_flip/general.json",

    save_path = "example/segmentation/output/active_learning/instance.json"

)

```

or

```python

SemanticSegmentationActiveLearning(

    pred_path = "example/segmentation/prediction/semantic/general.json",

    save_path = "example/segmentation/output/active_learning/semantic.json",

    loss_name = "entropy"

)

```

## **Examples**

+ **[detection]**: format conversion workflow

![.](https://github.com/bnbsking/COSMOduleS/blob/main/pictures/detection_workflow.png)

+ detection visualization

![.](https://github.com/bnbsking/COSMOduleS/blob/main/pictures/detection_visualization.jpg)

+ confusion

![.](https://github.com/bnbsking/COSMOduleS/blob/main/pictures/confusion.jpg)

+ prf curves

![.](https://github.com/bnbsking/COSMOduleS/blob/main/pictures/prf_curves.jpg)

## **More**

+ Feel free to ask if you have any question.

+ Notice not supported

    + segmentation general2coco

    + segmentation label merging

## **Acknowledgement**

+ Confusion Matrix reference [here](https://github.com/kaanakan/object_detection_confusion_matrix/blob/master/confusion_matrix.py)