https://github.com/kupynorest/instance_augmentation
[ECCV 2024] Official Repo for: Dataset Enhancement with Instance-Level Augmentations
https://github.com/kupynorest/instance_augmentation
augmentation computer-vision diffusion-models eccv2024 image-generation image-processing machine-learning pytorch
Last synced: 2 months ago
JSON representation
[ECCV 2024] Official Repo for: Dataset Enhancement with Instance-Level Augmentations
- Host: GitHub
- URL: https://github.com/kupynorest/instance_augmentation
- Owner: KupynOrest
- License: mit
- Created: 2023-12-12T13:09:38.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-09-02T12:55:41.000Z (about 1 year ago)
- Last Synced: 2025-04-20T10:40:33.785Z (6 months ago)
- Topics: augmentation, computer-vision, diffusion-models, eccv2024, image-generation, image-processing, machine-learning, pytorch
- Language: Python
- Homepage:
- Size: 2.46 MB
- Stars: 44
- Watchers: 1
- Forks: 4
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Dataset Enhancement with Instance-Level Augmentations
[](https://arxiv.org/abs/2406.08249)
[](https://eccv2024.ecva.net/)
[](https://www.robots.ox.ac.uk/~vgg/research/instance-augmentation/)
This is an official repository for the paper
```
Dataset Enhancement with Instance-Level Augmentations
Orest Kupyn, Christian Rupprecht
ECCV 2024
```
Instance Augmentation method augment images by redrawing individual objects in the scene retaining their original shape. This allows training with the unchanged class label (e.g. class, segmentation, detection, etc.). The generations are highly diverse and match the scene composition
Original | Augmented | Augmented
:-------------------------:|:-------------------------:|:-------------------------:
 |  | 
# Augmented Datasets
This repository contains links to several augmented datasets that can be used for various computer vision tasks, such as object detection, instance segmentation, and saliency detection.
## Datasets
1. **COCO Augmented Car**:
- Link: [https://thor.robots.ox.ac.uk/instance-augmentation/COCO_Augmented_Car.tar](https://thor.robots.ox.ac.uk/instance-augmentation/COCO_Augmented_Car.tar)
- Description: An anonymized version of the COCO dataset, focusing on the "car" class.
2. **COCO Augmented People**:
- Link: [https://thor.robots.ox.ac.uk/instance-augmentation/COCO_Augmented_People.tar](https://thor.robots.ox.ac.uk/instance-augmentation/COCO_Augmented_People.tar)
- Description: An anonymized verison of the COCO dataset, focusing on the "person" class.
3. **COCO Augmented**:
- Link: [https://thor.robots.ox.ac.uk/instance-augmentation/COCO_Augmented.tar](https://thor.robots.ox.ac.uk/instance-augmentation/COCO_Augmented.tar)
- Description: An augmented version of the entire COCO dataset.
4. **DUTS Augmented**:
- Link: [https://thor.robots.ox.ac.uk/instance-augmentation/DUTS_Augmented.tar](https://thor.robots.ox.ac.uk/instance-augmentation/DUTS_Augmented.tar)
- Description: An augmented version of the DUTS dataset, which is commonly used for saliency detection.
5. **DUTS SDXL (Experimental)**:
- Link: [https://thor.robots.ox.ac.uk/instance-augmentation/DUTS_SDXL.tar](https://thor.robots.ox.ac.uk/instance-augmentation/DUTS_SDXL.tar)
- Description: A larger, augmented version of the DUTS dataset.
6. **SHA512 Checksums**:
- Link: [https://thor.robots.ox.ac.uk/instance-augmentation/SHA512SUMS](https://thor.robots.ox.ac.uk/instance-augmentation/SHA512SUMS)
- Description: A file containing the SHA512 checksums for the above augmented datasets, which can be used to verify the integrity of the downloaded files.
Please note that these augmented datasets are provided for research purposes. If you plan to use these datasets in your projects, make sure to follow the appropriate licensing and citation requirements.
## Installation
The code uses **Python 3.8**.
#### Create a Conda virtual environment and Install The Package:
Make sure you have Conda installed.
```bash
make env
```
#### Run Test for the Package:
```bash
make pytest
```
#### Run on a folder of images:
An example is available in tests/test_pipeline.py - test_end_to_end
To predict instance masks:
```python
from instance_augmentation.pseudolabel_dataset import create_annotations
create_annotations("path_to_image_folder", "path_to_save_results", dataset_type="custom", class_names=["dog", "cat", "any_other_classes"])
```
To generate augmented images:
```python
from instance_augmentation.pipeline.dataset_generator import DatasetGenerator
from instance_augmentation.pipeline.readers import CustomDatasetReader
reader = CustomDatasetReader("path_to_image_folder", {}, "path_to_save_results/annotations.json")
dataset_generator = DatasetGenerator.from_params(
dataset_reader=reader,
save_folder="path_to_save_results",
preprocessing="resize",
target_image_size=1024,
base_inpainting_model="SG161222/RealVisXL_V3.0",
generator="inpaint_sdxl_adapter",
num_samples=1,
num_inference_steps=20,
control_methods=["t2i_depth", "t2i_sketch"],
control_weights=[0.9, 0.5],
)
dataset_generator.run()
```
To apply augmentations:
```python
import os
import cv2
import glob
from instance_augmentation.augment import Augmenter
augmenter = Augmenter("path_to_save_results", p=1.0)
for image_path in glob.glob("path_to_image_folder/*"):
image_name = os.path.split(image_path)[1]
original_image = cv2.cvtColor(cv2.imread(image_path), cv2.COLOR_BGR2RGB)
augmented_image = augmenter.augment_image(original_image, image_name)
```
## Citation
If you use the the method or this code - implicitly or explicitly - for your research projects, please cite the following paper:
```
@article{kupyn2024dataset,
title = {Dataset Enhancement with Instance-Level Augmentations},
author = {Kupyn, Orest and Rupprecht, Christian},
journal = {arXiv preprint arXiv:2406.08249},
year = {2024}
}
```