https://github.com/autodistill/autodistill-transformers
Use object detection models in Hugging Face Transformers to automatically label data to train a fine-tuned model.
https://github.com/autodistill/autodistill-transformers
computer-vision object-detection
Last synced: 5 months ago
JSON representation
Use object detection models in Hugging Face Transformers to automatically label data to train a fine-tuned model.
- Host: GitHub
- URL: https://github.com/autodistill/autodistill-transformers
- Owner: autodistill
- License: mit
- Created: 2023-11-03T20:49:41.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-12-05T09:25:13.000Z (over 2 years ago)
- Last Synced: 2025-01-30T21:06:04.544Z (over 1 year ago)
- Topics: computer-vision, object-detection
- Language: Python
- Homepage: https://docs.autodistill.com
- Size: 8.79 KB
- Stars: 1
- Watchers: 4
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Autodistill Transformers Module
This repository contains the code supporting the Transformers models model for use with [Autodistill](https://github.com/autodistill/autodistill).
[Transformers](https://github.com/huggingface/transformers), maintained by Hugging Face, features a range of state of the art models for Natural Language Processing (NLP), computer vision, and more.
This package allows you to write a function that calls a Transformers object detection model and use it to automatically label data. You can use this data to train a fine-tuned model using an architecture supported by Autodistill (i.e. [YOLOv8](https://github.com/autodistil/autodistill-yolov8), [YOLOv5](https://github.com/autodistil/autodistill-yolov5), or [DETR](https://github.com/autodistil/autodistill-detr)).
Read the full [Autodistill documentation](https://autodistill.github.io/autodistill/).
## Installation
To use Transformers with autodistill, you need to install the following dependency:
```bash
pip3 install autodistill-transformers
```
## Quickstart
The following example shows how to use the Transformers module to label images using the [Owlv2ForObjectDetection](https://huggingface.co/google/owlv2-large-patch14-ensemble) model.
You can update the `inference()` functon to use any object detection model supported in the Transformers library.
```python
import cv2
import torch
from autodistill.detection import CaptionOntology
from autodistill.utils import plot
from transformers import OwlViTForObjectDetection, OwlViTProcessor
from autodistill_transformers import TransformersModel
processor = OwlViTProcessor.from_pretrained("google/owlvit-base-patch32")
model = OwlViTForObjectDetection.from_pretrained("google/owlvit-base-patch32")
def inference(image, prompts):
inputs = processor(text=prompts, images=image, return_tensors="pt")
outputs = model(**inputs)
target_sizes = torch.Tensor([image.size[::-1]])
results = processor.post_process_object_detection(
outputs=outputs, target_sizes=target_sizes, threshold=0.1
)[0]
return results
base_model = TransformersModel(
ontology=CaptionOntology(
{
"a photo of a person": "person",
"a photo of a cat": "cat",
}
),
callback=inference,
)
# run inference
results = base_model.predict("image.jpg", confidence=0.1)
print(results)
# plot results
plot(
image=cv2.imread("image.jpg"),
detections=results,
classes=base_model.ontology.classes(),
)
# label a directory of images
base_model.label("./context_images", extension=".jpeg")
```
## License
This project is licensed under an [MIT license](LICENSE).
## 🏆 Contributing
We love your input! Please see the core Autodistill [contributing guide](https://github.com/autodistill/autodistill/blob/main/CONTRIBUTING.md) to get started. Thank you 🙏 to all our contributors!