https://github.com/autodistill/autodistill-altclip
AltCLIP model for use with Autodistill.
https://github.com/autodistill/autodistill-altclip
altclip autodistill clip computer-vision
Last synced: 4 months ago
JSON representation
AltCLIP model for use with Autodistill.
- Host: GitHub
- URL: https://github.com/autodistill/autodistill-altclip
- Owner: autodistill
- License: apache-2.0
- Created: 2023-11-01T17:53:22.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2023-12-05T09:17:01.000Z (over 1 year ago)
- Last Synced: 2024-12-30T19:48:44.907Z (6 months ago)
- Topics: altclip, autodistill, clip, computer-vision
- Language: Python
- Homepage: https://docs.autodistill.com
- Size: 18.6 KB
- Stars: 0
- Watchers: 4
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Autodistill AltCLIP Module
This repository contains the code supporting the AltCLIP base model for use with [Autodistill](https://github.com/autodistill/autodistill).
[AltCLIP](https://arxiv.org/abs/2211.06679v2) is a multi-modal vision model. With AltCLIP, you can compare the similarity between text and images, or the similarlity between two images. AltCLIP was trained on multi-lingual text-image pairs, which means it can be used for zero-shot classification with text prompts in different languages. [Read the AltCLIP paper for more information](https://arxiv.org/pdf/2211.06679v2.pdf).
The Autodistill AltCLIP module enables you to use AltCLIP for zero-shot classification.
Read the full [Autodistill documentation](https://autodistill.github.io/autodistill/).
Read the [CLIP Autodistill documentation](https://autodistill.github.io/autodistill/base_models/clip/).
## Installation
To use AltCLIP with autodistill, you need to install the following dependency:
```bash
pip3 install autodistill-altclip
```
## Quickstart
```python
from autodistill_altclip import AltCLIP
from autodistill.detection import CaptionOntology
# define an ontology to map class names to our AltCLIP prompt
# the ontology dictionary has the format {caption: class}
# where caption is the prompt sent to the base model, and class is the label that will
# be saved for that caption in the generated results
# then, load the model
base_model = AltCLIP(
ontology=CaptionOntology(
{
"person": "person",
"a forklift": "forklift"
}
)
)
results = base_model.predict("construction.jpg")
print(results)
```
## License
The AltCLIP model is licensed under an [Apache 2.0 license](LICENSE). See the [model README](https://github.com/FlagAI-Open/FlagAI/blob/master/examples/AltCLIP/README.md) for more information.
## 🏆 Contributing
We love your input! Please see the core Autodistill [contributing guide](https://github.com/autodistill/autodistill/blob/main/CONTRIBUTING.md) to get started. Thank you 🙏 to all our contributors!