https://github.com/dusty-nv/clip_trt

CLIP and SigLIP models optimized with TensorRT with a Transformers-like API
https://github.com/dusty-nv/clip_trt

Last synced: about 2 months ago
JSON representation

CLIP and SigLIP models optimized with TensorRT with a Transformers-like API

• Host: GitHub
• URL: https://github.com/dusty-nv/clip_trt
• Owner: dusty-nv
• License: mit
• Created: 2024-06-09T02:59:38.000Z (7 months ago)
• Default Branch: main
• Last Pushed: 2024-08-19T09:11:44.000Z (5 months ago)
• Last Synced: 2024-08-19T10:51:43.277Z (5 months ago)
• Language: Python
• Size: 67.4 KB
• Stars: 11
• Watchers: 3
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.md

Awesome Lists containing this project

README

        
# clip_trt

CLIP and SigLIP models optimized with TensorRT with a Transformers-like API

### Command-Line Example

```bash

python3 -m clip_trt \

  --inputs image_a.jpg image_b.jpg image_c.jpg \

  --inputs 'a dog' 'a cat' 'a bear' 'a lion'

```

### Code Example

```python

from clip_trt import CLIPModel

model = CLIPModel.from_pretrained(

    "openai/clip-vit-large-patch14-336",

    use_tensorrt=True,

    crop=False,

)

similarity = model(

    [

        'my_image.jpg',

        PIL.Image.open('image_2.jpg').convert('RGB'),

        np.ndarray((3,480,640), dtype=np.uint8),

        torch.ones((3,336,336), dtype=torch.uint8, device='cuda')

    ],

    [

        'a dog', 'a cat', 'a bear', 'a lion'

    ],

)

```

### Embeddings

```

image_embed = model.embed('xyz.jpg')

text_embed = model.embed('an elephant in the jungle')

```