https://github.com/facebookresearch/omnivore

Omnivore: A Single Model for Many Visual Modalities
https://github.com/facebookresearch/omnivore

Last synced: 9 days ago
JSON representation

Omnivore: A Single Model for Many Visual Modalities

• Host: GitHub
• URL: https://github.com/facebookresearch/omnivore
• Owner: facebookresearch
• License: other
• Archived: true
• Created: 2022-01-20T01:24:47.000Z (almost 3 years ago)
• Default Branch: main
• Last Pushed: 2022-11-12T15:16:35.000Z (about 2 years ago)
• Last Synced: 2024-11-02T03:06:43.264Z (12 days ago)
• Language: Python
• Homepage:
• Size: 6.84 MB
• Stars: 558
• Watchers: 19
• Forks: 38
• Open Issues: 8
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project

README

        
# Omnivorous modeling for visual modalities

This repository contains PyTorch pretrained models, inference examples for the following papers:

  Omnivore A single vision model for many different visual modalities, CVPR 2022 [bib]

```

@inproceedings{girdhar2022omnivore,

  title={{Omnivore: A Single Model for Many Visual Modalities}},

  author={Girdhar, Rohit and Singh, Mannat and Ravi, Nikhila and van der Maaten, Laurens and Joulin, Armand and Misra, Ishan},

  booktitle={CVPR},

  year={2022}

}

```

OmniMAE Single Model Masked Pretraining on Images and Videos  [bib]

```

@article{girdhar2022omnimae,

  title={OmniMAE: Single Model Masked Pretraining on Images and Videos},

  author={Girdhar, Rohit and El-Nouby, Alaaeldin and Singh, Mannat and Alwala, Kalyan Vasudev and Joulin, Armand and Misra, Ishan},

  journal={arXiv preprint arXiv:2206.08356},

  year={2022}

}

```

OmniVision Our training pipeline supporting the multi-modal vision research.[bib]

## Contributing

We welcome your pull requests! Please see [CONTRIBUTING](CONTRIBUTING.md) and [CODE_OF_CONDUCT](CODE_OF_CONDUCT.md) for more information.

## License

Omnivore is released under the CC-BY-NC 4.0 license. See [LICENSE](LICENSE) for additional details. However the Swin Transformer implementation is additionally licensed under the Apache 2.0 license (see [NOTICE](NOTICE) for additional details).