https://github.com/retkowsky/florence-2

Florence-2
https://github.com/retkowsky/florence-2

azure florence-2

Last synced: 2 days ago
JSON representation

Florence-2

• Host: GitHub
• URL: https://github.com/retkowsky/florence-2
• Owner: retkowsky
• Created: 2024-06-19T21:46:54.000Z (8 months ago)
• Default Branch: main
• Last Pushed: 2025-01-15T09:53:43.000Z (26 days ago)
• Last Synced: 2025-01-15T11:35:23.838Z (26 days ago)
• Topics: azure, florence-2
• Language: Jupyter Notebook
• Homepage:
• Size: 25.3 MB
• Stars: 53
• Watchers: 3
• Forks: 11
• Open Issues: 5
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# florence-2

Florence-2 - Advancing a Unified Representation for a Variety of Vision Tasks

We introduce Florence-2, a novel vision foundation model with a unified, prompt-based representation for a variety of computer vision and vision-language tasks.

While existing large vision models excel in transfer learning, they struggle to perform a diversity of tasks with simple instructions, a capability that implies handling the complexity of various spatial hierarchy and semantic granularity.

Florence-2 was designed to take text-prompt as task instructions and generate desirable results in text forms, whether it be captioning, object detection, grounding or segmentation. This multi-task learning setup demands large-scale, high-quality annotated data.

To this end, we co-developed FLD-5B that consists of 5.4 billion comprehensive visual annotations on 126 million images, using an iterative strategy of automated image annotation and model refinement. We adopted a sequence-to-sequence structure to train Florence-2 to perform versatile and comprehensive vision tasks. Extensive evaluations on numerous tasks demonstrated Florence-2 to be a strong vision foundation model contender with unprecedented zero-shot and fine-tuning capabilities.

https://huggingface.co/papers/2311.06242

https://huggingface.co/collections/microsoft/florence-6669f44df0d87d9c3bfb76de