{"id":20058694,"url":"https://github.com/roboflow/cvevals","last_synced_at":"2025-08-18T05:43:42.984Z","repository":{"id":162241088,"uuid":"635230902","full_name":"roboflow/cvevals","owner":"roboflow","description":"Evaluate the performance of computer vision models and prompts for zero-shot models (Grounding DINO, CLIP, BLIP, DINOv2, ImageBind, models hosted on Roboflow)","archived":false,"fork":false,"pushed_at":"2023-10-18T00:32:39.000Z","size":1047,"stargazers_count":35,"open_issues_count":11,"forks_count":4,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-05-04T23:52:18.974Z","etag":null,"topics":["computer-vision","model-evaluation"],"latest_commit_sha":null,"homepage":"https://roboflow.github.io/cvevals/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/roboflow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-02T08:49:16.000Z","updated_at":"2025-01-27T12:54:36.000Z","dependencies_parsed_at":null,"dependency_job_id":"0d895157-af00-4b62-ae63-a4a6bb6ba0a8","html_url":"https://github.com/roboflow/cvevals","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roboflow%2Fcvevals","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roboflow%2Fcvevals/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roboflow%2Fcvevals/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/roboflow%2Fcvevals/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/roboflow","download_url":"https://codeload.github.com/roboflow/cvevals/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252514993,"owners_count":21760484,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","model-evaluation"],"created_at":"2024-11-13T13:03:14.370Z","updated_at":"2025-05-05T14:31:33.124Z","avatar_url":"https://github.com/roboflow.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![CV Evaluations banner](images/model-eval-banner.png)\n\n# CV Evaluations 🔎\n\n`cvevals` is a framework for evaluating the results of computer vision models. \n\nRoboflow `evaluations` is a Python package for evaluating computer vision models.\n\nUsing evaluations, you can:\n\n1. Compare ground truth to a Roboflow model to benchmark and visualize model performance on images in bulk;\n2. Test different Grounding DINO prompts to see which one most effectively annotates a specified class in an image;\n3. Test different CLIP prompts to see which one most effectively classifies an image, and;\n4. Evaluate resuts of different confidence levels for active learning.\n\nPerformance is measured using an aggregate of the following metrics:\n\n- **Precision**\n- **Recall**\n- **F1 Score**\n\nThe following data formats are supported:\n\n- YOLOv5 PyTorch TXT (object detection)\n- Multiclass Classification TXT (classification)\n- Classification Folder (classification)\n\n## Getting Started\n\nTo get started, clone the repository and install the required dependencies:\n\n```bash\n\ngit clone https://github.com/roboflow/cvevals.git\ncd cvevals\npip install -r requirements.txt\npip install -e .\n```\n\nNow you're ready to use this package!\n\nOut of the box, we have created examples that let you evaluate the performance of the following models against your Roboflow datasets:\n\n- CLIP (Classification)\n- BLIP (Classification)\n- ALBEF (Classification)\n- Grounding DINO (Object Detection)\n- DINOv2 and SVM (Classification)\n- ImageBind (Classification)\n\n## License\n\nThis project is licensed under an [MIT License](LICENSE).\n\n## Contributing\n\nInterested in contributing to evaluations? Check out our [contributing guidelines](CONTRIBUTING.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Froboflow%2Fcvevals","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Froboflow%2Fcvevals","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Froboflow%2Fcvevals/lists"}