{"id":13645231,"url":"https://github.com/Cartucho/mAP","last_synced_at":"2025-04-21T13:32:25.374Z","repository":{"id":38238353,"uuid":"124381504","full_name":"Cartucho/mAP","owner":"Cartucho","description":"mean Average Precision - This code evaluates the performance of your neural net for object recognition.","archived":false,"fork":false,"pushed_at":"2024-08-15T13:09:32.000Z","size":15081,"stargazers_count":2921,"open_issues_count":98,"forks_count":909,"subscribers_count":38,"default_branch":"master","last_synced_at":"2024-10-29T15:03:16.806Z","etag":null,"topics":["average-precision","computer-vision","darkflow","darknet","detection","ground-truth","machine-learning","metrics","neural-network","object-detection","pascal-voc","yolo"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Cartucho.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-08T11:24:23.000Z","updated_at":"2024-10-28T08:30:17.000Z","dependencies_parsed_at":"2022-08-09T01:16:58.230Z","dependency_job_id":"627aac55-c904-4721-95e1-22e61dbd4e7b","html_url":"https://github.com/Cartucho/mAP","commit_stats":{"total_commits":99,"total_committers":12,"mean_commits":8.25,"dds":0.2222222222222222,"last_synced_commit":"3605865a350859e60c7b711838d09c4e0012c774"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cartucho%2FmAP","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cartucho%2FmAP/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cartucho%2FmAP/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cartucho%2FmAP/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Cartucho","download_url":"https://codeload.github.com/Cartucho/mAP/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223867903,"owners_count":17216975,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["average-precision","computer-vision","darkflow","darknet","detection","ground-truth","machine-learning","metrics","neural-network","object-detection","pascal-voc","yolo"],"created_at":"2024-08-02T01:02:31.672Z","updated_at":"2024-11-09T18:30:36.015Z","avatar_url":"https://github.com/Cartucho.png","language":"Python","funding_links":[],"categories":["Python","Object Detection Applications"],"sub_categories":[],"readme":"# mAP (mean Average Precision)\n\n[![GitHub stars](https://img.shields.io/github/stars/Cartucho/mAP.svg?style=social\u0026label=Stars)](https://github.com/Cartucho/mAP)\n\nThis code will evaluate the performance of your neural net for object recognition.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/15831541/37559643-6738bcc8-2a21-11e8-8a07-ed836f19c5d9.gif\" width=\"450\" height=\"300\" /\u003e\n\u003c/p\u003e\n\nIn practice, a **higher mAP** value indicates a **better performance** of your neural net, given your ground-truth and set of classes.\n\n## Citation\n\nThis project was developed for the following paper, please consider citing it:\n\n```bibtex\n@INPROCEEDINGS{8594067,\n  author={J. {Cartucho} and R. {Ventura} and M. {Veloso}},\n  booktitle={2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, \n  title={Robust Object Recognition Through Symbiotic Deep Learning In Mobile Robots}, \n  year={2018},\n  pages={2336-2341},\n}\n```\n\n## Table of contents\n\n- [Explanation](#explanation)\n- [Prerequisites](#prerequisites)\n- [Quick start](#quick-start)\n- [Running the code](#running-the-code)\n- [Authors](#authors)\n\n## Explanation\nThe performance of your neural net will be judged using the mAP criterium defined in the [PASCAL VOC 2012 competition](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/). We simply adapted the [official Matlab code](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit) into Python (in our tests they both give the same results).\n\nFirst (**1.**), we calculate the Average Precision (AP), for each of the classes present in the ground-truth. Finally (**2.**), we calculate the mAP (mean Average Precision) value.\n\n#### 1. Calculate AP\n\nFor each class:\n\nFirst, your neural net **detection-results** are sorted by decreasing confidence and are assigned to **ground-truth objects**. We have \"a match\" when they share the **same label and an IoU \u003e= 0.5** (Intersection over Union greater than 50%). This \"match\" is considered a true positive if that ground-truth object has not been already used (to avoid multiple detections of the same object). \n\n\u003cimg src=\"https://user-images.githubusercontent.com/15831541/37725175-45b9e1a6-2d2a-11e8-8c15-2fb4d716ca9a.png\" width=\"35%\" height=\"35%\" /\u003e\n\nUsing this criterium, we calculate the precision/recall curve. E.g:\n\n\u003cimg src=\"https://user-images.githubusercontent.com/15831541/43008995-64dd53ce-8c34-11e8-8a2c-4567b1311910.png\" width=\"45%\" height=\"45%\" /\u003e\n\nThen we compute a version of the measured precision/recall curve with **precision monotonically decreasing** (shown in light red), by setting the precision for recall `r` to the maximum precision obtained for any recall `r' \u003e r`.\n\nFinally, we compute the AP as the **area under this curve** (shown in light blue) by numerical integration.\nNo approximation is involved since the curve is piecewise constant.\n\n#### 2. Calculate mAP\n\nWe calculate the mean of all the AP's, resulting in an mAP value from 0 to 100%. E.g:\n\n\u003cimg src=\"https://user-images.githubusercontent.com/15831541/38933241-5f9556ae-4310-11e8-9d47-cb205f9b103b.png\"/\u003e\n\n\u003cimg src=\"https://user-images.githubusercontent.com/15831541/38933180-366b6fca-4310-11e8-99b9-17ad4b159b86.png\" /\u003e\n\n## Prerequisites\n\nYou need to install:\n- [Python](https://www.python.org/downloads/)\n\nOptional:\n- **plot** the results by [installing Matplotlib](https://matplotlib.org/users/installing.html) - Linux, macOS and Windows:\n    1. `python -mpip install -U pip`  \n    2.  `python -mpip install -U matplotlib`\n-  show **animation** by installing [OpenCV](https://www.opencv.org/):\n    1. `python -mpip install -U pip`\n    2. `python -mpip install -U opencv-python`  \n\n## Quick-start\nTo start using the mAP you need to clone the repo:\n\n```\ngit clone https://github.com/Cartucho/mAP\n```\n\n## Running the code\n\nStep by step:\n\n  1. [Create the ground-truth files](#create-the-ground-truth-files)\n  2. Copy the ground-truth files into the folder **input/ground-truth/**\n  3. [Create the detection-results files](#create-the-detection-results-files)\n  4. Copy the detection-results files into the folder **input/detection-results/**\n  5. Run the code:\n         ```\n         python main.py\n         ```\n\nOptional (if you want to see the **animation**):\n\n  6. Insert the images into the folder **input/images-optional/**\n\n\n#### PASCAL VOC, Darkflow and YOLO users\n\nIn the [scripts/extra](https://github.com/Cartucho/mAP/tree/master/scripts/extra) folder you can find additional scripts to convert **PASCAL VOC**, **darkflow** and **YOLO** files into the required format.\n\n#### Create the ground-truth files\n\n- Create a separate ground-truth text file for each image.\n- Use **matching names** for the files (e.g. image: \"image_1.jpg\", ground-truth: \"image_1.txt\").\n- In these files, each line should be in the following format:\n    ```\n    \u003cclass_name\u003e \u003cleft\u003e \u003ctop\u003e \u003cright\u003e \u003cbottom\u003e [\u003cdifficult\u003e]\n    ```\n- The `difficult` parameter is optional, use it if you want the calculation to ignore a specific detection.\n- E.g. \"image_1.txt\":\n    ```\n    tvmonitor 2 10 173 238\n    book 439 157 556 241\n    book 437 246 518 351 difficult\n    pottedplant 272 190 316 259\n    ```\n\n#### Create the detection-results files\n\n- Create a separate detection-results text file for each image.\n- Use **matching names** for the files (e.g. image: \"image_1.jpg\", detection-results: \"image_1.txt\").\n- In these files, each line should be in the following format:\n    ```\n    \u003cclass_name\u003e \u003cconfidence\u003e \u003cleft\u003e \u003ctop\u003e \u003cright\u003e \u003cbottom\u003e\n    ```\n- E.g. \"image_1.txt\":\n    ```\n    tvmonitor 0.471781 0 13 174 244\n    cup 0.414941 274 226 301 265\n    book 0.460851 429 219 528 247\n    chair 0.292345 0 199 88 436\n    book 0.269833 433 260 506 336\n    ```\n## Authors:\n* **João Cartucho**\n\n    Feel free to contribute\n\n    [![GitHub contributors](https://img.shields.io/github/contributors/Cartucho/mAP.svg)](https://github.com/Cartucho/mAP/graphs/contributors)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FCartucho%2FmAP","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FCartucho%2FmAP","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FCartucho%2FmAP/lists"}