{"id":13415375,"url":"https://github.com/facebookresearch/detr","last_synced_at":"2025-12-30T16:52:50.621Z","repository":{"id":37393094,"uuid":"267176885","full_name":"facebookresearch/detr","owner":"facebookresearch","description":"End-to-End Object Detection with Transformers","archived":true,"fork":false,"pushed_at":"2024-03-12T15:58:25.000Z","size":13185,"stargazers_count":14718,"open_issues_count":255,"forks_count":2601,"subscribers_count":148,"default_branch":"main","last_synced_at":"2025-09-25T12:36:18.094Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/facebookresearch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2020-05-26T23:54:52.000Z","updated_at":"2025-09-25T06:47:53.000Z","dependencies_parsed_at":"2023-02-19T16:31:12.814Z","dependency_job_id":"5dd49d8e-aeb9-43d3-922e-27d2fe8ea38b","html_url":"https://github.com/facebookresearch/detr","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/facebookresearch/detr","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2Fdetr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2Fdetr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2Fdetr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2Fdetr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/facebookresearch","download_url":"https://codeload.github.com/facebookresearch/detr/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookresearch%2Fdetr/sbom","scorecard":{"id":390932,"data":{"date":"2025-08-11","repo":{"name":"github.com/facebookresearch/detr","commit":"29901c51d7fe8712168b8d0d64351170bc0f83e0"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.9,"checks":[{"name":"Maintained","score":0,"reason":"project is archived","details":["Warn: Repository is archived."],"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":9,"reason":"Found 26/28 approved changesets -- score normalized to 9","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: containerImage not pinned by hash: Dockerfile:1: pin your Docker image by updating pytorch/pytorch:1.5-cuda10.1-cudnn7-runtime to pytorch/pytorch:1.5-cuda10.1-cudnn7-runtime@sha256:1f79d91afb716ef6da8f167829c5fd101959b284028435b92fa3778719a0439b","Info:   0 out of   1 containerImage dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Security-Policy","score":9,"reason":"security policy file detected","details":["Info: security policy file detected: github.com/facebookresearch/.github/SECURITY.md:1","Info: Found linked content: github.com/facebookresearch/.github/SECURITY.md:1","Warn: One or no descriptive hints of disclosure, vulnerability, and/or timelines in security policy","Info: Found text in security policy: github.com/facebookresearch/.github/SECURITY.md:1"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":0,"reason":"16 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-6rq9-53c3-f7vj","Warn: Project is vulnerable to: PYSEC-2023-38 / GHSA-ffxj-547x-5j7c","Warn: Project is vulnerable to: PYSEC-2025-10 / GHSA-h36j-8vv3-cj52","Warn: Project is vulnerable to: PYSEC-2024-223 / GHSA-h8wv-9h96-m4hr","Warn: Project is vulnerable to: PYSEC-2024-222 / GHSA-whh8-fjgc-qp73","Warn: Project is vulnerable to: PYSEC-2019-156 / GHSA-xp76-357g-9wqq","Warn: Project is vulnerable to: PYSEC-2023-102","Warn: Project is vulnerable to: PYSEC-2023-114","Warn: Project is vulnerable to: GHSA-3749-ghw9-m3mg","Warn: Project is vulnerable to: PYSEC-2022-43015 / GHSA-47fc-vmwq-366v","Warn: Project is vulnerable to: PYSEC-2025-41 / GHSA-53q9-r3pm-6pq6","Warn: Project is vulnerable to: PYSEC-2024-252 / GHSA-5pcm-hx3q-hm94","Warn: Project is vulnerable to: GHSA-887c-mr87-cxwp","Warn: Project is vulnerable to: PYSEC-2024-251 / GHSA-pg7h-5qx3-wjr3","Warn: Project is vulnerable to: PYSEC-2024-250","Warn: Project is vulnerable to: PYSEC-2024-259"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 29 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-18T17:47:41.539Z","repository_id":37393094,"created_at":"2025-08-18T17:47:41.539Z","updated_at":"2025-08-18T17:47:41.539Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":277021374,"owners_count":25746550,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-26T02:00:09.010Z","response_time":78,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T21:00:47.808Z","updated_at":"2025-09-27T07:32:10.189Z","avatar_url":"https://github.com/facebookresearch.png","language":"Python","funding_links":[],"categories":["Frameworks","Python","图像数据与CV","对象检测、分割","Table of Contents","Repos"],"sub_categories":["网络服务_其他","DETR变种"],"readme":"**DE⫶TR**: End-to-End Object Detection with Transformers\n========\n\n[![Support Ukraine](https://img.shields.io/badge/Support-Ukraine-FFD500?style=flat\u0026labelColor=005BBB)](https://opensource.fb.com/support-ukraine)\n\nPyTorch training code and pretrained models for **DETR** (**DE**tection **TR**ansformer).\nWe replace the full complex hand-crafted object detection pipeline with a Transformer, and match Faster R-CNN with a ResNet-50, obtaining **42 AP** on COCO using half the computation power (FLOPs) and the same number of parameters. Inference in 50 lines of PyTorch.\n\n![DETR](.github/DETR.png)\n\n**What it is**. Unlike traditional computer vision techniques, DETR approaches object detection as a direct set prediction problem. It consists of a set-based global loss, which forces unique predictions via bipartite matching, and a Transformer encoder-decoder architecture. \nGiven a fixed small set of learned object queries, DETR reasons about the relations of the objects and the global image context to directly output the final set of predictions in parallel. Due to this parallel nature, DETR is very fast and efficient.\n\n**About the code**. We believe that object detection should not be more difficult than classification,\nand should not require complex libraries for training and inference.\nDETR is very simple to implement and experiment with, and we provide a\n[standalone Colab Notebook](https://colab.research.google.com/github/facebookresearch/detr/blob/colab/notebooks/detr_demo.ipynb)\nshowing how to do inference with DETR in only a few lines of PyTorch code.\nTraining code follows this idea - it is not a library,\nbut simply a [main.py](main.py) importing model and criterion\ndefinitions with standard training loops.\n\nAdditionnally, we provide a Detectron2 wrapper in the d2/ folder. See the readme there for more information.\n\nFor details see [End-to-End Object Detection with Transformers](https://ai.facebook.com/research/publications/end-to-end-object-detection-with-transformers) by Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko.\n\nSee our [blog post](https://ai.facebook.com/blog/end-to-end-object-detection-with-transformers/) to learn more about end to end object detection with transformers.\n# Model Zoo\nWe provide baseline DETR and DETR-DC5 models, and plan to include more in future.\nAP is computed on COCO 2017 val5k, and inference time is over the first 100 val5k COCO images,\nwith torchscript transformer.\n\n\u003ctable\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003ename\u003c/th\u003e\n      \u003cth\u003ebackbone\u003c/th\u003e\n      \u003cth\u003eschedule\u003c/th\u003e\n      \u003cth\u003einf_time\u003c/th\u003e\n      \u003cth\u003ebox AP\u003c/th\u003e\n      \u003cth\u003eurl\u003c/th\u003e\n      \u003cth\u003esize\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003eDETR\u003c/td\u003e\n      \u003ctd\u003eR50\u003c/td\u003e\n      \u003ctd\u003e500\u003c/td\u003e\n      \u003ctd\u003e0.036\u003c/td\u003e\n      \u003ctd\u003e42.0\u003c/td\u003e\n      \u003ctd\u003e\u003ca href=\"https://dl.fbaipublicfiles.com/detr/detr-r50-e632da11.pth\"\u003emodel\u003c/a\u003e\u0026nbsp;|\u0026nbsp;\u003ca href=\"https://dl.fbaipublicfiles.com/detr/logs/detr-r50_log.txt\"\u003elogs\u003c/a\u003e\u003c/td\u003e\n      \u003ctd\u003e159Mb\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003eDETR-DC5\u003c/td\u003e\n      \u003ctd\u003eR50\u003c/td\u003e\n      \u003ctd\u003e500\u003c/td\u003e\n      \u003ctd\u003e0.083\u003c/td\u003e\n      \u003ctd\u003e43.3\u003c/td\u003e\n      \u003ctd\u003e\u003ca href=\"https://dl.fbaipublicfiles.com/detr/detr-r50-dc5-f0fb7ef5.pth\"\u003emodel\u003c/a\u003e\u0026nbsp;|\u0026nbsp;\u003ca href=\"https://dl.fbaipublicfiles.com/detr/logs/detr-r50-dc5_log.txt\"\u003elogs\u003c/a\u003e\u003c/td\u003e\n      \u003ctd\u003e159Mb\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003eDETR\u003c/td\u003e\n      \u003ctd\u003eR101\u003c/td\u003e\n      \u003ctd\u003e500\u003c/td\u003e\n      \u003ctd\u003e0.050\u003c/td\u003e\n      \u003ctd\u003e43.5\u003c/td\u003e\n      \u003ctd\u003e\u003ca href=\"https://dl.fbaipublicfiles.com/detr/detr-r101-2c7b67e5.pth\"\u003emodel\u003c/a\u003e\u0026nbsp;|\u0026nbsp;\u003ca href=\"https://dl.fbaipublicfiles.com/detr/logs/detr-r101_log.txt\"\u003elogs\u003c/a\u003e\u003c/td\u003e\n      \u003ctd\u003e232Mb\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e3\u003c/th\u003e\n      \u003ctd\u003eDETR-DC5\u003c/td\u003e\n      \u003ctd\u003eR101\u003c/td\u003e\n      \u003ctd\u003e500\u003c/td\u003e\n      \u003ctd\u003e0.097\u003c/td\u003e\n      \u003ctd\u003e44.9\u003c/td\u003e\n      \u003ctd\u003e\u003ca href=\"https://dl.fbaipublicfiles.com/detr/detr-r101-dc5-a2e86def.pth\"\u003emodel\u003c/a\u003e\u0026nbsp;|\u0026nbsp;\u003ca href=\"https://dl.fbaipublicfiles.com/detr/logs/detr-r101-dc5_log.txt\"\u003elogs\u003c/a\u003e\u003c/td\u003e\n      \u003ctd\u003e232Mb\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\nCOCO val5k evaluation results can be found in this [gist](https://gist.github.com/szagoruyko/9c9ebb8455610958f7deaa27845d7918).\n\nThe models are also available via torch hub,\nto load DETR R50 with pretrained weights simply do:\n```python\nmodel = torch.hub.load('facebookresearch/detr:main', 'detr_resnet50', pretrained=True)\n```\n\n\nCOCO panoptic val5k models:\n\u003ctable\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: right;\"\u003e\n      \u003cth\u003e\u003c/th\u003e\n      \u003cth\u003ename\u003c/th\u003e\n      \u003cth\u003ebackbone\u003c/th\u003e\n      \u003cth\u003ebox AP\u003c/th\u003e\n      \u003cth\u003esegm AP\u003c/th\u003e\n      \u003cth\u003ePQ\u003c/th\u003e\n      \u003cth\u003eurl\u003c/th\u003e\n      \u003cth\u003esize\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003e0\u003c/th\u003e\n      \u003ctd\u003eDETR\u003c/td\u003e\n      \u003ctd\u003eR50\u003c/td\u003e\n      \u003ctd\u003e38.8\u003c/td\u003e\n      \u003ctd\u003e31.1\u003c/td\u003e\n      \u003ctd\u003e43.4\u003c/td\u003e\n      \u003ctd\u003e\u003ca href=\"https://dl.fbaipublicfiles.com/detr/detr-r50-panoptic-00ce5173.pth\"\u003edownload\u003c/a\u003e\u003c/td\u003e\n      \u003ctd\u003e165Mb\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e1\u003c/th\u003e\n      \u003ctd\u003eDETR-DC5\u003c/td\u003e\n      \u003ctd\u003eR50\u003c/td\u003e\n      \u003ctd\u003e40.2\u003c/td\u003e\n      \u003ctd\u003e31.9\u003c/td\u003e\n      \u003ctd\u003e44.6\u003c/td\u003e\n      \u003ctd\u003e\u003ca href=\"https://dl.fbaipublicfiles.com/detr/detr-r50-dc5-panoptic-da08f1b1.pth\"\u003edownload\u003c/a\u003e\u003c/td\u003e\n      \u003ctd\u003e165Mb\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e2\u003c/th\u003e\n      \u003ctd\u003eDETR\u003c/td\u003e\n      \u003ctd\u003eR101\u003c/td\u003e\n      \u003ctd\u003e40.1\u003c/td\u003e\n      \u003ctd\u003e33\u003c/td\u003e\n      \u003ctd\u003e45.1\u003c/td\u003e\n      \u003ctd\u003e\u003ca href=\"https://dl.fbaipublicfiles.com/detr/detr-r101-panoptic-40021d53.pth\"\u003edownload\u003c/a\u003e\u003c/td\u003e\n      \u003ctd\u003e237Mb\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\nCheckout our [panoptic colab](https://colab.research.google.com/github/facebookresearch/detr/blob/colab/notebooks/DETR_panoptic.ipynb)\nto see how to use and visualize DETR's panoptic segmentation prediction.\n\n# Notebooks\n\nWe provide a few notebooks in colab to help you get a grasp on DETR:\n* [DETR's hands on Colab Notebook](https://colab.research.google.com/github/facebookresearch/detr/blob/colab/notebooks/detr_attention.ipynb): Shows how to load a model from hub, generate predictions, then visualize the attention of the model (similar to the figures of the paper)\n* [Standalone Colab Notebook](https://colab.research.google.com/github/facebookresearch/detr/blob/colab/notebooks/detr_demo.ipynb): In this notebook, we demonstrate how to implement a simplified version of DETR from the grounds up in 50 lines of Python, then visualize the predictions. It is a good starting point if you want to gain better understanding the architecture and poke around before diving in the codebase.\n* [Panoptic Colab Notebook](https://colab.research.google.com/github/facebookresearch/detr/blob/colab/notebooks/DETR_panoptic.ipynb): Demonstrates how to use DETR for panoptic segmentation and plot the predictions.\n\n\n# Usage - Object detection\nThere are no extra compiled components in DETR and package dependencies are minimal,\nso the code is very simple to use. We provide instructions how to install dependencies via conda.\nFirst, clone the repository locally:\n```\ngit clone https://github.com/facebookresearch/detr.git\n```\nThen, install PyTorch 1.5+ and torchvision 0.6+:\n```\nconda install -c pytorch pytorch torchvision\n```\nInstall pycocotools (for evaluation on COCO) and scipy (for training):\n```\nconda install cython scipy\npip install -U 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'\n```\nThat's it, should be good to train and evaluate detection models.\n\n(optional) to work with panoptic install panopticapi:\n```\npip install git+https://github.com/cocodataset/panopticapi.git\n```\n\n## Data preparation\n\nDownload and extract COCO 2017 train and val images with annotations from\n[http://cocodataset.org](http://cocodataset.org/#download).\nWe expect the directory structure to be the following:\n```\npath/to/coco/\n  annotations/  # annotation json files\n  train2017/    # train images\n  val2017/      # val images\n```\n\n## Training\nTo train baseline DETR on a single node with 8 gpus for 300 epochs run:\n```\npython -m torch.distributed.launch --nproc_per_node=8 --use_env main.py --coco_path /path/to/coco \n```\nA single epoch takes 28 minutes, so 300 epoch training\ntakes around 6 days on a single machine with 8 V100 cards.\nTo ease reproduction of our results we provide\n[results and training logs](https://gist.github.com/szagoruyko/b4c3b2c3627294fc369b899987385a3f)\nfor 150 epoch schedule (3 days on a single machine), achieving 39.5/60.3 AP/AP50.\n\nWe train DETR with AdamW setting learning rate in the transformer to 1e-4 and 1e-5 in the backbone.\nHorizontal flips, scales and crops are used for augmentation.\nImages are rescaled to have min size 800 and max size 1333.\nThe transformer is trained with dropout of 0.1, and the whole model is trained with grad clip of 0.1.\n\n\n## Evaluation\nTo evaluate DETR R50 on COCO val5k with a single GPU run:\n```\npython main.py --batch_size 2 --no_aux_loss --eval --resume https://dl.fbaipublicfiles.com/detr/detr-r50-e632da11.pth --coco_path /path/to/coco\n```\nWe provide results for all DETR detection models in this\n[gist](https://gist.github.com/szagoruyko/9c9ebb8455610958f7deaa27845d7918).\nNote that numbers vary depending on batch size (number of images) per GPU.\nNon-DC5 models were trained with batch size 2, and DC5 with 1,\nso DC5 models show a significant drop in AP if evaluated with more\nthan 1 image per GPU.\n\n## Multinode training\nDistributed training is available via Slurm and [submitit](https://github.com/facebookincubator/submitit):\n```\npip install submitit\n```\nTrain baseline DETR-6-6 model on 4 nodes for 300 epochs:\n```\npython run_with_submitit.py --timeout 3000 --coco_path /path/to/coco\n```\n\n# Usage - Segmentation\n\nWe show that it is relatively straightforward to extend DETR to predict segmentation masks. We mainly demonstrate strong panoptic segmentation results.\n\n## Data preparation\n\nFor panoptic segmentation, you need the panoptic annotations additionally to the coco dataset (see above for the coco dataset). You need to download and extract the [annotations](http://images.cocodataset.org/annotations/panoptic_annotations_trainval2017.zip).\nWe expect the directory structure to be the following:\n```\npath/to/coco_panoptic/\n  annotations/  # annotation json files\n  panoptic_train2017/    # train panoptic annotations\n  panoptic_val2017/      # val panoptic annotations\n```\n\n## Training\n\nWe recommend training segmentation in two stages: first train DETR to detect all the boxes, and then train the segmentation head.\nFor panoptic segmentation, DETR must learn to detect boxes for both stuff and things classes. You can train it on a single node with 8 gpus for 300 epochs with:\n```\npython -m torch.distributed.launch --nproc_per_node=8 --use_env main.py --coco_path /path/to/coco  --coco_panoptic_path /path/to/coco_panoptic --dataset_file coco_panoptic --output_dir /output/path/box_model\n```\nFor instance segmentation, you can simply train a normal box model (or used a pre-trained one we provide).\n\nOnce you have a box model checkpoint, you need to freeze it, and train the segmentation head in isolation.\nFor panoptic segmentation you can train on a single node with 8 gpus for 25 epochs:\n```\npython -m torch.distributed.launch --nproc_per_node=8 --use_env main.py --masks --epochs 25 --lr_drop 15 --coco_path /path/to/coco  --coco_panoptic_path /path/to/coco_panoptic  --dataset_file coco_panoptic --frozen_weights /output/path/box_model/checkpoint.pth --output_dir /output/path/segm_model\n```\nFor instance segmentation only, simply remove the `dataset_file` and `coco_panoptic_path` arguments from the above command line.\n\n# License\nDETR is released under the Apache 2.0 license. Please see the [LICENSE](LICENSE) file for more information.\n\n# Contributing\nWe actively welcome your pull requests! Please see [CONTRIBUTING.md](.github/CONTRIBUTING.md) and [CODE_OF_CONDUCT.md](.github/CODE_OF_CONDUCT.md) for more info.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebookresearch%2Fdetr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffacebookresearch%2Fdetr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebookresearch%2Fdetr/lists"}