{"id":43355870,"url":"https://github.com/yassersouri/task-driven-object-detection","last_synced_at":"2026-02-02T03:04:53.672Z","repository":{"id":37593323,"uuid":"180765742","full_name":"yassersouri/task-driven-object-detection","owner":"yassersouri","description":"Author's implementation of \"What Object Should I Use? - Task Driven Object Detection\" (CVPR 2019)","archived":false,"fork":false,"pushed_at":"2024-05-03T19:41:12.000Z","size":63,"stargazers_count":26,"open_issues_count":2,"forks_count":6,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-05-03T20:59:14.681Z","etag":null,"topics":["computer-vision","cvpr","cvpr2019","deep-learning"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/1904.03000","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yassersouri.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-04-11T10:05:47.000Z","updated_at":"2023-03-29T05:13:18.000Z","dependencies_parsed_at":"2022-09-01T09:12:17.268Z","dependency_job_id":null,"html_url":"https://github.com/yassersouri/task-driven-object-detection","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/yassersouri/task-driven-object-detection","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yassersouri%2Ftask-driven-object-detection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yassersouri%2Ftask-driven-object-detection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yassersouri%2Ftask-driven-object-detection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yassersouri%2Ftask-driven-object-detection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yassersouri","download_url":"https://codeload.github.com/yassersouri/task-driven-object-detection/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yassersouri%2Ftask-driven-object-detection/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29002651,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-02T01:32:03.847Z","status":"online","status_checked_at":"2026-02-02T02:00:07.448Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","cvpr","cvpr2019","deep-learning"],"created_at":"2026-02-02T03:04:53.052Z","updated_at":"2026-02-02T03:04:53.663Z","avatar_url":"https://github.com/yassersouri.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Task Driven Object Detection\n\nAuthor's implementation of the paper [\"What Object Should I Use? - Task Driven Object Detection\"](https://arxiv.org/abs/1904.03000).\n\nIf you use our code, please cite our paper.\n\n```latex\n@inproceedings{cocotasks2019,\n    Author    = {Sawatzky, Johann and Souri, Yaser and Grund, Christian and Gall, Juergen},\n    Title     = {{What Object Should I Use? - Task Driven Object Detection}},\n    Booktitle = {{CVPR}},\n    Year      = {2019}\n}\n```\n\nTable of Contents\n=================\n  * [Requirements](#requirements)\n     * [Installing COCO API](#installing-coco-api)\n  * [Getting The Data](#getting-the-data)\n     * [COCO Dataset](#coco-dataset)\n     * [COCO-Tasks Dataset](#coco-tasks-dataset)\n     * [Final Directory Structure](#final-directory-structure)\n  * [Reproducing Results](#reproducing-results)\n     * [General Information](#general-information)\n        * [Settings](#settings)\n        * [Seeds](#seeds)\n        * [Running on Detections vs. on Ground Truth Bounding Boxes](#running-on-detections-vs-on-ground-truth-bounding-boxes)\n        * [Changing the Detector](#changing-the-detector)\n     * [Baselines](#baselines)\n        * [1. Classifier Baseline](#1-classifier-baseline)\n        * [2. Ranker Baseline](#2-ranker-baseline)\n     * [Ablation Experiments and Proposed Method](#ablation-experiments-and-proposed-method)\n        * [1. Ablation: Joint Classifier](#1-ablation-joint-classifier)\n        * [2. Ablation: Joint Classifier   Class](#2-ablation-joint-classifier--class)\n        * [3. Ablation: Joint GGNN   Class (  Weighted Aggregation)](#3-ablation-joint-ggnn--class--weighted-aggregation)\n        * [4. Proposed Method (  Fusion)](#4-proposed-method--fusion)\n  * [Bugs, Feedback and Questions](#bugs-feedback-and-questions)\n\n## Requirements\n\nThe two main dependencies are:\n\n* Python 3.6\n* PyTorch 1.0.x\n\nOther dependencies are listed in the `requirements.txt` file.\n\n### Installing COCO API\n\nThere is only 1 special dependency and that is COCO's Python API. The code for the API is already included in the repository. You just need to do the following to compile it.\n\n```bash\ncd src/external/coco/PythonAPI\nmake\n```\n\n## Getting The Data\n\n### COCO Dataset\n\nWe need the train and validation COCO 2014 dataset. These can be downloaded from [here](http://cocodataset.org/#download). Or you can run the following commands to download them.\n\n```bash\nmkdir ~/mscoco  # I assume your main directory will be at your home directory.\n# what I usually do is, I symlink ~/mscoco to a place that has enough space.\nwget http://images.cocodataset.org/zips/train2014.zip\nwget http://images.cocodataset.org/zips/val2014.zip\nunzip train2014.zip -d ~/mscoco/\nunzip val2014.zip -d ~/mscoco/\n```\n\n### COCO-Tasks Dataset\n\nOther than the COCO dataset images, you need the COCO-Tasks dataset annotation. These can be downloaded from [here](https://coco-tasks.github.io/). Or you can run the following commands to download them. For the following command to run, you need [Git-LFS](https://git-lfs.github.com/) installed.\n\n```bash\ncd ~/mscoco  # The same root directory as above.\ngit clone -b cvpr2019 --depth 1 git@github.com:coco-tasks/dataset.git coco-tasks\n```\n\n### Final Directory Structure\n\nAt the end inside `~/mscoco` you should have a directory structure like this:\n\n```\n~/mscoco/\n├── train2014/\n│   ├── COCO_train2014_000000000009.jpg\n│   ├── COCO_train2014_000000000025.jpg\n│   └── ...\n├── val2014/\n│   ├── COCO_val2014_000000000042.jpg\n│   ├── COCO_val2014_000000000073.jpg\n│   └── ...\n└── coco-tasks/\n    ├── annotations/\n\t│   ├── task_1_test.json\n\t│   ├── task_1_train.json\n\t│   └── ...\n\t├── image_lists/\n    │   ├── imgIds_task_1_test.txt\n    │   ├── imgIds_task_1_train.txt\n    │   └── ...\n    ├── detection_faster.json\n    └── ...\n```\n\n## Reproducing Results\n\n### General Information\n\n#### Settings\n\nWe have a special settings file: `src/coco_tasks/settings.py`.\n\nThe main thing that is specified in the settings file is the location of stuff.\n\nHere is a description of the things that you might want to change:\n\n```python\nCOCO_ROOT  # location of the root folder for data\nTB_ROOT  # location that Tensorboard will write its data\nSAVING_DIRECTORY  # location where we will save trained models and results\n```\n\n#### Seeds\n\nWe tried for our results to be reproducible. We set the random seeds.\n\n```python\nrandom.seed(random_seed)\nnp.random.seed(random_seed)\ntorch.manual_seed(random_seed)\ntorch.cuda.manual_seed_all(random_seed)\n```\n\nBut this is not enough. There is still sources (e.g. cuDNN) that introduce stochasticity. To combat this, we run our training and testing 3 times with 3 different random seeds: 0, 1 and 2.\n\nOn all of the following training and testing scripts, one can easily set the seed using the `--random-seed 0` command line option. 0 is the default seed.\n\n#### Running on Detections vs. on Ground Truth Bounding Boxes\n\nAs mentioned in our paper, we train on ground truth bounding boxes and test in two settings: on detected bounding boxes (by default) and on ground truth bounding boxes (perfect detector setting).\n\nBy specifying `--test-on-gt True` you can perform the testing on the ground truth bounding box setting. This command line option works on all of the following python scripts. If you have already once run the code without this option, it means that you have already trained your model and tested it on detected bounding boxes. You can now only perform testing on ground truth bounding boxes by `--test-on-gt True --only-test True`. This will use the already trained models and only performs testing.\n\n#### Changing the Detector\n\nWe have trained Faster-RCNN and YOLOv2 of a subset of the COCO images that do not intersect with our test set. We them performed object detection on all images in our test set. The result is provided as part of our dataset release. If you prepared your data as I described above, they should be located at `~/mscoco/coco-tasks/detections_faster.json` and `~/mscoco/coco-tasks/detections_yolo.json`.\n\nBy default our code is set to use the Faster-RCNN detector, as specified in the `settings.py` file:\n\n```python\nCOCO_TASKS_TEST_DETECTIONS = os.path.join(COCO_TASKS_ROOT, \"detections_faster.json\")\n```\n\nYou can change that line in the `settings.py` file to use YOLO:\n\n```python\nCOCO_TASKS_TEST_DETECTIONS = os.path.join(COCO_TASKS_ROOT, \"detections_yolo.json\")\n```\n\n### Baselines\n\n#### 1. Classifier Baseline\n\nThis baseline corresponds to 4th row of Table 2 in our paper. Run the following\n\n```bash\npython src/single_task_classifier_baseline.py --task-number 1 --random-seed 0\n```\n\nThis is a single task baseline, which means the model is trained for each task separately.\n\nA good bash script to do all the training and testing would be:\n\n```bash\n#!/usr/bin/env bash\ntask_numbers=(1 2 3 4 5 6 7 8 9 10 11 12 13 14)\nseeds=(0 1 2)\nfor i in \"${task_numbers[@]}\";\ndo\n    for s in \"${seeds[@]}\";\n    do\n        python src/single_task_classifier_baseline.py --task-number ${i} --random-seed ${s}\n        python src/single_task_classifier_baseline.py --task-number ${i} --random-seed ${s} --test-on-gt True --only-test True\n    done\ndone\n```\n\n#### 2. Ranker Baseline\n\nThis baseline corresponds to 3rd row of Table 2 in our paper.\n\nThis baseline is different from all other baseline and methods in one major way: for training the ranker we have to use pairwise comparisons.\n\nTo generate these training pairs you need to first run the following script:\n\n```bash\npython src/create_pair_files_for_ranker.py\n```\n\nThen you can train the actual baseline:\n\n```bash\npython src/single_task_ranker_baseline.py --task-number 1 --seed 0\n```\n\n### Ablation Experiments and Proposed Method\n\n#### 1. Ablation: Joint Classifier\n\nThis ablation experiment corresponds to 2nd row of Table 3 in our paper: *(a) joint classifier*.\n\n```bash\npython src/ablation_joint_classifier.py --random-seed 0\n```\n\nThere is no need to run this script for tasks one by one, it will train for all tasks at the same time and will test for each task one by one.\n\n#### 2. Ablation: Joint Classifier + Class\n\nThis ablation experiment corresponds to 3rd row of Table 3 in our paper: *(b) joint classifier + class*.\n\n```bash\npython src/ablation_joint_classifier_withclass.py --random-seed 0\n```\n\n#### 3. Ablation: Joint GGNN + Class (+ Weighted Aggregation)\n\nThis ablation experiment corresponds to 4th and 5th rows of Table 3 in our paper: *(c) joint GGNN + class* and *(d) joint GGNN + class + w. aggreg.*\n\nNotice that the Weighted Aggregation only has an effect at test time. So if you train a model, you can test it with and without weighted aggregation.\n\nRun the code below to train a model for this ablation experiment and test it without weighted aggregation.\n\n```bash\npython src/ablation_ggnn.py --random-seed 0 --weighted-aggregation False\n```\n\nNow you can test it with weighted aggregation like this:\n\n```bash\npython src/ablation_ggnn.py --random-seed 0 --weighted-aggregation True --only-test True\n```\n\n#### 4. Proposed Method (+ Fusion)\n\nThis corresponds to our proposed method which are reported in 6th and 7th rows of Table 3 in our paper: *(e) proposed* and *(f) proposed + fusion*.\n\nNotice that fusion only has an effect at test time.\n\nTraining a model and testing it without fusion:\n\n```bash\npython src/ggnn_full.py --random-seed 0 --fusion none\n```\n\nTesting the trained model with fusion:\n\n```bash\npython src/ggnn_full.py --random-seed 0 --fusion avg\n```\n\n\n\n## Bugs, Feedback and Questions\n\nFeel free and open issues on this repository or contact me directly: souri@iai.uni-bonn.de\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyassersouri%2Ftask-driven-object-detection","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyassersouri%2Ftask-driven-object-detection","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyassersouri%2Ftask-driven-object-detection/lists"}