{"id":20434196,"url":"https://github.com/vt-vl-lab/ican","last_synced_at":"2025-07-29T12:32:23.008Z","repository":{"id":104150612,"uuid":"146820698","full_name":"vt-vl-lab/iCAN","owner":"vt-vl-lab","description":"[BMVC 2018] iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection","archived":false,"fork":false,"pushed_at":"2020-08-24T06:42:33.000Z","size":26738,"stargazers_count":260,"open_issues_count":12,"forks_count":57,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-04-10T03:52:29.470Z","etag":null,"topics":["action-recognition","human-object-interaction","visual-relationship-detection"],"latest_commit_sha":null,"homepage":"https://gaochen315.github.io/iCAN/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vt-vl-lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-08-31T00:22:49.000Z","updated_at":"2024-11-21T07:56:19.000Z","dependencies_parsed_at":null,"dependency_job_id":"c779144f-e74b-4d15-9a4e-489a7484a509","html_url":"https://github.com/vt-vl-lab/iCAN","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/vt-vl-lab/iCAN","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vt-vl-lab%2FiCAN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vt-vl-lab%2FiCAN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vt-vl-lab%2FiCAN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vt-vl-lab%2FiCAN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vt-vl-lab","download_url":"https://codeload.github.com/vt-vl-lab/iCAN/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vt-vl-lab%2FiCAN/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267686337,"owners_count":24127715,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-29T02:00:12.549Z","response_time":2574,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["action-recognition","human-object-interaction","visual-relationship-detection"],"created_at":"2024-11-15T08:25:01.114Z","updated_at":"2025-07-29T12:32:22.972Z","avatar_url":"https://github.com/vt-vl-lab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"**This repository is no longer maintained.** I am no longer actively maintaining iCAN. Please refer to our ECCV 2020 work [DRG](https://github.com/vt-vl-lab/DRG) for a stronger HOI detection framework in PyTorch.\n\n\n# iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection \n\nOfficial TensorFlow implementation for [iCAN: Instance-Centric Attention Network \nfor Human-Object Interaction Detection](https://arxiv.org/abs/1808.10437).\n\nSee the [project page](https://gaochen315.github.io/iCAN/) for more details. Please contact Chen Gao (chengao@vt.edu) if you have any questions.\n\n\u003cimg src='misc/HOI.gif'\u003e\n\n## Prerequisites\n\nThis codebase was developed and tested with Python2.7, Tensorflow 1.1.0 or 1.2.0, CUDA 8.0 and Ubuntu 16.04.\n\n\n## Installation\n1. Clone the repository. \n    ```Shell\n    git clone https://github.com/vt-vl-lab/iCAN.git\n    ```\n2. Download V-COCO and HICO-DET dataset. Setup V-COCO and COCO API. Setup HICO-DET evaluation code.\n    ```Shell\n    chmod +x ./misc/download_dataset.sh \n    ./misc/download_dataset.sh \n    \n    # Assume you cloned the repository to `iCAN_DIR'.\n    # If you have downloaded V-COCO or HICO-DET dataset somewhere else, you can create a symlink\n    # ln -s /path/to/your/v-coco/folder Data/\n    # ln -s /path/to/your/hico-det/folder Data/\n    ```\n    \n## Evaluate V-COCO and HICO-DET detection results\n1. Download detection results\n    ```Shell\n    chmod +x ./misc/download_detection_results.sh \n    ./misc/download_detection_results.sh\n    ```\n2. Evaluate V-COCO detection results using iCAN\n    ```Shell\n    python tools/Diagnose_VCOCO.py eval Results/300000_iCAN_ResNet50_VCOCO.pkl\n    ```\n3. Evaluate V-COCO detection results using iCAN (Early fusion)\n    ```Shell\n    python tools/Diagnose_VCOCO.py eval Results/300000_iCAN_ResNet50_VCOCO_Early.pkl\n    ```\n3. Evaluate HICO-DET detection results using iCAN\n    ```Shell\n    cd Data/ho-rcnn\n    matlab -r \"Generate_detection; quit\"\n    cd ../../\n    ```\n    Here we evaluate our best detection results under ```Results/HICO_DET/1800000_iCAN_ResNet50_HICO```. If you want to evaluate a different detection result, please specify the filename in ```Data/ho-rcnn/Generate_detection.m``` accordingly.\n   \n## Error diagnose on V-COCO\n1. Diagnose V-COCO detection results using iCAN\n    ```Shell\n    python tools/Diagnose_VCOCO.py diagnose Results/300000_iCAN_ResNet50_VCOCO.pkl\n    ```\n2. Diagnose V-COCO detection results using iCAN (Early fusion)\n    ```Shell\n    python tools/Diagnose_VCOCO.py diagnose Results/300000_iCAN_ResNet50_VCOCO_Early.pkl\n    ```\n\n## Training\n1. Download COCO pre-trained weights and training data\n    ```Shell\n    chmod +x ./misc/download_training_data.sh \n    ./misc/download_training_data.sh\n    ```\n2. Train an iCAN on V-COCO\n    ```Shell\n    python tools/Train_ResNet_VCOCO.py --model iCAN_ResNet50_VCOCO --num_iteration 300000\n    ```\n3. Train an iCAN (Early fusion) on V-COCO\n    ```Shell\n    python tools/Train_ResNet_VCOCO.py --model iCAN_ResNet50_VCOCO_Early --num_iteration 300000\n4. Train an iCAN on HICO-DET\n    ```Shell\n    python tools/Train_ResNet_HICO.py --num_iteration 1800000\n    ```\n    \n## Testing\n1. Test an iCAN on V-COCO\n    ```Shell\n     python tools/Test_ResNet_VCOCO.py --model iCAN_ResNet50_VCOCO --num_iteration 300000\n    ```\n2. Test an iCAN (Early fusion) on V-COCO\n    ```Shell\n     python tools/Test_ResNet_VCOCO.py --model iCAN_ResNet50_VCOCO_Early --num_iteration 300000\n    ```\n3. Test an iCAN on HICO-DET\n    ```Shell\n    python tools/Test_ResNet_HICO.py --num_iteration 1800000\n    ```\n\n## Visualizing V-COCO detections\nCheck ```tools/Visualization.ipynb``` to see how to visualize the detection results.\n\n## Demo/Test on your own images\n0. To get the best performance, we use [Detectron](https://github.com/facebookresearch/Detectron) as our object detector. For a simple demo purpose, we use [tf-faster-rcnn](https://github.com/endernewton/tf-faster-rcnn) in this section instead.\n1. Clone and setup the tf-faster-rcnn repository.\n    ```Shell\n    cd $iCAN_DIR\n    chmod +x ./misc/setup_demo.sh \n    ./misc/setup_demo.sh\n    ```\n2. Put your own images to ```demo/``` folder.\n3. Detect all objects\n    ```Shell\n    # images are saved in $iCAN_DIR/demo/\n    python ../tf-faster-rcnn/tools/Object_Detector.py --img_dir demo/ --img_format png --Demo_RCNN demo/Object_Detection.pkl\n    ``` \n4. Detect all HOIs\n    ```Shell\n    python tools/Demo.py --img_dir demo/ --Demo_RCNN demo/Object_Detection.pkl --HOI_Detection demo/HOI_Detection.pkl\n    ```\n5. Check ```tools/Demo.ipynb``` to visualize the detection results.\n\n## Citation\nIf you find this code useful for your research, please consider citing the following papers:\n\n    @inproceedings{gao2018ican,\n    author    = {Gao, Chen and Zou, Yuliang and Huang, Jia-Bin}, \n    title     = {iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection}, \n    booktitle = {British Machine Vision Conference},\n    year      = {2018}\n    }\n\n## Acknowledgement\nCodes are built upon [tf-faster-rcnn](https://github.com/endernewton/tf-faster-rcnn). We thank [Jinwoo Choi](https://github.com/jinwchoi) for the code review.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvt-vl-lab%2Fican","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvt-vl-lab%2Fican","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvt-vl-lab%2Fican/lists"}