{"id":13936584,"url":"https://github.com/mfigurnov/sact","last_synced_at":"2025-07-19T22:31:01.862Z","repository":{"id":19134167,"uuid":"85963492","full_name":"mfigurnov/sact","owner":"mfigurnov","description":"Spatially Adaptive Computation Time for Residual Networks","archived":false,"fork":false,"pushed_at":"2023-03-24T22:20:45.000Z","size":945,"stargazers_count":245,"open_issues_count":3,"forks_count":55,"subscribers_count":13,"default_branch":"master","last_synced_at":"2024-08-08T23:24:02.105Z","etag":null,"topics":["tensorflow"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mfigurnov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-03-23T15:06:00.000Z","updated_at":"2024-03-13T14:58:54.000Z","dependencies_parsed_at":"2022-08-28T06:34:50.459Z","dependency_job_id":"38da3cbd-1e03-40c2-bce3-cd32e27a7cbf","html_url":"https://github.com/mfigurnov/sact","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mfigurnov%2Fsact","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mfigurnov%2Fsact/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mfigurnov%2Fsact/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mfigurnov%2Fsact/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mfigurnov","download_url":"https://codeload.github.com/mfigurnov/sact/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226686729,"owners_count":17666928,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["tensorflow"],"created_at":"2024-08-07T23:02:48.723Z","updated_at":"2024-11-27T04:31:13.338Z","avatar_url":"https://github.com/mfigurnov.png","language":"Python","funding_links":[],"categories":["Python","**Conditional (Adaptive) Computing**"],"sub_categories":["**Tensor Decomposition**"],"readme":"# Spatially Adaptive Computation Time for Residual Networks\n\nThis code implements a deep learning architecture based on Residual Network that dynamically adjusts the number of executed layers for the regions of the image.\nThe architecture is end-to-end trainable, deterministic and problem-agnostic.\nThe included code applies this to the CIFAR-10 an ImageNet image classification problems.\nIt is implemented using TensorFlow and TF-Slim.\n\nPaper describing the project:\n\nMichael Figurnov, Maxwell D. Collins, Yukun Zhu, Li Zhang, Jonathan Huang, Dmitry Vetrov, Ruslan Salakhutdinov. Spatially Adaptive Computation Time for Residual Networks. *CVPR 2017* [[arxiv]](https://arxiv.org/abs/1612.02297).\n\nImage (with detections)          | Ponder cost map\n:-------------------------------:|:--------------------------------------:\n![](pics/export-image-442041.jpg)|![](pics/export-image-442041-ponder.jpg)\n\n## Setup\n\nInstall prerequisites:\n\n``` bash\npip install -r requirements.txt  # CPU\npip install -r requirements-gpu.txt  # GPU\n```\n\nPrerequisite packages:\n - Python 2.x/3.x (mostly tested with Python 2.7)\n - Tensorflow 1.0\n - NumPy\n - (Optional) nose\n - (Optional) h5py\n - (Optional) matplotlib\n\nRun tests. It takes a couple of minutes:\n\n``` bash\nnosetests --logging-level=WARNING\n```\n\n## CIFAR-10\n\nDownload and convert CIFAR-10 dataset:\n\n``` bash\nPYTHONPATH=external python external/download_and_convert_cifar10.py --dataset_dir=\"${HOME}/tensorflow/data/cifar10\"\n```\n\nLet's train and continuously evaluate a CIFAR-10 Adaptive Computation Time model with five residual units per block (ResNet-32):\n\n``` bash\nexport ACT_LOGDIR='/tmp/cifar10_resnet_5_act_1e-2'\npython cifar_main.py --model_type=act --model=5 --tau=0.01 --train_log_dir=\"${ACT_LOGDIR}/train\" --save_summaries_secs=300 \u0026\npython cifar_main.py --model_type=act --model=5 --tau=0.01 --checkpoint_dir=\"${ACT_LOGDIR}/train\" --eval_dir=\"${ACT_LOGDIR}/eval\" --mode=eval\n```\n\nOr, for _spatially_ adaptive computation time (SACT):\n\n``` bash\nexport SACT_LOGDIR='/tmp/cifar10_resnet_5_sact_1e-2'\npython cifar_main.py --model_type=sact --model=5 --tau=0.01 --train_log_dir=\"${SACT_LOGDIR}/train\" --save_summaries_secs=300 \u0026\npython cifar_main.py --model_type=sact --model=5 --tau=0.01 --checkpoint_dir=\"${SACT_LOGDIR}/train\" --eval_dir=\"${SACT_LOGDIR}/eval\" --mode=eval\n```\n\nTo download and evaluate a [pretrained ResNet-32 SACT model](https://s3.us-east-2.amazonaws.com/sact-models/cifar10_resnet_5_sact_1e-2.tar.gz) (1.8 MB file):\n\n``` bash\nmkdir -p models \u0026\u0026 curl https://s3.us-east-2.amazonaws.com/sact-models/cifar10_resnet_5_sact_1e-2.tar.gz | tar xv -C models\npython cifar_main.py --model_type=sact --model=5 --tau=0.01 --checkpoint_dir='models/cifar10_resnet_5_sact_1e-2' --mode=eval --eval_dir='/tmp' --evaluate_once\n```\n\nThis model is expected to achieve an accuracy of 91.82%, with the output looking like so:\n\n```\neval/Accuracy[0.9182]\neval/Mean Loss[0.59591407]\nTotal Flops/mean[82393168]\nTotal Flops/std[7588926]\n...\n```\n\n## ImageNet\n\nFollow the [instructions](https://github.com/tensorflow/models/tree/master/research/inception#getting-started) to prepare the ImageNet dataset in TF-Slim format.\nThe default directory for the dataset is `~/tensorflow/imagenet`.\nYou can change it with the `--dataset_dir` flag.\n\nWe initialized all ACT/SACT models with a [pretrained ResNet-101 model](https://s3.us-east-2.amazonaws.com/sact-models/imagenet_101.tar.gz) (159MB file).\n\nDownload [pretrained ResNet-101 SACT model](https://s3.us-east-2.amazonaws.com/sact-models/imagenet_101_sact_5e-3.tar.gz), trained with tau=0.005 (160 MB file):\n``` bash\nmkdir -p models \u0026\u0026 curl https://s3.us-east-2.amazonaws.com/sact-models/imagenet_101_sact_5e-3.tar.gz | tar xv -C models\n```\n\nEvaluate the pretrained model\n``` bash\npython imagenet_eval.py --model_type=sact --model=101 --tau=0.005 --checkpoint_dir=models/imagenet_101_sact_5e-3 --eval_dir=/tmp --evaluate_once\n```\n\nExpected output:\n```\neval/Accuracy[0.75609803]\neval/Recall@5[0.9274632117722329]\nTotal Flops/mean[1.1100941e+10]\nTotal Flops/std[4.5691142e+08]\n...\n```\n\nNote that evaluation on the full validation dataset will take some time using only CPU.\nAdd the arguments `--num_examples=10 --batch_size=10` for a quicker test.\n\nDraw some images from ImageNet validation set and the corresponding ponder cost maps:\n\n``` bash\npython imagenet_export.py --model_type=sact --model=101 --tau=0.005 --checkpoint_dir=models/imagenet_101_sact_5e-3 --export_path=/tmp/maps.h5 --batch_size=1 --num_examples=200\n\nmkdir /tmp/maps\npython draw_ponder_maps.py --input_file=/tmp/maps.h5 --output_dir=/tmp/maps\n```\n\nExample visualizations. See Figure 9 of the paper for more\n\nImage                      | Ponder cost map\n:-------------------------:|:----------------------------:\n![](pics/20.92_93_im.jpg)  | ![](pics/20.92_93_ponder.png)\n![](pics/22.28_95_im.jpg)  | ![](pics/22.28_95_ponder.png)\n![](pics/26.75_36_im.jpg)  | ![](pics/26.75_36_ponder.png)\n\nApply the pretrained model to your own jpeg images.\nFor best results, first resize them to somewhere between 320x240 and 640x480.\n\n``` bash\npython2 imagenet_ponder_map.py --model=101 --checkpoint_dir=models/imagenet_101_sact_5e-3 --images_pattern=pics/gasworks.jpg --output_dir output/\n```\n\nImage                 | Ponder cost map                | Colorbar\n:--------------------:|:------------------------------:|---------\n![](pics/gasworks.jpg)| ![](pics/gasworks_ponder.jpg)  | ![](pics/gasworks_colorbar.jpg)\n![](pics/cat.jpg)     | ![](pics/cat_ponder.jpg)       | ![](pics/cat_colorbar.jpg)\n\nNote that an ImageNet-pretrained model tends to ignore people - there is no \"person\" class in ImageNet!\n\n## Disclaimer\n\nThis is not an official Google product.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmfigurnov%2Fsact","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmfigurnov%2Fsact","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmfigurnov%2Fsact/lists"}