{"id":13454863,"url":"https://github.com/akosiorek/attend_infer_repeat","last_synced_at":"2025-05-05T10:31:33.814Z","repository":{"id":77090981,"uuid":"98886404","full_name":"akosiorek/attend_infer_repeat","owner":"akosiorek","description":"A Tensorfflow implementation of Attend, Infer, Repeat","archived":false,"fork":false,"pushed_at":"2018-12-09T15:08:08.000Z","size":5164,"stargazers_count":81,"open_issues_count":2,"forks_count":20,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-08T21:33:12.127Z","etag":null,"topics":["attend-infer-repeat","attention","attention-mechanism","computer-graphics","computer-vision","generative-model","neural-networks","rnn","tensorflow","vae"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/akosiorek.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-07-31T12:27:17.000Z","updated_at":"2025-01-21T04:06:45.000Z","dependencies_parsed_at":"2023-02-24T17:46:21.268Z","dependency_job_id":null,"html_url":"https://github.com/akosiorek/attend_infer_repeat","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akosiorek%2Fattend_infer_repeat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akosiorek%2Fattend_infer_repeat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akosiorek%2Fattend_infer_repeat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akosiorek%2Fattend_infer_repeat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/akosiorek","download_url":"https://codeload.github.com/akosiorek/attend_infer_repeat/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252480491,"owners_count":21754782,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["attend-infer-repeat","attention","attention-mechanism","computer-graphics","computer-vision","generative-model","neural-networks","rnn","tensorflow","vae"],"created_at":"2024-07-31T08:00:58.724Z","updated_at":"2025-05-05T10:31:33.331Z","avatar_url":"https://github.com/akosiorek.png","language":"Python","funding_links":[],"categories":["Models/Projects"],"sub_categories":[],"readme":"# Attend, Infer, Repeat: Fast Scene Understanding with Generative Models\n\nThis is an **unofficial** Tensorflow implementation of Attend, Infear, Repeat (AIR), as presented in the following paper:\n[S. M. Ali Eslami](http://arkitus.com/) et. al., [Attend, Infer, Repeat: Fast Scene Understanding with Generative Models](https://papers.nips.cc/paper/6230-attend-infer-repeat-fast-scene-understanding-with-generative-models).\n\n* **Author (of the implementation)**: Adam Kosiorek, Oxford Robotics Institue, University of Oxford\n* **Email**: adamk(at)robots.ox.ac.uk\n* **Webpage**: http://akosiorek.github.io/\n\nI describe the implementation and the issues I run into while working on it in [this blog post](http://akosiorek.github.io/ml/2017/09/03/implementing-air.html).\n\n\n## Installation\nInstall [Tensorflow v1.1.0rc1](https://www.tensorflow.org/versions/r1.1/install/), [Sonnet v1.1](https://github.com/deepmind/sonnet/tree/3fd7d9d342d9683df83a44ebd048ef0d5668266b) and the following dependencies\n (using `pip install -r requirements.txt` (preferred) or `pip install [package]`):\n* matplotlib==1.5.3\n* numpy==1.12.1\n* attrdict==2.0.0\n* scipy==0.18.1\n\n## Sample Results\n\nAIR learns to reconstruct objects by painting them one by one in a blank canvas. The below figure comes from a model trained for 175k iterations; the maximum number of steps is set to 3, but there are never more than 2 objects.\nThe first row shows the input images, rows 2-4 are reconstructions at steps 1, 2 and 3 (with marked location of the attention glimpse in red, if it exists). Rows 4-7 are the reconstructed image crops, and above each crop is the\nprobability of executing 1, 2 or 3 steps. If the reconstructed crop is black and there is \"0 with ...\" written above it, it means that this step was not used.\n\n![AIR results](https://raw.githubusercontent.com/akosiorek/attend_infer_repeat/master/results/progress_fig_175000.jpg \"AIR results\")\n\n## Data  \nRun `./scripts/create_dataset.sh`\nThe script creates train and validation datasets of multi-digit MNIST.\n\n## Training\nRun `./scripts/train_multi_mnist.sh`\nThe training script will run for 300k iteratios and will save model checkpoints and training progress figures every 10k iterations in `results/multi_mnist`. Tensorflow summaries are also stored in the same folder and Tensorboard can be used for monitoring.\n\nThe model seems to be very sensitive to initialisation. It might be necessary to run training multiple times before achieving count step accuracy close to the one reported in the paper.\n\n## Experimentation\nThe jupyter notebook available at `attend_infer_repeat/experiment.ipynb` can be used for experimentation.\n\n## Citation\n\nIf you find this repo useful in your research, please consider citing the original paper:\n\n    @incollection{Eslami2016,\n        title = {Attend, Infer, Repeat: Fast Scene Understanding with Generative Models},\n        author = {Eslami, S. M. Ali and Heess, Nicolas and Weber, Theophane and Tassa, Yuval and Szepesvari, David and kavukcuoglu, koray and Hinton, Geoffrey E},\n        booktitle = {Advances in Neural Information Processing Systems 29},\n        editor = {D. D. Lee and M. Sugiyama and U. V. Luxburg and I. Guyon and R. Garnett},\n        pages = {3225--3233},\n        year = {2016},\n        publisher = {Curran Associates, Inc.},\n        url = {http://papers.nips.cc/paper/6230-attend-infer-repeat-fast-scene-understanding-with-generative-models.pdf}\n    }\n\n\n## License\n\nThis program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.\n\nThis program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License along with this program. If not, see  \u003chttp://www.gnu.org/licenses/\u003e.\n\n\n## Release Notes\n**Version 1.0**\n* Original **unofficial** implementation; contains the multi-digit MNIST experiment.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakosiorek%2Fattend_infer_repeat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fakosiorek%2Fattend_infer_repeat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakosiorek%2Fattend_infer_repeat/lists"}