{"id":20424001,"url":"https://github.com/emalagoli92/pondernet-tensorflow","last_synced_at":"2026-04-14T23:34:14.827Z","repository":{"id":61853044,"uuid":"423999807","full_name":"EMalagoli92/PonderNet-TensorFlow","owner":"EMalagoli92","description":"TensorFlow 2.X reimplementation of PonderNet: Learning to Ponder, Andrea Banino, Jan Balaguer, Charles Blundell.","archived":false,"fork":false,"pushed_at":"2023-01-13T18:20:19.000Z","size":295,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-05T04:42:37.448Z","etag":null,"topics":["adaptive-computation-time","artficial-intelligence","deep-learning","pytorch","recurrent-neural-network","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EMalagoli92.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-11-02T21:09:39.000Z","updated_at":"2025-01-12T18:47:45.000Z","dependencies_parsed_at":"2023-02-09T16:45:58.853Z","dependency_job_id":null,"html_url":"https://github.com/EMalagoli92/PonderNet-TensorFlow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/EMalagoli92/PonderNet-TensorFlow","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EMalagoli92%2FPonderNet-TensorFlow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EMalagoli92%2FPonderNet-TensorFlow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EMalagoli92%2FPonderNet-TensorFlow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EMalagoli92%2FPonderNet-TensorFlow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EMalagoli92","download_url":"https://codeload.github.com/EMalagoli92/PonderNet-TensorFlow/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EMalagoli92%2FPonderNet-TensorFlow/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31819906,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-14T18:05:02.291Z","status":"ssl_error","status_checked_at":"2026-04-14T18:05:01.765Z","response_time":153,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adaptive-computation-time","artficial-intelligence","deep-learning","pytorch","recurrent-neural-network","tensorflow"],"created_at":"2024-11-15T07:08:25.240Z","updated_at":"2026-04-14T23:34:14.797Z","avatar_url":"https://github.com/EMalagoli92.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n  \u003ca href=\"https://www.tensorflow.org\"\u003e![TensorFLow](https://img.shields.io/badge/TensorFlow-2.X-orange?style=for-the-badge) \n  \u003ca href=\"https://github.com/EMalagoli92/PonderNet-TensorFlow/blob/main/LICENSE\"\u003e![License](https://img.shields.io/github/license/EMalagoli92/PonderNet-TensorFlow?style=for-the-badge) \n  \u003ca href=\"https://www.python.org\"\u003e![Python](https://img.shields.io/badge/python-%3E%3D%203.9-blue?style=for-the-badge)\u003c/a\u003e  \n  \n\u003c/div\u003e\n\n# PonderNet - TensorFlow\n\nTensorFlow 2.X reimplementation of [PonderNet: Learning to Ponder](https://arxiv.org/abs/2107.05407), Andrea Banino, Jan Balaguer, Charles Blundell.\n\n## Table of contents\n- [Abstract](#abstract)\n- [Experiment on Parity Task](#paritytask)    \n- [Installation](#installation)\n- [Usage](#usage)    \n- [Acknowledgement](#acknowledgement)    \n- [Citations](#citations)\n- [License](#license)    \n    \n\u003cdiv id=\"abstract\"/\u003e\n\n## Abstract\nIn standard neural networks the amount of computation used grows with the size of the inputs, but not with the complexity of the problem being learnt. To overcome this limitation we introduce PonderNet, a new algorithm that learns to adapt the amount of computation based on the complexity of the problem at hand. PonderNet learns end-to-end the number of computational steps to achieve an effective compromise between training prediction accuracy, computational cost and generalization. On a complex synthetic problem, PonderNet dramatically improves performance over previous adaptive computation methods and additionally succeeds at extrapolation tests where traditional neural networks fail. Also, our method matched the current state of the art results on a real world question and answering dataset, but using less compute. Finally, PonderNet reached state of the art results on a complex task designed to test the reasoning capabilities of neural networks.\n\n\u003cdiv id=\"paritytask\"/\u003e\n    \n## Experiment  on Parity Task\nThe input of the parity task is a vector with 0's 1's and −1's. The output is the parity of 1's - one if there is an odd number of 1's and zero otherwise. The input is generated by making a random number of elements in the vector either 1 or −1's.\n\n![Alt text](https://raw.githubusercontent.com/EMalagoli92/PonderNet-TensorFlow/main/assets/images/parity_task.png)    \n\u003cp align = \"center\"\u003e\u003csub\u003ePerformance on the parity task. a) Interpolation. Top: accuracy for both PonderNet (blue) and ACT (orange). Bottom: number of ponder steps at evaluation time. Error bars are calculated over 10 random seeds. b) Extrapolation. Top: accuracy for both PonderNet (blue) and ACT (orange). Bottom: number of ponder steps at evaluation time. Error bars are calculated over 10 random seeds. c) Total number of compute steps calculated as the number of actual forward passes performed by each network. Blue is PonderNet, Green is ACT and Orange is an RNN without adaptive compute.\u003csub\u003e\u003c/sub\u003e\u003c/p\u003e    \n    \n\u003cdiv id=\"installation\"/\u003e\n    \n## Installation\nClone the repo and install necessary packages \n```\ngit clone https://github.com/EMalagoli92/PonderNet-TensorFlow.git\npip install -r requirements.txt\n```\nTested on *Ubuntu 20.04.4 LTS x86_64*, *python 3.9.7*.    \n    \n\u003cdiv id=\"usage\"/\u003e\n    \n## Usage\nTrain a PonderNet on Parity Task\n```\npython __main__.py    \n```\n  \n\n\n\n\u003cdiv id=\"acknowledgement\"/\u003e\n\n## Acknowledgement\n[PonderNet](https://nn.labml.ai/adaptive_computation/ponder_net/index.html) (Official PyTorch Implementation)\n    \n\n\u003cdiv id=\"citations\"/\u003e\n    \n## Citations\n```bibtex\n@misc{banino2021pondernet,\n      title={PonderNet: Learning to Ponder}, \n      author={Andrea Banino and Jan Balaguer and Charles Blundell},\n      year={2021},\n      eprint={2107.05407},\n      archivePrefix={arXiv},\n      primaryClass={cs.LG}\n}\n```\n\n\n\u003cdiv id=\"license\"/\u003e\n\n## License\nThis work is made available under the [MIT License](https://github.com/EMalagoli92/PonderNet-TensorFlow/blob/main/LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femalagoli92%2Fpondernet-tensorflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Femalagoli92%2Fpondernet-tensorflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femalagoli92%2Fpondernet-tensorflow/lists"}