{"id":16858780,"url":"https://github.com/titu1994/keras-adabound","last_synced_at":"2025-03-17T05:32:24.560Z","repository":{"id":150946812,"uuid":"172848127","full_name":"titu1994/keras-adabound","owner":"titu1994","description":"Keras implementation of AdaBound","archived":false,"fork":false,"pushed_at":"2019-11-04T01:02:07.000Z","size":3672,"stargazers_count":130,"open_issues_count":5,"forks_count":35,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-02-27T18:27:00.109Z","etag":null,"topics":["deep-learning","keras","optimizer"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/titu1994.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-27T05:18:04.000Z","updated_at":"2024-06-19T02:13:40.000Z","dependencies_parsed_at":null,"dependency_job_id":"62bfd579-2780-4992-b502-68f58f2a1366","html_url":"https://github.com/titu1994/keras-adabound","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/titu1994%2Fkeras-adabound","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/titu1994%2Fkeras-adabound/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/titu1994%2Fkeras-adabound/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/titu1994%2Fkeras-adabound/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/titu1994","download_url":"https://codeload.github.com/titu1994/keras-adabound/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243846989,"owners_count":20357296,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","keras","optimizer"],"created_at":"2024-10-13T14:15:10.285Z","updated_at":"2025-03-17T05:32:23.861Z","avatar_url":"https://github.com/titu1994.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AdaBound for Keras\n\nKeras port of [AdaBound Optimizer for PyTorch](https://github.com/Luolc/AdaBound), from the paper [Adaptive Gradient Methods with Dynamic Bound of Learning Rate.](https://openreview.net/forum?id=Bkg3g2R9FX)\n\n## Usage\n\nAdd the `adabound.py` script to your project, and import it. Can be a dropin replacement for `Adam` Optimizer. \n\nAlso supports `AMSBound` variant of the above, equivalent to `AMSGrad` from Adam.\n\n```python\nfrom adabound import AdaBound\n\noptm = AdaBound(lr=1e-03,\n                final_lr=0.1,\n                gamma=1e-03,\n                weight_decay=0.,\n                amsbound=False)\n```\n\n## Results\n\nWith a wide ResNet 34 and horizontal flips data augmentation, and 100 epochs of training with batchsize 128, it hits 92.16% (called v1).\n\nWeights are available inside the [Releases tab](https://github.com/titu1994/keras-adabound/releases/tag/0.1)\n\n#### NOTE\n - The smaller ResNet 20 models have been removed as they did not perform as expected and were depending on a flaw during the initial implementation. The ResNet 32 shows the actual performance of this optimizer.\n\n\u003e With a small ResNet 20 and width + height data + horizontal flips data augmentation, and 100 epochs of training with batchsize 1024, it hits 89.5% (called v1).\n\n\u003e On a small ResNet 20 with only width and height data augmentations, with batchsize 1024 trained for 100 epochs, the model gets close to 86% on the test set (called v3 below).\n\n\n### Train Set Accuracy\n\n\u003cimg src=\"https://github.com/titu1994/keras-adabound/blob/master/images/train_acc.PNG?raw=true\" height=50% width=100%\u003e\n\n### Train Set Loss\n\n\u003cimg src=\"https://github.com/titu1994/keras-adabound/blob/master/images/train_loss.PNG?raw=true\" height=50% width=100%\u003e\n\n### Test Set Accuracy\n\n\u003cimg src=\"https://github.com/titu1994/keras-adabound/blob/master/images/val_acc.PNG?raw=true\" height=50% width=100%\u003e\n\n### Test Set Loss\n\n\u003cimg src=\"https://github.com/titu1994/keras-adabound/blob/master/images/val_loss.PNG?raw=true\" height=50% width=100%\u003e\n\n# Requirements\n- Keras 2.2.4+ \u0026 Tensorflow 1.12+ (Only supports TF backend for now).\n- Numpy\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftitu1994%2Fkeras-adabound","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftitu1994%2Fkeras-adabound","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftitu1994%2Fkeras-adabound/lists"}