{"id":22209499,"url":"https://github.com/wilmeragsgh/adabnn","last_synced_at":"2025-06-11T05:38:19.025Z","repository":{"id":171536832,"uuid":"163737571","full_name":"wilmeragsgh/adabnn","owner":"wilmeragsgh","description":"Code related to thesis work: \"AdaBnn: Binarized Neural Networks trained with adaptive structural learning\"","archived":false,"fork":false,"pushed_at":"2020-11-10T14:42:01.000Z","size":3393,"stargazers_count":9,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-06-01T04:01:24.745Z","etag":null,"topics":["deep-learning","keras","neural-networks","scikit-learn","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wilmeragsgh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2019-01-01T13:11:05.000Z","updated_at":"2023-12-14T14:52:45.000Z","dependencies_parsed_at":null,"dependency_job_id":"136ec17d-34cf-4bab-b150-4a4c4ae8d1b4","html_url":"https://github.com/wilmeragsgh/adabnn","commit_stats":null,"previous_names":["wilmeragsgh/adabnn"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wilmeragsgh%2Fadabnn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wilmeragsgh%2Fadabnn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wilmeragsgh%2Fadabnn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wilmeragsgh%2Fadabnn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wilmeragsgh","download_url":"https://codeload.github.com/wilmeragsgh/adabnn/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wilmeragsgh%2Fadabnn/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":258596353,"owners_count":22726378,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","keras","neural-networks","scikit-learn","tensorflow"],"created_at":"2024-12-02T19:31:04.345Z","updated_at":"2025-06-11T05:38:19.002Z","avatar_url":"https://github.com/wilmeragsgh.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AdaBnn\n\n[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/wilmeragsgh/adabnn/blob/master/LICENSE)\n\nCode related to thesis work: **\"AdaBnn: Binarized Neural Networks trained with adaptive structural learning\"** [[pdf]](https://mega.nz/#!SkFhxCSI!YQO-ZYQl5tFlEGpkl2nR13zzFAzJeT5iCgZt8AzvIsQ)\n\nThis repository contains currently two colaboratory notebooks:\n* [English](https://colab.research.google.com/github/wilmeragsgh/adabnn/blob/master/experiments_en.ipynb)\n* [Spanish](https://colab.research.google.com/github/wilmeragsgh/adabnn/blob/master/experiments_es.ipynb)\n\nThat document experiments made with an experimental **Keras** based implementation of the AdaNet algorithm presented in “[AdaNet: Adaptive Structural Learning of Artificial Neural Networks](http://proceedings.mlr.press/v70/cortes17a.html)” at [ICML 2017](https://icml.cc/Conferences/2017), for learning the structure of a neural network as an ensemble of subnetworks. Also, AdaBnn is presented as a modification of AdaNet that imposed binary constraints on running time to try to increase performance in terms of time and as a way of regularization based on \"[Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1](https://arxiv.org/abs/1602.02830)\".\n\nAlso, separated code is included containing Adanet and AdaBnn implementations with its documentation.\n\n## Some findings\n\nAccording to the experiments provided in the notebooks:\n\n* Binarizing the weights of the network, in the case of adaptive structural learning, have similar effects of having a high mutation rate in genetic algorithms, the pattern of learning is much more hard to follow between iterations, doesn't hold incremental performance in T iterations.\n* Adam optimization it's more appropiate for such AdaBnn structure in most cases as well as fewer iterations (T parameter on the paper).\n* Currently Binarizing AdaNet doesn't hold that much of improvement but it could be a start point for adding constraints for weights/activations as regularization method for adaptive structural learning.\n\n## Futher work\n\nFurther work may include the binarization process as part of the convolution sub network, which was the initial proposal of (M Courbariaux ,2016).\n\n## Example\n\nAfter importing dependencies and declaring each model (proposed implementation of AdaNet and AdaBnn) they can be trained and used with:\n\n```python\nepochs = 25 \nB = 150 # neurons on each layer of the sub network  \nT = 2 # number of iterations of the adanet algorithm (maximum complexity of the model)\nconf = dict({\n    'network': {\n        'activation': BinaryRelu, \n        'output_activation': 'sigmoid',# softmax for multiple clases\n        'optimizer': keras.optimizers.Adam(lr=0.0001),#Adam\n        'loss': 'binary_crossentropy' # F for adanet traditional\n        },\n    'training':{\n        'batch_size': 100,\n        'epochs': epochs\n        },\n    'adabnn':{\n        'B': B,\n        'T': T,\n        'delta': 1.01,\n        'seed': 42\n        }\n    })\nmodel, rend = AdaBnn(x_train,y_train,conf,verbose=0) # rend will have accuracy/loss at each iteration T\ntraining_results = model.evaluate(x_test,y_test,verbose=0)\n```\n\n## Citing this Work\n\nIf you use these implementations for academic research, please cite the following:\n\n    @misc{gonzalez2018adabnn,\n      author    = {Wilmer Gonzalez},\n      title     = {AdaBnn: Binarized Neural Networks trained with adaptive structural learning},\n      year      = {2018},\n      publisher = {GitHub},\n      journal = {GitHub repository},\n      howpublished = {\\url{https://github.com/wilmeragsgh/adabnn}},\n    }\n\n## License\n\nIncluded functions and notebooks are released under the [Apache License 2.0](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwilmeragsgh%2Fadabnn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwilmeragsgh%2Fadabnn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwilmeragsgh%2Fadabnn/lists"}