{"id":15706467,"url":"https://github.com/jrieke/evolution-learning","last_synced_at":"2025-05-12T18:55:22.932Z","repository":{"id":40970337,"uuid":"235677675","full_name":"jrieke/evolution-learning","owner":"jrieke","description":"🐣  Code for my master thesis \"Biologically Plausible Deep Learning through Neuroevolution\"","archived":false,"fork":false,"pushed_at":"2022-09-30T19:57:14.000Z","size":8452,"stargazers_count":6,"open_issues_count":3,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-05-06T09:23:22.704Z","etag":null,"topics":["backpropagation","biologically-plausible-learning","evolutionary-algorithms","feedback-alignment","hebbian-learning","machine-learning","master-thesis","neat","neural-networks","research"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jrieke.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-01-22T22:11:09.000Z","updated_at":"2023-10-24T21:29:25.000Z","dependencies_parsed_at":"2023-01-18T10:16:15.116Z","dependency_job_id":null,"html_url":"https://github.com/jrieke/evolution-learning","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jrieke%2Fevolution-learning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jrieke%2Fevolution-learning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jrieke%2Fevolution-learning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jrieke%2Fevolution-learning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jrieke","download_url":"https://codeload.github.com/jrieke/evolution-learning/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253804872,"owners_count":21967050,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backpropagation","biologically-plausible-learning","evolutionary-algorithms","feedback-alignment","hebbian-learning","machine-learning","master-thesis","neat","neural-networks","research"],"created_at":"2024-10-03T20:23:09.607Z","updated_at":"2025-05-12T18:55:22.899Z","avatar_url":"https://github.com/jrieke.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Biologically Plausible Deep Learning through Neuroevolution\n\nThis is the code for my master thesis at TU Berlin and BCCN Berlin. \n\nIt combines an evolutionary approach with different learning algorithms \n(backpropagation, feedback alignment, Hebbian learning, last-layer learning) \non MNIST. You can read the complete thesis \n[here](master-thesis-johannes-rieke-final.pdf). \n\n![](figures/circles.png)\n\n\n## Abstract\n\nRecently, deep learning has been increasingly used as a framework to understand \ninformation processing in the brain. Most existing research in this area has \nfocused on finding biologically plausible learning algorithms. In this thesis, \nwe investigate how the architecture of a neural network interacts with such \nlearning algorithms. Inspired by the intricate wiring of biological brains, we \nuse an evolutionary algorithm to develop network architectures by mutating \nneurons and connections. Within the evolution loop, the synaptic weights of the \nnetworks are trained with one of four learning algorithms, which vary in \ncomplexity and biological plausibility (backpropagation, feedback alignment, \nHebbian learning, last-layer learning). We investigate how the evolved network \narchitectures differ between learning algorithms, both in terms of their \nperformance on image classification with MNIST as well as their topologies. We \nshow that for all learning algorithms, the evolved architectures learn much \nbetter than random networks – i.e. evolution and learning interact. Also, we \nfind that more complex learning algorithms (e.g. backpropagation) can make use \nof larger, deeper, and more densely connected networks, which achieve higher \nperformances. For simpler and biologically more plausible learning algorithms \n(e.g. Hebbian learning), however, the evolution process prefers smaller \nnetworks. Our results demonstrate that the architecture of a neural network \nplays an important role in its ability to learn. Using a range of learning \nalgorithms, we show that evolutionary processes can find architectures of \nartificial neural networks that are particularly suited for learning – similar \nto how biological evolution potentially shaped the complex wiring of neurons \nin the brain.\n\n\n## Usage\n\nExperiments are run like this:\n\n    python run-wann-gym.py params/wann-gym/CartPoleSwingUp.yaml --num_workers 4\n\nLet's analyze each part individually: \n\n`run-wann-gym.py` is the experiment script. This specific script evolves a \nweight agnostic neural network on a gym environment. The repo contains 4 \ndifferent experiment scripts:\n\n- **run-wann-gym.py**: Evolve weight agnostic neural networks on gym \nenvironment. Works with any OpenAI gym; parameter files for CartPole-v0, \nCartPoleSwingUp and BipedalWalker-v2 are in the params folder (with the same \nparameter values as in Gaier \u0026 Ha 2019). \n- **run-wann-classification.py**: Evolve weight agnostic neural networks on an \nimage classification dataset. Param files for sklearn digits and MNIST (same \nvalues as Gaier \u0026 Ha 2019) are given. \n- **run-wlnn-mnist.py**: Simultaneously evolve architecture of a network and \ntrain its weights on an image classification dataset, using one of several \nlearning algorithms (evolution is outer loop and develops the architecture, \ntraining is inner loop and changes the weights). \n- **run-wlnn-mnist-no-evolution.py**: Train the weights of a neural network on \nan image classification dataset, using one of several learning algorithms \n(backprop, feedback alignment, Hebbian; no evolution!). \n\n\n`params/wann-gym/CartPoleSwingUp.yaml` is the parameter file in yaml format. \nThis specific file defines parameters for the CartPoleSwingUp task as in Gaier \n\u0026 Ha (2019). The folder params contains many more parameter files.\n\n`--num_workers 4` defines a parameter from the terminal (specifically, it tells \nthe script to use 4 cores in parallel). You can define all parameters on the \nterminal as well as in parameter files. Terminal parameters overwrite the values \nfrom parameter files. See below for a complete list of parameters. \n\n\n## Log files\n\nLog files for the experiments in the thesis are available in \n[this Google Drive folder](https://drive.google.com/drive/folders/1uoGx9glqYXKxW0OqW37F1IHJa2OxPHmx?usp=sharing). \nEach log is one folder, which contains a log file of the complete std output \nof the experiment, the experiment parameters, tensorboard files with lots of \nmetrics, and (for some logs) all networks stored at regular intervals as json \nfiles.\n\nFor chapter 4.1 in the thesis (replicating weight agnostic neural \nnetworks):\n\n- CartPoleSwingUp: [2019-11-30_02-34-46](https://drive.google.com/open?id=1r-Saxv-o6fl2OaowxPDOl3DOO0B-PNRE)\n- BipedalWalker: [2019-12-01_22-18-42](https://drive.google.com/open?id=1FN4Ml7MjRyjD50wunRpvQsH1uumpiyVr)\n- MNIST: [2019-11-28_23-42-24](https://drive.google.com/open?id=1CYBhBOOrQ2xUtFfXmm3lkHIHx66WL-x2)\n\n\nFor chapter 4.2 in the thesis (combining evolution and learning):\n\n- Backpropagation on MNIST: [2020-01-17_16-08-55](https://drive.google.com/open?id=1Q1eaHus7JsFOYTp6q6P8q7KZQyrvDZOi)\n- Feedback alignment on MNIST: [2020-01-17_03-02-27](https://drive.google.com/open?id=1rQOqCThtghdbRgDY9r3Vnl2p-CIbIqlK)\n- Hebbian learning on MNIST: [2020-01-17_03-05-57](https://drive.google.com/open?id=1h5rJp4-08Wi1bzCbD7yLZYbhSJ7L4hIl)\n- Last-layer learning on MNIST: [2020-01-17_03-08-48](https://drive.google.com/open?id=1CN3PMcKr9WleQU6THCOEH1RVNWHK-LGn)\n\n\n## Parameters\n\nNote that some of thes parameters only have an effect on specific scripts. \n\n    evolution options:\n      --cull_ratio CULL_RATIO\n                            Fraction of worst networks to leave out of the\n                            breeding pool\n      --elite_ratio ELITE_RATIO\n                            Fraction of best networks to pass on to the new\n                            population directly\n      --num_generations NUM_GENERATIONS\n                            Number of generations for evolution\n      --population_size POPULATION_SIZE\n                            Number of networks in the population\n      --p_add_connection P_ADD_CONNECTION\n                            Probability of adding a connection during mutation\n      --p_add_node P_ADD_NODE\n                            Probability of adding a node during mutation\n      --p_change_activation P_CHANGE_ACTIVATION\n                            Probability of changing the activation of a node\n                            during mutation\n      --tournament_size TOURNAMENT_SIZE\n                            Number of networks that compete during tournament\n                            selection\n      --inherit_weights INHERIT_WEIGHTS\n                            Keep weights after mutating a network\n      --num_mutations_per_generation NUM_MUTATIONS_PER_GENERATION\n                            The number of mutations to carry out at each\n                            generation\n      --p_complexity_objective P_COMPLEXITY_OBJECTIVE\n                            The fraction of generations to rank according to\n                            complexity and reward (otherwise only reward)\n    \n    evaluation options:\n      --batch_size BATCH_SIZE\n                            Batch size for evaluation (in run-wann-\n                            classification.py) or learning (in run-wlnn-mnist-no-\n                            evolution.py and run-wlnn-mnist.py)\n      --batch_size_eval BATCH_SIZE_EVAL\n                            Batch size for test set evaluation of learned networks\n      --evaluation_period {integral,last_batch,last_ten_batches,first_ten_batches,last_epoch}\n                            Which training period should be used to evaluate\n                            networks. The options are: \"integral\" the mean across\n                            the whole training duration, \"last_batch\" the values\n                            in the final batch, \"last_ten_batches\" the values in\n                            the last ten batches, \"first_ten_batches\" the values\n                            in the first ten batches and \"last_epoch\" the values\n                            averaged in the last epoch only.\n      --num_trials NUM_TRIALS\n                            How often to run the gym environment during evaluation\n    \n    training options:\n      --learning_rule LEARNING_RULE\n                            Learning rule to train network\n      --lr LR               Learning rate\n      --num_epochs NUM_EPOCHS\n                            Number of epochs for learning\n      --optimizer OPTIMIZER\n                            Optimizer to train network (sgd, adam or adadelta)\n      --train_only_outputs  If this option is selected, only the weights to the\n                            output units will be learned. Else, all weights will\n                            be learned.\n    \n    architecture options:\n      --num_inputs NUM_INPUTS\n                            Number of input neurons\n      --num_outputs NUM_OUTPUTS\n                            Number of output neurons\n      --p_initial_connection_enabled P_INITIAL_CONNECTION_ENABLED\n                            Probability of enabling a connection between input and\n                            output layer at the start of evolution\n    \n    task and dataset options:\n      params_file           A yaml file with parameters (see folder params for\n                            examples)\n      --dataset DATASET     Dataset for classification (digits or mnist)\n      --env_name ENV_NAME   Name of the gym environment\n    \n    computational options:\n      --num_workers NUM_WORKERS\n                            Number of workers to run on\n      --take_argmax_action TAKE_ARGMAX_ACTION\n                            Use argmax of the network output as the action in the\n                            environment\n      --use_cuda USE_CUDA   Use cuda devices if available\n      --use_torch USE_TORCH\n                            Use torch instead of numpy\n    \n    miscellaneous options:\n      --out_dir OUT_DIR     The path to the output directory\n      --overwrite_output    Overwrite data in the output folder if it already\n                            exists.\n\n\n## Requirements\n\n- Python 3\n- scientific computing libraries (numpy, ...) contained in Anaconda\n- torch: [installation instructions](https://pytorch.org/)\n- gym (needs v0.9.4 for CartPoleSwingUp): `pip install gym==0.9.4`\n- box2d-py (only for BipedalWalker-v2 task): `pip install box2d-py`\n- joblib: `conda install joblib` or `pip install joblib`\n- mnist: `pip install mnist`\n- cv2: `conda install opencv` or `pip install opencv-python`\n- GPUtil: `pip install GPUtil`\n- tensorboardX: `pip install tensorboardX`\n- pytest (for testing): `pip install pytest`\n\nNote that the requirements.txt file is only for circle-ci and should not be \nconsidered to work on any environment.\n\n\n## Citation\n\nIf you use this code, please cite my thesis:\n\n    @mastersthesis{Rieke2020,\n        author = {Johannes Rieke}, \n        title = {Biologically Plausible Deep Learning through Neuroevolution},\n        school = {Technical University of Berlin},\n        year = 2020,\n        url = {https://github.com/jrieke/evolution-learning}\n    }\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjrieke%2Fevolution-learning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjrieke%2Fevolution-learning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjrieke%2Fevolution-learning/lists"}