{"id":18044869,"url":"https://github.com/zsxkib/nat-cw2","last_synced_at":"2025-04-05T03:27:14.724Z","repository":{"id":68481169,"uuid":"311351087","full_name":"zsxkib/NAT-CW2","owner":"zsxkib","description":"Particle Swarm Optimisation, Genetic Algorithm/Programming for (Gradient-Free) Neural Network Optimisation","archived":false,"fork":false,"pushed_at":"2020-12-03T14:05:27.000Z","size":5298,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-01T13:08:36.903Z","etag":null,"topics":["ga","genetic-algorithm","genetic-programming","gp","gradient-free-optimization","natural-computing","particle-swarm-optimisation","pso"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zsxkib.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-11-09T13:42:05.000Z","updated_at":"2020-12-03T14:05:29.000Z","dependencies_parsed_at":null,"dependency_job_id":"2c4b5168-843b-49c2-8330-3a92e81b3dfe","html_url":"https://github.com/zsxkib/NAT-CW2","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zsxkib%2FNAT-CW2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zsxkib%2FNAT-CW2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zsxkib%2FNAT-CW2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zsxkib%2FNAT-CW2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zsxkib","download_url":"https://codeload.github.com/zsxkib/NAT-CW2/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247284927,"owners_count":20913689,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ga","genetic-algorithm","genetic-programming","gp","gradient-free-optimization","natural-computing","particle-swarm-optimisation","pso"],"created_at":"2024-10-30T18:11:04.617Z","updated_at":"2025-04-05T03:27:14.703Z","avatar_url":"https://github.com/zsxkib.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PSO, GA and GP for Neural Network Optimisation (Gradient-Free Methods)\nNatural Computing 2020/2021: Assignment 2\n\n## Abstract\nWe explore optimisation of Neural Networks with differing amounts of hidden neurons and layers using gradient-free algorithms. We see that in general, classical optimisation methods reminiscent of Stochastic Gradient Descent tend to perform better than gradient-free methods such as Particle Swarm Optimisation, Genetic Algorithms, and Genetic Programming. We see that although optima reached by gradient-free methods are not as 'good' as classical gradient-based methods, they are nevertheless good enough. That is, we reach very small errors on test sets generalising to unseen data. The main difference being time to converge.\n\n## Overview\nWe will explore gradient-free optimisation methods for optimising Neural Networks (NN). Including Particle Swarm Optimisation (PSO), Genetic Algorithms (GA), and Genetic Programming (GP). \n\nGradient-free algorithms will then be compared to baseline NNs optimised using standard gradient-based methods (view Appendix 1 for a description of baseline). Including the Stochastic gradient descent (SGD) classical way to optimise NNs, Adam an optimiser similar to SGD, and Limited memory Broyden Fletcher Goldfarb Shanno (LBFGS) quasi-Newton method for optimisation.\n\n## Install dependencies\n\n```\npip install -r requirements.txt\n```\n\n## Task 1: Particle Swarm Optimisation\n\n****\n\nRun the notebook \"Task1.ipynb\" under the directory \"Task 1 Particle swarm optimisation\" and follow the instructions\n\n## Task 2: Genetic Algorithm\n\n****\n\nRun the notebook \"GeneticAlgorithm.ipynb\" under the directory \"Task 2 Genetic Algorithms\" and follow the instructions\n\n## Task 3: Genetic Programming\n\n****\n\nRun the notebook \"GeneticProgramming.ipynb\" under the directory \"Task 3 Genetic Programming\" and follow the instructions\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzsxkib%2Fnat-cw2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzsxkib%2Fnat-cw2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzsxkib%2Fnat-cw2/lists"}