{"id":18305754,"url":"https://github.com/yaricom/goneat_ns","last_synced_at":"2025-04-05T16:32:46.626Z","repository":{"id":33863418,"uuid":"136015056","full_name":"yaricom/goNEAT_NS","owner":"yaricom","description":"This project provides GOLang implementation of Neuro-Evolution of Augmenting Topologies (NEAT) with Novelty Search optimization aimed to solve deceptive tasks with strong local optima","archived":false,"fork":false,"pushed_at":"2024-12-09T17:19:46.000Z","size":4761,"stargazers_count":40,"open_issues_count":0,"forks_count":7,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-21T06:51:21.417Z","etag":null,"topics":["artificial-neural-networks","augmenting-topologies","explainable-ai","explainable-artificial-intelligence","golang","modular-ai","neat","neuroevolution","novelty-search","reinforcement-learning-algorithms","unsupervised-learning","unsupervised-learning-algorithms","unsupervised-machine-learning"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yaricom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"patreon":"io42","custom":["https://www.buymeacoffee.com/io42"]}},"created_at":"2018-06-04T11:27:25.000Z","updated_at":"2025-03-13T18:51:49.000Z","dependencies_parsed_at":"2024-06-19T00:12:10.060Z","dependency_job_id":"ab456959-5862-43de-b0eb-74d67a822590","html_url":"https://github.com/yaricom/goNEAT_NS","commit_stats":null,"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yaricom%2FgoNEAT_NS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yaricom%2FgoNEAT_NS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yaricom%2FgoNEAT_NS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yaricom%2FgoNEAT_NS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yaricom","download_url":"https://codeload.github.com/yaricom/goNEAT_NS/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247366913,"owners_count":20927620,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-neural-networks","augmenting-topologies","explainable-ai","explainable-artificial-intelligence","golang","modular-ai","neat","neuroevolution","novelty-search","reinforcement-learning-algorithms","unsupervised-learning","unsupervised-learning-algorithms","unsupervised-machine-learning"],"created_at":"2024-11-05T15:35:35.437Z","updated_at":"2025-04-05T16:32:46.619Z","avatar_url":"https://github.com/yaricom.png","language":"Go","funding_links":["https://patreon.com/io42","https://www.buymeacoffee.com/io42","https://img.buymeacoffee.com/button-api/?text=Buy"],"categories":[],"sub_categories":[],"readme":"# goNEAT-NS 🇺🇦 [![Made in Ukraine](https://img.shields.io/badge/made_in-ukraine-ffd700.svg?labelColor=0057b7)](https://u24.gov.ua)\n\n[![version](https://img.shields.io/github/v/tag/yaricom/goNEAT_NS.svg?sort=semver)](https://github.com/yaricom/goNEAT_NS/releases/latest)\n[![GoDoc](https://godoc.org/github.com/yaricom/goNEAT_NS/neatns?status.svg)](https://godoc.org/github.com/yaricom/goNEAT_NS/neatns) [![Go version](https://img.shields.io/badge/go-1.21-blue.svg)](https://github.com/moovweb/gvm) [![license](https://img.shields.io/github/license/yaricom/goNEAT_NS.svg)](https://github.com/yaricom/goNEAT_NS/blob/master/LICENSE) [![yaricom/goNEAT](https://tokei.rs/b1/github/yaricom/goNEAT_NS?category=lines)](https://github.com/yaricom/goNEAT_NS) [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.10951521.svg)](https://doi.org/10.5281/zenodo.10951521)\n\n| Branch | Tests                                                                                                                                           | Coverage                                                                                                                                      | Linting                                                                                                                                               | Code Security                                                                                                                                                                 |\n|--------|-------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| master | [![CI](https://github.com/yaricom/goNEAT_NS/actions/workflows/ci.yml/badge.svg)](https://github.com/yaricom/goNEAT_NS/actions/workflows/ci.yml) | [![codecov](https://codecov.io/gh/yaricom/goNEAT_NS/branch/master/graph/badge.svg?token=as31613DnV)](https://codecov.io/gh/yaricom/goNEAT_NS) | [![Lint](https://github.com/yaricom/goNEAT_NS/actions/workflows/lint.yml/badge.svg)](https://github.com/yaricom/goNEAT_NS/actions/workflows/lint.yml) | [![CodeQL](https://github.com/yaricom/goNEAT_NS/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/yaricom/goNEAT_NS/actions/workflows/codeql-analysis.yml) |\n\n## Overview\n\nThis repository provides implementation of [Neuro-Evolution of Augmented Topologies (NEAT)][1] with Novelty Search\noptimization implemented in GoLang.\n\nThe Neuro-Evolution (NE) is an artificial evolution of Neural Networks (NN) using genetic algorithms in order to find\noptimal NN parameters and topology. Neuro-Evolution of NN may assume search for optimal weights of connections between\nNN nodes as well as search for optimal topology of resulting NN. The NEAT method implemented in this work do search for\nboth: optimal connections weights and topology for given task (number of NN nodes per layer and their interconnections).\n\nThe Novelty Search optimization allows solving deceptive tasks with strong local optima which can not be solved by\ntraditional objective-based fitness optimization functions. One of such problems is maze navigation where non-objective\nsearch methods like novelty search may outperform more traditional objective-based search methods. Our goal in this\nresearch is to test this hypothesis.\n\n## Minimum Requirements\n\nThe source code written and compiled against GO 1.21.x.\n\n## Installation\n\nMake sure that you have at least GO 1.17.x. environment installed onto your system and execute following command:\n```bash\n\ngo get -t github.com/yaricom/goNEAT_NS/v2\n```\n\nThis project is dependent on [goNEAT][3] project which will be installed automatically when command above executed.\n\nFor new projects, consider using the v2 of the library with the following import:\n\n```go\nimport \"github.com/yaricom/goNEAT_NS/v2\"\n```\n\n## Implementation Details\n\nThe Novelty Search optimization method implemented on top of the [goNEAT][3] library using following essential types:\n\n* [NoveltyItem](neatns/novelty_item.go) is the structure to encapsulate information about discovered novel item. It is stored in the `NoveltyArchive` and used to determine the novelty of subsequent items discovered during the evolutionary process.\n* [NoveltyArchive](neatns/novelty_archive.go) is a collection of `NoveltyItem`s discovered so far which was added by passing `novelty threshold`. Thus, the archive tries to maintain only `NoveltyItem`s which distributed evenly through the solution search space. Such approach tries to explore any possible solution instead on focusing on single solution, which can seem beneficial right now but can fail to produce the successful solver. \n\nIn order to use `NoveltyArchive` you need to provide the `NoveltyMetric` implementation which is most suitable for the use case. \n\nFor example, in *maze solver example* the `NoveltyMetric` calculates distance between path of evaluated candidate solver and already added to the archive `NoveltyItem`s. Thus, novelty of the solver determined by uniqueness of the path through the maze it was able to find during the maze solving simulation. The source code looks like following (see [maze_ns](examples/maze/maze_ns.go)):\n\n```go\nvar noveltyMetric neatns.NoveltyMetric = func(x, y *neatns.NoveltyItem) float64 {\n\tdiff := histDiff(x.Data, y.Data)\n\treturn diff\n}\n\n// calculates item-wise difference between two vectors\nfunc histDiff(left, right []float64) float64 {\n    size := len(left)\n    diffAccum := 0.0\n    for i := 0; i \u003c size; i++ {\n    diff := left[i] - right[i]\n    diffAccum += math.Abs(diff)\n    }\n    return diffAccum / float64(size)\n}\n```\n\nFor more details how to use Novelty Search implementation with [goNEAT][3] library please refer to the [maze solver example](examples/maze/maze_ns.go).\n\nThereafter, we discuss maze solver examples and compare traditional objective-based optimization against Novelty Search optimization.\n\n## Examples Overview\n\nAn illustrative domain for testing novelty search should have a deceptive fitness landscape. In such a domain, a search\nalgorithm following the fitness gradient may perform worse than an algorithm following novelty gradients because novelty\ncannot be deceived with respect to the objective; it ignores objective fitness entirely.\n\nA compelling, easily-visualized domain with this property is a two-dimensional maze navigation task, wherein an agent\nmust navigate through a maze to a chosen goal point. A reasonable fitness function for such a domain is how close the\nmaze navigator is to the goal at the end of the evaluation. Thus, dead ends that lead close to the goal are local optima\nto which an objective-based algorithm may converge, which makes a good model for deceptive problems in general. Moreover,\nby varying the structure of the maze and the starting and goal point of the robot, various classes of problems can\nbe modeled (e.g. removing the exterior walls of a maze results in a more unconstrained problem).\n\n### The Maze Agent\n\nA maze agent controlling by Artificial Neural Network [ANN] must navigate from starting point to the exit within given number\nof time steps, i.e. in fixed time. This excludes dumb random search covering majority of maze locations which will take\ngreat amount of time steps to be executed. The task is complicated by cul-de-sacs that prevent a direct route and that\ncreate local optima in the fitness landscape.\n\n![alt text][maze_agent_scheme]\n\nThe agent has *six rangefinders* (blue arrows) that indicate the distance to the nearest obstacle and *four pie-slice radar sensors* (slices of red circle)\nthat act as a compass towards the goal (maze exit), activating when a line from the goal to the center of the agent falls within the pie-slice.\n\nThe radar sensors has following FOV zones relative to agent's heading direction: FRONT, LEFT, BACK, RIGHT, or in degrees relative to the\nagent's heading: (315.0 ~ 405.0), (45.0 ~ 135.0), (135.0 ~ 225.0), (225.0 ~ 315.0). The range finder sensors\nmonitor following directions relative to the agent heading: RIGHT, FRONT-RIGHT, FRONT, FRONT-LEFT, LEFT, BACK or in degrees relative\nto agent's heading: -90.0, -45.0, 0.0, 45.0, 90.0, -180.0.\n\nThe agent’s two effectors result in forces that respectively turn and propel the robot, i.e. change it's *linear and angular velocity*.\n\n![alt text][seed_genome_graph]\n\nThus the seed genome of maze solving agent has following configuration (see [seed genome](data/mazestartgenes) for details):\n\n* ten input (sensor) neurons: six for range finders [RIGHT, FRONT-RIGHT, FRONT, FRONT-LEFT, LEFT, BACK] plus four for slice\nradar sensors with 45 degree FOV [FRONT, LEFT, BACK, RIGHT] (blue)\n* two output neurons: angular (neuron #13) and linear (neuron #14) velocity controlling effectors (red)\n* one hidden neuron (#12) to introduce non linearity (green)\n* one bias neuron (#1) to avoid zero saturation when input neurons is not activated (yellow)\n\nThe input neurons as following:\n\n* Range Finders: #2 - RIGHT, #3 - FRONT-RIGHT, #4 - FRONT, #5 - FRONT-LEFT, #6 - LEFT, #7 - BACK\n* Radar Sensors: #8 - FRONT, #9 - LEFT, #10 - BACK, #11 - RIGHT\n\n\nDuring NEAT algorithm execution with Novelty Search optimization the provided seed genome will be complexified by\nadding new nodes/links and adjusting link weights.\n\n## Experiments and Performance evaluation\n\nIn order to test hypothesis that novelty search based optimization outperforms traditional objective-based\noptimization two experiments will be studied:\n\n* the maze navigation with novelty search optimization\n* the maze navigation with objective-based optimization\n\nWe will execute experiments within maze configurations of two difficulty levels:\n\n* medium difficulty map\n* hard difficulty map\n\n### 1. The Maze Navigation with Novelty Search Optimization\n\nIn this experiment evaluated the performance of maze agent controlled by ANN which is created by NEAT algorithm with\nNovelty Search based optimization. The mentioned optimization is based on *novelty metric* calculation for each agent\nafter particular time steps of maze navigation simulation for that agent is performed. The novelty metric biases the\nsearch in a fundamentally different way than the objective-based fitness function based on distance from agent to exit.\nThe novelty metric determines the behavior-space through which search will proceed. Therefore, because what is important\nin a maze is where the solving agent ends, for the maze domain, the behavior of a navigator is defined as its ending position.\nThe novelty metric is then the N-nearest neighbor distance novelty between the ending positions of all known solving agents.\n\n\nThe effect of this novelty metric is to reward the robot for ending in a place where none have ended before; the method\nof traversal is ignored. This measure reflects that what is important is reaching a certain location (i.e. the goal)\nrather than the method of locomotion. Thus, although the novelty metric has no knowledge of the final goal, a solution\nthat reaches the goal can appear novel. In addition, the comparison between fitness-based and novelty-based search is\nfair because both scores are computed only based on the distance of the final position of the robot from other points.\n\n#### 1.1. To run experiment with medium difficulty maze map execute following commands:\n\n```bash\n\ncd $GOPATH/src/github.com/yaricom/goNEAT_NS\ngo run executor.go -out ./out/medium_mazens -context ./data/maze.neat -genome ./data/mazestartgenes -maze ./data/medium_maze.txt -experiment MAZENS\n```\nor \n```bash\nmake run-maze-ns-medium\n```\n\n**Where**: `./data/maze.neat` is the configuration of NEAT execution context, .`/data/mazestartgenes` is the start genome\nconfiguration, and `./data/medium_maze.txt` is a maze environment configuration.\n\nThis command will execute one trial with 2000 generations (or less if winner is found) over population of 250 organisms.\n\nThe experiment results will be similar to the following:\n\n```\nAverage\n\tWinner Nodes:\t16.0\n\tWinner Genes:\t29.0\n\tWinner Evals:\t16111.0\nMean\n\tComplexity:\t28.5\n\tDiversity:\t22.1\n\tAge:\t\t64.8\n```\n\nWhere:\n- **Winner nodes/genes** is number of units and links between in produced Neural Network which was able to solve XOR problem.\n- **Winner evals** is the number of evaluations of intermediate organisms/genomes before winner was found.\n- **Mean Complexity** is an average complexity (number of nodes + number of links) of best organisms per epoch for all epochs.\n- **Mean Diversity** is an average diversity (number of species) per epoch for all epochs\n- **Mean Age** is an average age of surviving species per epoch for all epochs\n\nAfter 64 generations was found near optimal winner genome configuration able to control maze solving agent and find the exit\nwith spatial error about 1.9% at the exit point. The artificial neural network produced by this genome has only 16 units\n(neurons) with three hidden neurons.\n\n![alt text][mazens_medium_winner_genome_graph]\n\nDuring the experiment novelty search optimization resulted in growing two additional hidden units (neurons) and\nintroducing recurrent link at the output neuron #13 (angular velocity effector). The recurrent link at that output neuron\nseems to have extreme importance as it's introduced at each winner genome configuration generated by solution. It seems\nreasonable because neuron #13 effects steering of the agent and need to learn more complex behaviour than neuron #14 (linear\nvelocity control).\n\nIt's interesting to note hidden neuron #91 which seems to learn complex behaviour of steering to the maze exit\nwhen it is detected rightward or behind of the agent. We've made such assumptions because of its connections with input\nsensors #2, #7 (range finders: *RIGHT, BACK*) and #10, #11 (radar sensors: *BACK, RIGHT*).\n\nThe hidden neuron #293 connected with input sensor #11 (radar sensor: *RIGHT*) learned to affect agent's steering in the direction\nof maze exit as most of the times it is at the right bottom relative to the agent.\n\nThe hidden neuron #12 which is introduced in seed genome operates as main control-and-relay switch relaying signals from sensors\n and other hidden neurons to the effectors (neurons #13, #14).\n\n![alt text][mazens_medium_winner_records]\n\nAbove is a rendering of the maze solving simulation by agents controlled with ANNs generated from genomes of all organisms\nintroduced into population until winner is found. The agents is *color coded* depending on which species the source organism\nbelongs. The fitness of agent is measured as a relative distance between it's final destination and maze exit after running simulation\nfor particular number of time steps (400 in our setup).\n\nThe initial agent position is at the top-left corner marked with green circle and maze exit at the bottom-right marked with red circle.\n\nThe top plot shows final destinations of the most fit agents (fitness \u003e= 0.8) and bottom is the rest. The results is given\nfor experimental run with winner genome configuration presented above. At that experiment was produced 32 species among which\nthe most fit ones has amounted to eight.\n\nThe path of the winner that was able to solve the medium maze is shown in the following plot.\n\n![alt text][mazens_medium_winner_path]\n\nAs you can see from the plot above the evolution guided by Novelty Search was able to find the effective\nmaze solver agent. The trajectory of the solver agent is close to optimal taking into account motion\ndynamics of the simulated robot.\n\n\n#### 1.2. To run experiment with hard difficulty maze map execute following commands:\n\n```bash\n\ncd $GOPATH/src/github.com/yaricom/goNEAT_NS\ngo run executor.go -out ./out/hard_mazens -context ./data/maze.neat -genome ./data/mazestartgenes -maze ./data/hard_maze.txt -experiment MAZENS\n\n```\nor\n```bash\nmake run-maze-ns-hard\n```\n\n![alt text][mazens_hard_winner_genome_graph]\n\nAfter 109 generations of population was found near optimal winner genome configuration able to guide maze solving agent through\nhard maze and approach the maze exit with spatial error of 2.5%. The artificial neural network produced by this genome\nhas only 17 units (neurons) with four hidden neurons to model complex learned behaviour.\n\nThe optimal genome configuration produced by growing three additional hidden units and multiple new links compared to seed\ngenome. It's interesting to note that recurrent link at output neuron #13 (angular velocity effector) was routed through\ntwo hidden neurons in contrast with medium maze where neuron #13 was simply linked to itself. This may result in more complex\nbehaviour learned especially taking into account that link pass through neuron #42 affected by range finder: *LEFT* and radar: *BACK*.\nThe neuron #42 also affected by connection with neuron #643 (affected by range finder: *LEFT*). As a result we may assume that\nit learned how steer agent when maze exit is behind and wall is at the left of it, i.e. to follow the left wall by moving\nforward.\n\nOther important point to note is about possible learned behaviour encoded by hidden neuron #297 - it's affected by input\nrange finder sensors detecting distance to obstacles at *RIGHT* and *FRONT* direction. Looking at maze configuration we\nmay assume that this neuron learned to avoid left chamber trap with extremely strong local optimum of fitness based on\nthe distance to the maze exit.\n\n![alt text][mazens_hard_winner_records]\n\nAbove is the visualization of hard maze solving by all agents until winner is found. The initial agent position is at the\nbottom-left and maze exit at the top-left of the maze. The agents is color coded based on species they belong. So, each\ndot of similar color is the final position of agent controlled by organism belonging to the same species.\n\nThe top plot shows final destinations of the most fit agents (fitness \u003e= 0.8) and bottom is the rest. The fitness of agent\nis measured as distance from it's final position to the maze exit after 400 time steps of simulation.\n\nFrom the plot we can see that winner species produced organisms that control agents in such a way that its final destinations\nis evenly distributed through the maze. As a result it was possible to produce control ANN able to solve the maze.\n\nThe path of successful solver agent is shown on the plot below.\n\n![alt text][mazens_hard_winner_path]\n\nWith hard maze configuration, the evolutionary process guided by the Novelty Search also was able to find the near-optimal path through the maze.\n\n\n### 2. The Maze Navigation with Objective-Based Fitness Optimization\n\nIn this experiment evaluated the performance of maze agent controlled by ANN which is created by NEAT algorithm with\n*objective-based fitness* optimization. The mentioned optimization is based on maximizing solving agent's fitness by following\nits objective, i.e. the distance from agent to exit. As with previous experiment the behavior of a navigator is defined as its\nending position in a maze. The fitness function is then the squared Euclidean distance between the ending position of the agent\nand maze exit.\n\nThe effect of this fitness function is to reward the solving agent for ending in a place as close to the maze exit as possible.\n\n#### 2.1. To run experiment with medium difficulty maze map execute following commands:\n\n```bash\n\ncd $GOPATH/src/github.com/yaricom/goNEAT_NS\ngo run executor.go -out ./out/mazeobj -context ./data/maze.neat -genome ./data/mazestartgenes -maze ./data/medium_maze.txt -experiment MAZEOBJ\n```\nor\n```bash\nmake run-maze-objective-medium\n```\n**Where**: `./data/maze.neat` is the configuration of NEAT execution context, .`/data/mazestartgenes` is the start genome\nconfiguration, and `./data/medium_maze.txt` is a maze environment configuration.\n\nThis command will execute one trial with 2000 generations (or less if winner is found) over population of 250 organisms.\n\nThe experiment results will be similar to the following:\n\n```\nAverage\n\tWinner Nodes:\t22.0\n\tWinner Genes:\t49.0\n\tWinner Evals:\t62168.0\nMean\n\tComplexity:\t44.6\n\tDiversity:\t27.5\n\tAge:\t\t222.0\n```\n\n![alt text][mazeobj_medium_winner_genome_graph]\n\nAfter 248 generations of population was found near optimal winner genome configuration able to guide maze solving agent through\nmedium maze and approach the maze exit with spatial error of 1.8%. The artificial neural network produced by this genome\nhas 22 units (neurons) with nine hidden neurons to model complex learned behaviour.\n\nThe genotype of the winning agent presented above has more complicated structure compared to the near optimal genome created\nby *Novelty Search* based optimization from the first experiment with more redundant neurons and links. Due to added\ncomplexity, the produced organism is less energy efficient and harder to execute at the inference time.\n\n\n![alt text][mazeobj_medium_winner_records]\n\nAbove is a rendering of the maze solving simulation by agents controlled with ANNs generated from genomes of all organisms\nintroduced into population until winner is found. The agents is color coded depending on which species the source organism\nbelongs. The fitness of agent is measured as a relative distance between it's final destination and maze exit after running\nsimulation for particular number of time steps (400 in our setup).\n\nThe initial agent position is at the top-left corner marked with green circle and maze exit at the bottom-right marked with red circle.\n\nBy comparing it with simulation based on Novelty Search optimization it may be seen that agent's final destinations is less\nevenly distributed through the maze space and some areas left completely unexplored.\n\nThe plot with path of the successful solver agent through the maze is shown on the plot below.\n\n![alt text][mazeobj_medium_winner_path]\n\nAs you can see from the plot above the goal-oriented objective search was able to find the less optimal path through the \nmaze (compare it with the NS based search above).\n\n#### 2.2. To run experiment with hard difficulty maze map execute following commands:\n\n```bash\n\ncd $GOPATH/src/github.com/yaricom/goNEAT_NS\ngo run executor.go -out ./out/mazeobj_hard -context ./data/maze.neat -genome ./data/mazestartgenes -maze ./data/hard_maze.txt -experiment MAZEOBJ\n```\nor\n```bash\nmake run-maze-objective-medium\n```\n\n**Where**: `./data/maze.neat` is the configuration of NEAT execution context, .`/data/mazestartgenes` is the start genome\nconfiguration, and `./data/hard_maze.txt` is a maze environment configuration.\n\nAfter 10 trials objective-based optimization function was **unable to produce any successful hard maze solving agent**. The hard\nmaze has deceptive cul-de-sacs, where objective-based fitness function has strong local optima. Thus when neuroevolution is\nbased on this fitness function it's unable to make a leap to the next level of genome complexity able to solve hard maze\ndeceptive traps. At the same time **novelty search based fitness function** was able to produce maze solvers able to crack\nhard maze configuration with the same ease as medium maze configuration.\n\n![alt text][mazeobj_hard_failure_records]\n\nAbove is renderings of hard maze solving agents' final destinations for several failed trials. For all 10 executed trials\nthe renderings looks similar and by examining them it can be easy detected mentioned local optima traps which prevent any\nproduced organism from solving hard maze configuration.\n\n\n## Discussion\n\nHere tested two approaches to perform solution search optimization with **NEAT** algorithm:\n\n* Novelty Search \n* Objective-based fitness optimization\n\nThe Novelty Search optimization demonstrated outstanding performance for solving of deceptive tasks when strong local optima present, such as maze solving. \n\nWe evaluated experiments based on two configurations of maze environments varied by complexity: \n\n* medium  \n* hard  \n\n### Medium maze results\n\nWith medium maze configuration both fitness function optimization methods was able to produce agents able to solve the maze:\n\n* the **Novelty Search** based agent was able to solve maze in 10 from 10 trials\n* the **Objective-Based** agent was able to solve medium maze in 9 from 10 trials\n\nThe novelty search optimization also resulted in producing more energy efficient and elegant genome for solver agent. The\nabsolute winner with NS optimization has only 15 neurons with 19 links between (Fitness: 0.984) compared to objective-based\noptimization where the best agent has 60 neurons with 214 links between (Fitness: 0.987). The provided fitness value describe\nhow close final agent's position to the maze exit after 400 time steps (where 1.0 means exact match). Full statistics of\nexperiment provided further.\n\n```\nMedium Maze Objective-Based:\n============================\n+++ Solved 9 trials from 10 +++\n\nChampion found in 3 trial run\n\tWinner Nodes:\t60\n\tWinner Genes:\t214\n\tWinner Evals:\t235020\n\n\tDiversity:\t26\n\tComplexity:\t274\n\tAge:\t\t320\n\tFitness:\t1.0\n\nAverage among winners\n\tWinner Nodes:\t34.0\n\tWinner Genes:\t110.2\n\tWinner Evals:\t160756.8\n\n\tDiversity:\t20.1\n\tComplexity:\t144.2\n\tAge:\t\t251.0\n\tFitness:\t1.0\n\nAverages for all organisms evaluated during experiment\n\tDiversity:\t18.5\n\tComplexity:\t105.5\n\tAge:\t\t165.5\n\tFitness:\t0.8\n```\n```\nMedium Maze Novelty Search:\n============================\n+++ Solved 10 trials from 10 +++\n\nChampion found in 3 trial run\n\tWinner Nodes:\t15\n\tWinner Genes:\t19\n\tWinner Evals:\t13838\n\n\tDiversity:\t23\n\tComplexity:\t34\n\tAge:\t\t45\n\tFitness:\t1.0\n\nAverage among winners\n\tWinner Nodes:\t17.1\n\tWinner Genes:\t29.1\n\tWinner Evals:\t25132.1\n\n\tDiversity:\t23.0\n\tComplexity:\t46.2\n\tAge:\t\t78.2\n\tFitness:\t1.0\n\nAverages for all organisms evaluated during experiment\n\tDiversity:\t17.6\n\tComplexity:\t43.5\n\tAge:\t\t41.1\n\tFitness:\t0.5\n```\n\n\n### Hard maze results\n\nWith hard maze configuration objective-based optimization method *failed to produce any agent able to solve this maze.*\nAt the same time Novelty Search based optimization is able to avoid deceptive strong local optima introduced in hard maze\nand produce effective solver agents in less than 300 generations over the same ten trial executions.\n\n```\nHard Maze Novelty Search:\n============================\n+++ Solved 10 trials from 10 +++\n\nChampion found in 4 trial run\n\tWinner Nodes:\t27\n\tWinner Genes:\t66\n\tWinner Evals:\t64314\n\n\tDiversity:\t27\n\tComplexity:\t93\n\tAge:\t\t235\n\tFitness:\t1.0\n\nAverage among winners\n\tWinner Nodes:\t17.6\n\tWinner Genes:\t34.2\n\tWinner Evals:\t29530.9\n\n\tDiversity:\t25.6\n\tComplexity:\t51.8\n\tAge:\t\t75.5\n\tFitness:\t1.0\n\nAverages for all organisms evaluated during experiment\n\tDiversity:\t18.6\n\tComplexity:\t46.4\n\tAge:\t\t42.7\n\tFitness:\t0.3\n```\n\n### Conclusion\n\nAs it was shown by experimental data, the Novelty Search optimization, where fitness of agent is based on novelty of the solution\nit was able to find, considerably outperforms traditional objective-based optimization and even was able to solve task where\ntraditional method failed completely.\n\nWe believe that novelty search optimization can be successfully applied to produce optimal solving agents in many areas where\n strong deceptive local fitness optima is blocking traditional objective-based methods from finding optimal or any solutions.\n\nFor more information about Novelty Search optimization please refer to original works:\n\n* [Novelty Search and the Problem with Objectives][4]\n* [Evolution Through the Search for Novelty][5]\n\n## Auxiliary Tools\n\nDuring this project development was created several tools to help with results visualization and pre-/post-processing.\n\n### Genome to GraphML converter\n\nHelps with conversion of genome data into GraphML to render genome as a graph with help of specialized software such as\n[Cytoscape](http://www.cytoscape.org)\n\nUse following command to run it:\n```bash\n\ncd $GOPATH/src/github.com/yaricom/goNEAT_NS\npython tools/genome_utils.py [in_file] --out [out_file]\n\n```\n**Where**:\n\n- `in_file` the input file to read genome data from, e.g [seed genome](data/mazestartgenes)\n- `out_file` the output file to write GraphML presentation\n\n### The agents' data records visualizer for maze solving simulations\n\nAllows visualizing recorded data of maze solving agents color coded by species they belong and separated into two groups:\nthe best and other.\n\nUse following command to run it:\n\n```bash\n\ncd $GOPATH/src/github.com/yaricom/goNEAT_NS\ngo run tools/maze_utils.go -records [records_file] -maze [maze_file] -out [out_file] -width [width] -height [height] -operation [operation]\n\n```\n**Where**:\n\n- `records_file` the file holding recorded data of maze solving by population agents\n- `maze_file` the maze configuration file, e.g. [medium_maze.txt](data/medium_maze.txt)\n- `out_file` the output file [PNG]\n- `width` the plot canvas width\n- `height` the plot canvas height\n- `operation` the name of drawing operation to perform [**draw_agents** or **draw_path**]\n  - `draw_agents` the drawing operation to render collected records of solver agents\n  - `draw_path` the operation to render path of successful maze solver through the maze\n\n\n## References:\n\n1. The original C++ NEAT implementation created by Kenneth O. Stanley, [NEAT Home Page][1]\n2. Other NEAT implementations can be found at [NEAT Software Catalog][2]\n3. Joel Lehman and Kenneth O. Stanley, [Novelty Search and the Problem with Objectives][4], Genetic Programming: Theory and Practice IX (GPTP 2011), New York, NY: Springer, 2011. pp 37–56\n4. Joel Lehman, [Evolution through the search for novelty][5], B.S. Ohio State University, 2007\n5. Kenneth O. Stanley, [PhD Dissertation: Efficient Evolution of Neural Networks through Complexification][9], Department of Computer Sciences, The University of Texas at Austin, Technical Report~AI-TR-04–39, August 2004\n6. Iaroslav Omelianenko, [Novelty Search for Autonomous Artificial Intelligent Agents][8], Medium, 2018\n7. Iaroslav Omelianenko, [Neuroevolution - evolving Artificial Neural Networks topology from the scratch][7], Medium, 2018\n8. Iaroslav Omelianenko, [The GoLang NEAT implementation][3], GitHub, 2018\n\n## Citing\n\nIf you find our work useful, please consider citing:\n\n```text\n@software{omelianenko_iaroslav_2024_10951521,\n  author       = {Omelianenko, Iaroslav},\n  title        = {{The GOLang implementation of Neuro-Evolution of Augmenting Topologies (NEAT) with Novelty Search optimization}},\n  month        = apr,\n  year         = 2024,\n  note         = {If you use this software, please cite it as below.},\n  publisher    = {Zenodo},\n  version      = {v4.0.2},\n  doi          = {10.5281/zenodo.10951521},\n  url          = {https://doi.org/10.5281/zenodo.10951521}\n}\n```\n\nThis source code maintained and managed by [Iaroslav Omelianenko][6]\n\n\u003ca href=\"https://www.buymeacoffee.com/io42\"\u003e\u003cimg src=\"https://img.buymeacoffee.com/button-api/?text=Buy me a coffee\u0026emoji=\u0026slug=io42\u0026button_colour=be38f3\u0026font_colour=ffffff\u0026font_family=Comic\u0026outline_colour=ffffff\u0026coffee_colour=FFDD00\"\u003e\u003c/a\u003e\n\n[1]:http://www.cs.ucf.edu/~kstanley/neat.html\n[2]:http://eplex.cs.ucf.edu/neat_software/\n[3]:https://github.com/yaricom/goNEAT\n[4]:https://doi.org/10.1007/978-1-4614-1770-5_3\n[5]:http://joellehman.com/lehman-dissertation.pdf\n[6]:https://io42.space\n[7]:https://medium.com/@io42/neuroevolution-evolving-artificial-neural-networks-topology-from-the-scratch-d1ebc5540d84\n[8]:https://medium.com/@io42/novelty-search-for-autonomous-artificial-intelligent-agents-78ca5143c537\n[9]:https://nn.cs.utexas.edu/downloads/papers/stanley.phd04.pdf\n\n\n[maze_agent_scheme]: https://github.com/yaricom/goNEAT_NS/blob/master/contents/maze-agent.png \"The maze agent with input sensors\"\n[seed_genome_graph]: https://github.com/yaricom/goNEAT_NS/blob/master/contents/seed_genome.png \"The seed genome graph\"\n\n[mazens_medium_winner_genome_graph]: https://github.com/yaricom/goNEAT_NS/blob/master/contents/NS_medium_16/mazens_winner_16.png \"The graph for near optimal winner genome generated by novelty search for medium maze\"\n[mazens_medium_winner_records]: https://github.com/yaricom/goNEAT_NS/blob/master/contents/NS_medium_16/ns_medium_maze_16.png \"The plot of maze agent records for medium maze when novelty search optimization applied\"\n[mazens_medium_winner_path]: https://github.com/yaricom/goNEAT_NS/blob/master/contents/ns_medium_solver_path.png \"The plot with path through the medium maze of the successful solver agent with NS optimization\"\n\n[mazens_hard_winner_genome_graph]: https://github.com/yaricom/goNEAT_NS/blob/master/contents/NS_hard_17/17_hard_mazens_winner.png \"The graph for near optimal winner genome generated by novelty search for hard maze\"\n[mazens_hard_winner_records]: https://github.com/yaricom/goNEAT_NS/blob/master/contents/NS_hard_17/17_ns_hard_maze.png \"The plot of maze agent records for hard maze when novelty search optimization applied\"\n[mazens_hard_winner_path]: https://github.com/yaricom/goNEAT_NS/blob/master/contents/ns_hard_solver_path.png \"The plot with path through the hard maze of the successful solver agent with NS optimization\"\n\n[mazeobj_medium_winner_genome_graph]: https://github.com/yaricom/goNEAT_NS/blob/master/contents/OBJ_medium_22/22_maze_obj_winner.png \"The graph for near optimal winner genome generated by objective-based fitness optimization for medium maze\"\n[mazeobj_medium_winner_records]: https://github.com/yaricom/goNEAT_NS/blob/master/contents/OBJ_medium_22/22_obj_medium_maze.png \"The plot of maze agent records for medium maze when objective-based fitness optimization applied\"\n[mazeobj_medium_winner_path]: https://github.com/yaricom/goNEAT_NS/blob/master/contents/obj_medium_solver_path.png \"The plot with path through the hard maze of the successful solver agent with goal-oriented objective search\"\n\n[mazeobj_hard_failure_records]: https://github.com/yaricom/goNEAT_NS/blob/master/contents/OBJ_hard/hard_mazeobj.png \"The plot with hard maze solving agents final positions for hard maze when objective-based fitness optimization applied\"\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyaricom%2Fgoneat_ns","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyaricom%2Fgoneat_ns","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyaricom%2Fgoneat_ns/lists"}