{"id":18551325,"url":"https://github.com/morvanzhou/mevo","last_synced_at":"2025-04-09T22:31:34.503Z","repository":{"id":65640216,"uuid":"596354733","full_name":"MorvanZhou/mevo","owner":"MorvanZhou","description":"Evolutionary algorithms, alternative to Reinforcement Learning","archived":false,"fork":false,"pushed_at":"2023-02-14T02:58:51.000Z","size":1720,"stargazers_count":38,"open_issues_count":0,"forks_count":6,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-24T12:55:32.462Z","etag":null,"topics":["evolution-strategies","evolutionary-algorithms","genetic-algorithm","neural-evolution","reinforcement-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MorvanZhou.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-02-02T01:57:13.000Z","updated_at":"2025-03-03T19:13:37.000Z","dependencies_parsed_at":"2023-02-19T15:15:17.552Z","dependency_job_id":null,"html_url":"https://github.com/MorvanZhou/mevo","commit_stats":{"total_commits":28,"total_committers":1,"mean_commits":28.0,"dds":0.0,"last_synced_commit":"8904213a86c0d9dc72983f56c288eaf8d23f2641"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MorvanZhou%2Fmevo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MorvanZhou%2Fmevo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MorvanZhou%2Fmevo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MorvanZhou%2Fmevo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MorvanZhou","download_url":"https://codeload.github.com/MorvanZhou/mevo/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248123595,"owners_count":21051500,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["evolution-strategies","evolutionary-algorithms","genetic-algorithm","neural-evolution","reinforcement-learning"],"created_at":"2024-11-06T21:08:39.934Z","updated_at":"2025-04-09T22:31:33.441Z","avatar_url":"https://github.com/MorvanZhou.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MEvo\n\n---\n[![Unittest](https://github.com/MorvanZhou/mevo/actions/workflows/python-package.yml/badge.svg)](https://github.com/MorvanZhou/mevo/actions/workflows/python-package.yml)\n[![GitHub license](https://img.shields.io/github/license/MorvanZhou/mevo)](https://github.com/MorvanZhou/mevo/blob/master/LICENSE)\n\nMEvo is a evolutionary algorithm package, which implements Genetic Algorithm and Evolution Strategies Algorithms.\nIt can be used to form an agent strategy or optimize math problems.\n\n# Quick look\n\nTraining a cartpole policy by using Genetic Algorithm.\n\n```python\nimport mevo\nimport gymnasium\nimport numpy as np\n\n\n# define a fitness function to get fitness for every individual\ndef fitness_fn(ind: mevo.individuals.Individual, conf: dict) -\u003e float:\n    ep_r = 0\n    env = gymnasium.make('CartPole-v1')\n    for _ in range(2):\n        s, _ = env.reset()\n        for _ in range(500):  # in one episode\n            logits = ind.predict(s)\n            a = np.argmax(logits)\n            s, r, done, _, _ = env.step(a)\n            ep_r += r\n            if done:\n                break\n    return ep_r\n\n\n# training\nwith mevo.GeneticAlgoNet(max_size=20, layer_size=[4, 8, 2], drop_rate=0.7, mutate_rate=0.5) as pop:\n    for generation in range(40):\n        pop.evolve(fitness_fn=fitness_fn)\n        print(f\"generation={generation}, top_fitness={pop.top.fitness:.f2}\")\n```\n\nAfter only 40 generations in a population (20 individuals), it gives a greate result.\nUse following code to visualize the learned policy.\n\n```python\n# deploy the best individual\nenv = gymnasium.make('CartPole-v1', render_mode=\"human\")\nwhile True:\n    s, _ = env.reset()\n    while True:  # in one episode\n        logits = pop.top.predict(s)\n        a = np.argmax(logits)\n        s, _, done = env.step(a)[:3]\n        if done:\n            break\n```\n\n![cartpole](https://github.com/MorvanZhou/mevo/raw/main/images/cartpole.gif)\n\n# What is MEvo\n\nIn the MEvo, the smallest data segment is Gene. A set of gene can be packed into a Chromosome.\nChromosomes then formed an Individual. A Population consists of many individuals.\n\n![data](https://github.com/MorvanZhou/mevo/raw/main/images/dataStructural.png)\n\nIt is different from the classical Genetic Algorithms whose chromosome only consists of a 1D array.\nIn other words, chromosome in the MEvo is n-D array with different shapes.\n\n![data](https://github.com/MorvanZhou/mevo/raw/main/images/dataShape.png)\n\n**Why making data shape varied?**\n\nBecause we want to possibly evolve a neural network which has multiple layers in various shape.\n\nMEvo has two different algorithm families.\n\n- Genetic Algorithm family\n- Evolution Strategy family\n\nThe Genetic Algorithm basically has two steps:\n\n1. crossover the chromosomes in a population\n2. mutate gene in new generation\n\nThe following image shows how crossover on two parents chromosomes,\nand how to mutate the new chromosome.\n\n![crossoverMutation](https://github.com/MorvanZhou/mevo/raw/main/images/crossoverMutation.png)\n\nAnd MEvo's Evolution Strategy has two different steps:\n\n1. mutate gene (normal distribution) from same chromosomes\n2. update this chromosome by the performance of all mutated children\n\n![es](https://github.com/MorvanZhou/mevo/raw/main/images/es.png)\n\n# Parallel training\n\nMEvo support parallel training. Simply set `n_worker` \u003e 1 to unlock your machine power for training on multiple cores.\nWhen `n_worker=-1` then use all your cores.\n\nSimply replace the `pop` definition by including `n_worker` from previous code.\n\n```python\n# parallel training\nif __name__ == \"__main__\":\n    with mevo.GeneticAlgoNet(max_size=20, layer_size=[4, 8, 2], n_worker=-1, drop_rate=0.7, mutate_rate=0.5) as pop:\n        for generation in range(40):\n            pop.evolve(fitness_fn=fitness_fn)\n            print(f\"generation={generation}, top_fitness={pop.top.fitness:.2f}\")\n```\n\nNote that the parallel code must be run under `if __name__ == \"__main__\"：` scope,\notherwise a python multiprocessing error will occur.\n\n# Install\n\nUsing pip to install MEvo\n\n```shell\npip3 install mevo\n```\n\n# Methods and Parameters\n\nMEvo support following populations:\n\n- mevo.GeneticAlgoInt()\n- mevo.GeneticAlgoFloat()\n- mevo.GeneticAlgoOrder()\n- mevo.GeneticAlgoNet()\n- mevo.EvolutionStrategyNet()\n\n## A classical genetic algorithm problem\n\nsuch as Travel Sales Problem (TSP) can be solved by following classical Populations:\n\n- mevo.GeneticAlgoInt()\n- mevo.GeneticAlgoFloat()\n- mevo.GeneticAlgoOrder()\n\n### A Travel Sales Problem example using `pop = mevo.GeneticAlgoOrder()`:\n\n```python\nimport mevo\nimport numpy as np\n\npositions = [np.random.rand(2) for _ in range(20)]\n\n\ndef distance_fitness_fn(ind: mevo.individuals.Individual, conf: dict) -\u003e float:\n    order = [c.data[0] for c in ind.chromosomes]\n    cost = 0\n    for i in range(len(order) - 1):\n        p1, p2 = positions[order[i]], positions[order[i + 1]]\n        cost += np.square(p1 - p2).sum()\n    fitness = -cost\n    return fitness\n\n\npop = mevo.GeneticAlgoOrder(\n    max_size=50,\n    chromo_size=len(positions),\n    drop_rate=0.3,\n    mutate_rate=0.01,\n)\npop.run(fitness_fn=distance_fitness_fn, step=30)\n```\n\n### An optimization problem example using `pop = mevo.GeneticAlgoInt()`:\n\n```python\nimport mevo\nimport numpy as np\n\n\ndef wave_fitness_fn(ind: mevo.individuals.Individual, conf: dict) -\u003e float:\n    binary = [c.data for c in ind.chromosomes]\n    c = np.concatenate(binary, axis=0)\n    a = 2 ** np.arange(len(c))[::-1]\n    decimal = c.dot(a)\n\n    x = decimal / float(2 ** len(c) - 1) * 5\n    o = np.sin(10 * x) * x + np.cos(2 * x) * x\n    return o\n\n\npop = mevo.GeneticAlgoInt(\n    max_size=20,\n    chromo_size=10,\n    drop_rate=0.3,\n    chromo_initializer=mevo.chromosomes.initializers.RandomInt(0, 2),\n    mutate_rate=0.01,\n)\npop.run(step=20, fitness_fn=wave_fitness_fn)\n```\n\n## Deep Net Evolution\n\nFor policy learning or Reinforcement learning alternative, the following two methods has their advantages.\n\n- mevo.GeneticAlgoNet()\n- mevo.EvolutionStrategyNet()\n\n|              | Reinforcement Learning               | MEvo                                                                                                  |\n|--------------|--------------------------------------|-------------------------------------------------------------------------------------------------------|\n| Training     | Has forward and backward propagation | Only has forward propagation, but need crossover or mutation operation (lighter than backpropagation) |\n| Exploration  | Needs carefully set explore policy   | Different children setting automatically ensure the exploration                                       |\n| Memory needs | Can only keep one set of parameters  | In each generation, must compute all children's parameters (parallel computing save time)             |\n| Network Size | Generally large and deep net         | With a large scale exploration, a relatively small net can perform a good job                         |\n\n### A `mevo.GeneticAlgoNet()` example:\n\n```python\nimport mevo\nimport gymnasium\nimport numpy as np\n\n\ndef fitness_fn(ind: mevo.individuals.Individual, conf: dict) -\u003e float:\n    ep_r = 0\n    env = gymnasium.make('Pendulum-v1')\n    env.reset(seed=conf[\"seed\"])\n    for _ in range(2):\n        s, _ = env.reset()\n        for _ in range(150):  # in one episode\n            logits = ind.predict(s)\n            a = np.tanh(logits) * 2\n            s, r, _, _, _ = env.step(a)\n            ep_r += r\n    return ep_r\n\n\ndef train():\n    with mevo.GeneticAlgoNet(\n            max_size=30,\n            layer_size=[3, 32, 1],\n            drop_rate=0.7,\n            mutate_rate=0.5,\n            n_worker=-1,\n    ) as pop:\n        for ep in range(700):\n            pop.evolve(\n                fitness_fn=fitness_fn,\n            )\n            print(ep, pop.top.fitness)\n    return pop.top\n\n\ndef show(top):\n    env = gymnasium.make('Pendulum-v1', render_mode=\"human\")\n    while True:\n        s, _ = env.reset()\n        for _ in range(200):  # in one episode\n            logits = top.predict(s)\n            a = np.tanh(logits) * 2\n            s, _, _, _, _ = env.step(a)\n\n\nif __name__ == \"__main__\":\n    top = train()\n    show(top)\n```\n\n![es](https://github.com/MorvanZhou/mevo/raw/main/images/pendulum_ga.gif)\n\n### A `mevo.EvolutionStrategyNet()` example:\n\n```python\nimport mevo\nimport gymnasium\nimport numpy as np\n\n\ndef fitness_fn(ind: mevo.individuals.EvolutionStrategyDense, conf: dict) -\u003e float:\n    ep_r = 0\n    seed = conf[\"seed\"]\n    index = conf[\"index\"]\n\n    env = gymnasium.make('Pendulum-v1')\n\n    # ! must set seed and clone when using mevo.EvolutionStrategyNet()\n    env.reset(seed=conf[\"seed\"])\n    c_ind = ind.clone_with_mutate(index, seed)\n    # ###############\n\n    for _ in range(2):\n        s, _ = env.reset()\n        for _ in range(100):\n            logits = c_ind.predict(s)\n            a = np.tanh(logits) * 2\n            s, r, _, _, _ = env.step(a)\n            ep_r += r\n    return ep_r\n\n\ndef train():\n    with mevo.EvolutionStrategyNet(\n            max_size=15,\n            layer_size=[3, 32, 1],\n            mutate_strength=0.05,\n            learning_rate=0.1,\n            n_worker=-1,\n            seed=2\n    ) as pop:\n        for ep in range(700):\n            pop.evolve(fitness_fn=fitness_fn)\n            print(ep, pop.top.fitness)\n    return pop.top\n\n\ndef show(top):\n    env = gymnasium.make(\n        'Pendulum-v1',\n        render_mode=\"human\"\n    )\n    while True:\n        s, _ = env.reset()\n        for _ in range(200):  # in one episode\n            logits = top.predict(s)\n            a = np.tanh(logits) * 2\n            s, _, _, _, _ = env.step(a)\n\n\nif __name__ == \"__main__\":\n    top = train()\n    show(top)\n```\n\n![es](https://github.com/MorvanZhou/mevo/raw/main/images/pendulum_es.gif)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmorvanzhou%2Fmevo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmorvanzhou%2Fmevo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmorvanzhou%2Fmevo/lists"}