{"id":13405676,"url":"https://github.com/mdibaiee/flappy-es","last_synced_at":"2025-12-30T00:29:19.114Z","repository":{"id":66324800,"uuid":"86984080","full_name":"mdibaiee/flappy-es","owner":"mdibaiee","description":"Flappy Bird AI using Evolution Strategies","archived":false,"fork":false,"pushed_at":"2021-11-08T11:20:16.000Z","size":3487,"stargazers_count":141,"open_issues_count":0,"forks_count":16,"subscribers_count":16,"default_branch":"master","last_synced_at":"2024-07-31T19:46:59.088Z","etag":null,"topics":["artificial-intelligence","evolution-strategies","evolution-strategy","flappy-bird","machine-learning","reinforcement-learning","unsupervised-learning"],"latest_commit_sha":null,"homepage":"https://lab.mahdi.blog/flappy-bird/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mdibaiee.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-04-02T12:16:35.000Z","updated_at":"2024-01-04T16:12:52.000Z","dependencies_parsed_at":"2023-03-09T20:45:19.881Z","dependency_job_id":null,"html_url":"https://github.com/mdibaiee/flappy-es","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdibaiee%2Fflappy-es","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdibaiee%2Fflappy-es/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdibaiee%2Fflappy-es/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdibaiee%2Fflappy-es/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mdibaiee","download_url":"https://codeload.github.com/mdibaiee/flappy-es/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221458135,"owners_count":16825271,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","evolution-strategies","evolution-strategy","flappy-bird","machine-learning","reinforcement-learning","unsupervised-learning"],"created_at":"2024-07-30T19:02:08.162Z","updated_at":"2025-12-30T00:29:19.076Z","avatar_url":"https://github.com/mdibaiee.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"Playing Flappy Bird using Evolution Strategies\n==============================================\n\nAfter reading [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://blog.openai.com/evolution-strategies/), I wanted to experiment something using Evolution Strategies, and Flappy Bird has always been one of my favorites when it comes to Game experiments. A simple yet challenging game.\n\nThe model learns to play very well after 3000 epochs, but not completely flawless and it rarely loses in difficult cases (high difference between two wall entrances).\nTraining process is pretty fast as there is no backpropagation, and is not very costy in terms of memory as there is no need to record actions as in policy gradients.\n\nHere is a demonstration of the model after 3000 epochs (~5 minutes on an Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz):\n\n![after training](/demo/flappy-success.gif)\n\nBefore training:\n\n![Before training](/demo/flappy-lose.gif)\n\nThere is also a [a web version](https://lab.mahdi.blog/flappy-bird/) available for ease of access.\n\nFor each frame the bird stays alive, +0.1 score is given to him. For each wall he passes, +10 score is given.\n\nDemonstration of rewards for individuals and the mean reward over time (y axis is logarithmic):\n![reward chart](fig-log.jpg)\n\nTry it yourself\n---------------\nYou need python3.5 and pip for installing and running the code.\n\nFirst, install dependencies (you might want to create a [virtualenv](https://virtualenv.pypa.io)):\n\n```\npip install -r requirements\n```\n\nThe pretrained parameters are in a file named `load.npy` and will be loaded when you run `train.py` or `demo.py`.\n\n`train.py` will train the model, saving the parameters to `saves/\u003cTIMESTAMP\u003e/save-\u003cITERATION\u003e`.\n\n`demo.py` shows the game in a GTK window so you can see how the AI actually plays (like the GIF above).\n\n`play.py` if you feel like playing the game yourself, space: jump, once lost, press enter to play again. :grin:\n\n_pro tip: reach 100 score and you will become THUG FOR LIFE :smoking:_\n\nNotes\n-----\n\nIt seems training past a maximum point reduces performance, learning rate decay might help with that.\nMy interpretation is that after finding a local maximum for accumulated reward and being able to receive high rewards,\nthe updates become pretty large and will pull the model too much to sides, thus the model will enter a state of oscillation.\n\nTo try it yourself, there is a `long.npy` file, rename it to `load.npy` (backup `load.npy` before doing so) and run `demo.py`,\nyou will see the bird failing more often than not. `long.py` was trained for only 100 more epochs than `load.npy`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmdibaiee%2Fflappy-es","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmdibaiee%2Fflappy-es","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmdibaiee%2Fflappy-es/lists"}