{"id":15008852,"url":"https://github.com/gsurma/atari","last_synced_at":"2025-04-09T16:19:30.613Z","repository":{"id":40655529,"uuid":"147760545","full_name":"gsurma/atari","owner":"gsurma","description":"AI research environment for the Atari 2600 games 🤖.","archived":false,"fork":false,"pushed_at":"2022-08-30T20:20:09.000Z","size":28286,"stargazers_count":257,"open_issues_count":12,"forks_count":81,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-04-09T16:19:17.979Z","etag":null,"topics":["ai","artificial-intelligence","atari","breakout","ddqn","dqn","gym","machine-learning","ml","python","python2","q-l","reinforcement-learning","rl","space-invaders"],"latest_commit_sha":null,"homepage":"https://gsurma.github.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gsurma.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"patreon":"gsurma"}},"created_at":"2018-09-07T02:37:31.000Z","updated_at":"2025-04-04T02:14:36.000Z","dependencies_parsed_at":"2023-01-16T16:15:15.733Z","dependency_job_id":null,"html_url":"https://github.com/gsurma/atari","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gsurma%2Fatari","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gsurma%2Fatari/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gsurma%2Fatari/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gsurma%2Fatari/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gsurma","download_url":"https://codeload.github.com/gsurma/atari/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248065281,"owners_count":21041872,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","artificial-intelligence","atari","breakout","ddqn","dqn","gym","machine-learning","ml","python","python2","q-l","reinforcement-learning","rl","space-invaders"],"created_at":"2024-09-24T19:21:03.095Z","updated_at":"2025-04-09T16:19:30.593Z","avatar_url":"https://github.com/gsurma.png","language":"Python","funding_links":["https://patreon.com/gsurma"],"categories":[],"sub_categories":[],"readme":"\u003ch3 align=\"center\"\u003e\n  \u003cimg src=\"assets/atari_icon_web.png\" width=\"300\"\u003e\n\u003c/h3\u003e\n\n# Atari\n\nResearch Playground built on top of [OpenAI's Atari Gym](https://gym.openai.com/envs/#atari), prepared for implementing various Reinforcement Learning algorithms.\n\nIt can emulate any of the following games:\n\t\n\u003e ['Asterix', 'Asteroids',\n\u003e 'MsPacman', 'Kaboom', 'BankHeist', 'Kangaroo',\n\u003e 'Skiing', 'FishingDerby', 'Krull', 'Berzerk',\n\u003e 'Tutankham', 'Zaxxon', 'Venture', 'Riverraid',\n\u003e 'Centipede', 'Adventure', 'BeamRider', 'CrazyClimber',\n\u003e 'TimePilot', 'Carnival', 'Tennis', 'Seaquest',\n\u003e 'Bowling', 'SpaceInvaders', 'Freeway', 'YarsRevenge',\n\u003e 'RoadRunner', 'JourneyEscape', 'WizardOfWor',\n\u003e 'Gopher', 'Breakout', 'StarGunner', 'Atlantis',\n\u003e 'DoubleDunk', 'Hero', 'BattleZone', 'Solaris',\n\u003e 'UpNDown', 'Frostbite', 'KungFuMaster', 'Pooyan',\n\u003e 'Pitfall', 'MontezumaRevenge', 'PrivateEye',\n\u003e 'AirRaid', 'Amidar', 'Robotank', 'DemonAttack',\n\u003e 'Defender', 'NameThisGame', 'Phoenix', 'Gravitar',\n\u003e 'ElevatorAction', 'Pong', 'VideoPinball', 'IceHockey',\n\u003e 'Boxing', 'Assault', 'Alien', 'Qbert', 'Enduro',\n\u003e 'ChopperCommand', 'Jamesbond']\n\nCheck out corresponding Medium article: [Atari - Reinforcement Learning in depth 🤖 (Part 1: DDQN)](https://towardsdatascience.com/atari-reinforcement-learning-in-depth-part-1-ddqn-ceaa762a546f)\n\n## Purpose\nThe ultimate goal of this project is to implement and compare various RL approaches with atari games as a common denominator.\n\n## Usage\n\n1. Clone the repo.\n2. Go to the project's root folder.\n3. Install required packages`pip install -r requirements.txt`.\n4. Launch atari. I recommend starting with help command to see all available modes `python atari.py --help`.\n\n\n## DDQN\n### Hyperparameters\n\t* GAMMA = 0.99\n\t* MEMORY_SIZE = 900000\n\t* BATCH_SIZE = 32\n\t* TRAINING_FREQUENCY = 4\n\t* TARGET_NETWORK_UPDATE_FREQUENCY = 40000\n\t* MODEL_PERSISTENCE_UPDATE_FREQUENCY = 10000\n\t* REPLAY_START_SIZE = 50000\n\t* EXPLORATION_MAX = 1.0\n\t* EXPLORATION_MIN = 0.1\n\t* EXPLORATION_TEST = 0.02\n\t* EXPLORATION_STEPS = 850000\n\n### Model Architecture\nDeep Convolutional Neural Network by [DeepMind](https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf)\n\n\u003cimg src=\"assets/network_architecture.png\" width=\"500\"\u003e\n\n\n\t* Conv2D (None, 32, 20, 20)\n\t* Conv2D (None, 64, 9, 9)\n\t* Conv2D (None, 64, 7, 7)\n\t* Flatten (None, 3136)\n\t* Dense (None, 512)\n\t* Dense (None, 4)\n\t\n\tTrainable params: 1,686,180\n\n\n\n### Performance\nAfter 5M of steps (**~40h** on Tesla K80 GPU or **~90h** on 2.9 GHz Intel i7 Quad-Core CPU):\n\n#### SpaceInvaders\n\n*Training:*\n\n\u003cimg src=\"assets/SpaceInvaders/ddqn/training_loss.png\" width=\"500\"\u003e\n\u003cimg src=\"assets/SpaceInvaders/ddqn/training_score.png\" width=\"500\"\u003e\n\nNormalized score - each reward clipped to (-1, 1)\n\n*Testing:*\n\n\u003cimg src=\"assets/SpaceInvaders/ddqn/space_invaders.gif\" width=\"440\"\u003e\n\n\u003cimg src=\"assets/SpaceInvaders/ddqn/score.png\" width=\"500\"\u003e\n\nHuman average: **~372**\n\nDDQN average: **~479 (128%)**\n\n---\n\n#### Breakout\n\n*Training:*\n\n\u003cimg src=\"assets/Breakout/ddqn/training_loss.png\" width=\"500\"\u003e\n\u003cimg src=\"assets/Breakout/ddqn/training_score.png\" width=\"500\"\u003e\n\nNormalized score - each reward clipped to (-1, 1)\n\n*Testing:*\n\n\u003cimg src=\"assets/Breakout/ddqn/breakout.gif\" width=\"440\"\u003e\n\u003cimg src=\"assets/Breakout/ddqn/score.png\" width=\"500\"\u003e\n\nHuman average: **~28**\n\nDDQN average: **~62 (221%)**\n\n\n\n## Genetic Evolution\n\n\n#### Atlantis\n\n*Training:*\n\n\u003cimg src=\"assets/Atlantis/ge/score.png\" width=\"500\"\u003e\n\nNormalized score - each reward clipped to (-1, 1)\n\n*Testing:*\n\n\u003cimg src=\"assets/Atlantis/ge/atlantis.gif\" width=\"440\"\u003e\n\u003cimg src=\"assets/Atlantis/ge/testing_score.png\" width=\"500\"\u003e\n\n\nHuman average: **~29,000**\n\nGE average: **31,000 (106%)**\n\n## Author\n\n**Greg (Grzegorz) Surma**\n\n[**PORTFOLIO**](https://gsurma.github.io)\n\n[**GITHUB**](https://github.com/gsurma)\n\n[**BLOG**](https://medium.com/@gsurma)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgsurma%2Fatari","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgsurma%2Fatari","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgsurma%2Fatari/lists"}