{"id":15138390,"url":"https://github.com/ugurkanates/spacexreinforcementlearning","last_synced_at":"2025-10-23T15:30:24.800Z","repository":{"id":110721921,"uuid":"274231892","full_name":"ugurkanates/SpaceXReinforcementLearning","owner":"ugurkanates","description":"SpaceX Falcon 9 simulated with Reinforcement Learning algorithms such as D4PG,SAC and PPO.","archived":false,"fork":false,"pushed_at":"2020-06-28T14:19:59.000Z","size":10453,"stargazers_count":57,"open_issues_count":0,"forks_count":7,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-01-30T19:08:08.738Z","etag":null,"topics":["reinforcement-learning","reinforcement-learning-algorithms","reinforcement-learning-environments","rocket","spacex"],"latest_commit_sha":null,"homepage":"https://medium.com/@paypaytr/spacex-falcon-9-landing-with-rl-7dde2374eb71","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ugurkanates.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.rst","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-06-22T20:09:24.000Z","updated_at":"2024-12-17T12:37:19.000Z","dependencies_parsed_at":null,"dependency_job_id":"85719f79-0b2d-45fe-95cd-aa70895aa1a9","html_url":"https://github.com/ugurkanates/SpaceXReinforcementLearning","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ugurkanates%2FSpaceXReinforcementLearning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ugurkanates%2FSpaceXReinforcementLearning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ugurkanates%2FSpaceXReinforcementLearning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ugurkanates%2FSpaceXReinforcementLearning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ugurkanates","download_url":"https://codeload.github.com/ugurkanates/SpaceXReinforcementLearning/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":237843742,"owners_count":19375195,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["reinforcement-learning","reinforcement-learning-algorithms","reinforcement-learning-environments","rocket","spacex"],"created_at":"2024-09-26T07:23:09.040Z","updated_at":"2025-10-23T15:30:17.822Z","avatar_url":"https://github.com/ugurkanates.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SpaceX Falcon 9 Landing with Reinforcement Learning \n\n# ![GIF](videogifwhatever.gif)\n\nThe Falcon 9, developed by aerospace company SpaceX, means it is now possible to reuse the first-stage of the rocket, by flying it safely back to Earth.\n\nAn achievement once seemed so impossible that it lead creation of multiple “fake SpaceX landing videos-explanations” now is widely agreed upon about how amazing behind the tech related to it.\n\nWhile today I’m not here nor capable of you giving rocket engineering course just wanted to show this quick little diagram from SpaceX to understand a little bit more.\n\nEnvironment is created by [Sven Niederberger](https://github.com/EmbersArc) and it's based on LunarLander environment by OpenAI.\n\n \n\nIt offers you to include or exclude Velocity information for your Agent's Observation. \n\nYou can either train in **DISCRETE ACTION**  or **CONTINUOUS ACTION** modes.\nYou can find more information at (gym/env/box2d/rocket_lander.py)\n\n\n\n### STATE VARIABLES  \nThe state consists of the following variables:\n  * X position  \n  * Y position  \n  * Angle  \n  * First Leg Ground Contact Indicator  \n  * Second Leg Ground Contact Indicator  \n  * Throttle  \n  * Engine Gimbal  \n  \nIf VEL_STATE is set to true, the velocities are included:  \n  * X velocity  \n  * Y velocity  \n  * Angular Velocity  \n  \nAll state variables are normalized for improved training.\n    \n\n## Discrete control inputs are:\n* Gimbal Left\n* Gimbal Right\n* Throttle Up\n* Throttle Down\n* Use First Control Thruster\n* Use Second Control Thruster\n* No Action\n## Continuous control inputs are:\n* Gimbal (Left/Right)\n* Throttle (Up/Down)\n* Control Thruster (Left/Right)\n\nSince this is repository includes a modified version of **OpenAI GYM** , I won't recommend to install with PIP.\n\nYou can basically struct your code to execute from this directory so your own installed GYM package wouldn't interfere with this project. \n\nThere are also Google Colab Notebooks that's ready to train and can record videos \u0026 save your network modules into your Google Drive.\nIf you work on local machine you can basically skip Drive related commands.\n\nThere are 3 agents in this project.I left ***PPO*** as default. If you want to run ***D4PG*** or ***SAC*** algorithms just rename LIB folders correctly\n(lib -\u003e lib_PPO ,  libSAC-\u003e lib  or libD4PG -\u003e lib) you get the idea.\n\nAgent creation done thanks to PTan library. You can find more information about it link below.\n\n[PTan Agent Network Library](https://github.com/Shmuma/ptan)\n\nIf you have anyting to ask regarding to project or RL in general feel free to ask me via e-mail in my profile.\nHave a nice flight !\n\n\n## Tensorboard X \n\nIn order to see logs of your train run , you can execute command below\n\ntensorboard --logdir runs/ --host localhost\n\n## Name of training run\n\nDon't forget to change training run each time you execute code or otherwise it will overwrite of older files.\nAlso don't forget to change video_directory if you want to save videos.\n\n## How to test \n\nI already saved bunch of network models in **/rocket_saved_networks/** folder.  You can try them out with play function ( feel free to edit if you have custom agents or if you try other networks you may need to load actor model etc.)\n\n\n[![Everything Is AWESOME](youtube.png)](https://www.youtube.com/watch?v=z-j82WmZojY \"Everything Is AWESOME\")\n\n## Detailed Medium Post\n\nhttps://medium.com/@paypaytr/spacex-falcon-9-landing-with-rl-7dde2374eb71\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fugurkanates%2Fspacexreinforcementlearning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fugurkanates%2Fspacexreinforcementlearning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fugurkanates%2Fspacexreinforcementlearning/lists"}