{"id":26753165,"url":"https://github.com/oneiben/2d-drone-rl-navigation","last_synced_at":"2026-04-17T11:31:07.847Z","repository":{"id":283247031,"uuid":"951127562","full_name":"Oneiben/2d-drone-rl-navigation","owner":"Oneiben","description":"A reinforcement learning-based 2D drone gate navigation system built from scratch using PyTorch.","archived":false,"fork":false,"pushed_at":"2025-03-22T09:06:11.000Z","size":468,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-28T13:17:58.888Z","etag":null,"topics":["actor-critic","actor-critic-algorithm","advantage-actor-critic","deep-reinforcement-learning","drone-navigation","pygame","pytorch","reinforcement-learning","robotics","scratch-implementation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Oneiben.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-19T07:48:51.000Z","updated_at":"2025-03-22T09:06:15.000Z","dependencies_parsed_at":"2025-03-19T09:34:33.347Z","dependency_job_id":null,"html_url":"https://github.com/Oneiben/2d-drone-rl-navigation","commit_stats":null,"previous_names":["oneiben/2d-drone-rl-navigation"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oneiben%2F2d-drone-rl-navigation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oneiben%2F2d-drone-rl-navigation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oneiben%2F2d-drone-rl-navigation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oneiben%2F2d-drone-rl-navigation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Oneiben","download_url":"https://codeload.github.com/Oneiben/2d-drone-rl-navigation/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246034312,"owners_count":20712857,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["actor-critic","actor-critic-algorithm","advantage-actor-critic","deep-reinforcement-learning","drone-navigation","pygame","pytorch","reinforcement-learning","robotics","scratch-implementation"],"created_at":"2025-03-28T13:18:02.000Z","updated_at":"2026-04-17T11:31:07.817Z","avatar_url":"https://github.com/Oneiben.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 2D Drone Gate Navigation (Scratch RL Implementation)\n\n![Simulation Overview](./media/gifs/simulation_overview.gif)\n\n---\n\n## Table of Contents\n\n- [2D Drone Gate Navigation (Scratch RL Implementation)](#2d-drone-gate-navigation-scratch-rl-implementation)\n  - [Table of Contents](#table-of-contents)\n  - [Overview](#overview)\n    - [Key Features](#key-features)\n  - [Advantage Actor-Critic (A2C) Algorithm](#advantage-actor-critic-a2c-algorithm)\n  - [Environment Details](#environment-details)\n    - [Observations](#observations)\n    - [Actions](#actions)\n    - [Reward Function](#reward-function)\n  - [Project Structure](#project-structure)\n  - [Installation](#installation)\n  - [Usage](#usage)\n  - [Contributing](#contributing)\n  - [License](#license)\n\n---\n\n## Overview\n\nThis project implements a **2D drone gate navigation system** using **reinforcement learning (RL) from scratch**. Unlike many RL projects that use pre-built libraries such as **Stable-Baselines3** or **RLlib**, this project is entirely built from scratch, using only **PyTorch** to define the neural networks.\n\nBy implementing RL from scratch, this project provides a deeper understanding of:\n- **Policy gradient methods**\n- **Advantage estimation (A2C)**\n- **Training loop structure**\n- **Reward shaping for real-time environments**\n\nThe objective is to train a quadrotor to successfully navigate through gates using **position-based observations** and a set of **discrete actions**.\n\n### Key Features\n\n- **Scratch RL Implementation:** No external RL libraries, just PyTorch.\n- **Deep Reinforcement Learning (DRL):** Implements the **Advantage Actor-Critic (A2C) algorithm** from scratch.\n- **2D Simulation:** Built with **PyGame**, enabling real-time visualization.\n- **Custom Reward Function:** Designed for efficient gate navigation.\n- **Custom Observation Space:** Provides relevant state information for optimal learning.\n\n---\n\n## Advantage Actor-Critic (A2C) Algorithm\n\nA2C is a policy gradient method that uses an **actor-network** to decide actions and a **critic-network** to evaluate the chosen actions. The critic helps reduce variance in policy updates, making learning more stable.\n\n- **Actor:** Learns to take actions based on observations.\n- **Critic:** Evaluates the action taken and provides a learning signal.\n- **Advantage:** Helps the network understand whether an action is better or worse than expected.\n\n![Actor Critic](./media/diagrams/actor_critic_diagram.png)\n\n---\n\n## Environment Details\n\n### Observations\n\nThe quadrotor receives the following state information at each step:\n\n1. **Quadrotor Position**:\n   - `x`: The horizontal position of the quadrotor.\n   - `y`: The vertical position of the quadrotor.\n\n2. **Quadrotor Direction**:\n   - `direction`: The current orientation of the quadrotor, represented as an integer:\n     - `0`: Right\n     - `1`: Down\n     - `2`: Left\n     - `3`: Up\n\n3. **Previous Quadrotor Position**:\n   - `prev_x`: The horizontal position of the quadrotor in the previous step.\n   - `prev_y`: The vertical position of the quadrotor in the previous step.\n\n4. **Relative Position to the Gate**:\n   - `gate_x \u003e quad_x`: `1` if the gate is to the right of the quadrotor, otherwise `0`.\n   - `gate_x \u003c quad_x`: `1` if the gate is to the left of the quadrotor, otherwise `0`.\n   - `gate_y \u003e quad_y`: `1` if the gate is below the quadrotor, otherwise `0`.\n   - `gate_y \u003c quad_y`: `1` if the gate is above the quadrotor, otherwise `0`.\n\n---\n\n### Actions\n\nThe agent can take one of the following **discrete actions**:\n1. **Increase left rotor thrust**\n2. **Increase right rotor thrust**\n3. **Equal thrust (hovering)**\n\n### Reward Function\n\nThe reward function encourages efficient gate navigation:\n- **Positive Reward**: Given when the quadrotor successfully passes through a gate or moves toward it.\n- **Negative Reward**: Assigned for collisions with obstacles or moving away from the gate.\n\n---\n\n## Project Structure\n\n```plaintext\n2d-drone-rl-navigation/\n├── media/\n│   ├── diagrams/\n│   │  └── actor_critic_diagram.jpg\n│   ├── gifs/\n│   │  └── simulation_overview.gif\n├── src/                  # Main scripts for training and testing\n│   ├── train.py          # Training agent\n│   ├── model.py          # Neural network definitions (Actor and Critic)\n│   ├── environment/      # Environment, Reward function, Observations, and agent functions\n│   ├── memory.py         # Step_Memory and Episode_Memory\n│   ├── test.py           # Testing the trained model\n├── Arial.tff             # pygame module for loading and rendering fonts\n├── LICENSE\n├── README.md\n└── requirements.txt      # Dependencies            \n```\n\n---\n\n## Installation\n\n1. **Clone the Repository:**\n   ```bash\n   git clone https://github.com/Oneiben/2d-drone-rl-navigation.git\n   cd 2d-drone-rl-navigation\n   ```\n\n2. **Create a Virtual Environment (Recommended):**\n   ```bash\n   python -m venv venv\n   source venv/bin/activate  # On Linux/macOS\n   venv\\Scripts\\activate    # On Windows\n   ```\n\n3. **Install Dependencies:**\n   ```bash\n   pip install -r requirements.txt\n   ```\n\n---\n\n## Usage\n\n1. **Train the reinforcement learning model:**\n   ```bash\n   python src/train.py\n   ```\n\n2. **Evaluate a trained model:**\n   ```bash\n   python src/test.py\n   ```\n\n---\n\n## Contributing\n\nContributions are welcome! Follow these steps to contribute:\n\n1. Fork the repository.\n2. Create a new branch:\n   ```bash\n   git checkout -b feature-name\n   ```\n3. Make your changes and commit:\n   ```bash\n   git commit -m \"Description of changes\"\n   ```\n4. Push the changes and open a pull request.\n\n---\n\n## License\n\nThis project is licensed under the **MIT License**. See the [LICENSE](LICENSE) file for more details.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foneiben%2F2d-drone-rl-navigation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foneiben%2F2d-drone-rl-navigation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foneiben%2F2d-drone-rl-navigation/lists"}