https://github.com/bashmocha/reinforcement-learning-in-airsim
Autonomous Driving in AirSim by Reinforcement Learning
https://github.com/bashmocha/reinforcement-learning-in-airsim
airsim autonomous-driving autonomous-vehicles deep-q-learning deep-reinforcement-learning path-tracing reinforcement-learning
Last synced: 7 months ago
JSON representation
Autonomous Driving in AirSim by Reinforcement Learning
- Host: GitHub
- URL: https://github.com/bashmocha/reinforcement-learning-in-airsim
- Owner: BashMocha
- Created: 2025-01-04T16:04:37.000Z (9 months ago)
- Default Branch: master
- Last Pushed: 2025-03-08T12:45:30.000Z (7 months ago)
- Last Synced: 2025-03-19T15:17:06.273Z (7 months ago)
- Topics: airsim, autonomous-driving, autonomous-vehicles, deep-q-learning, deep-reinforcement-learning, path-tracing, reinforcement-learning
- Language: Python
- Homepage:
- Size: 6.09 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
## Autonomous Driving in AirSim by Reinforcement Learning
> Autonomous driving for path tracking in AirSim by applying the Deep-Q Network algorithm. See the [documentation](https://github.com/BashMocha/Reinforcement-Learning-in-AirSim/blob/master/docs/Reinforcement_Learning_on_Autonomous_Vehicles.pdf) for a detailed explanation.
https://github.com/user-attachments/assets/b0d4f22b-c0f4-4c71-89e6-18a2df161953
---
## Simulation Environment
[AirSim's](https://github.com/microsoft/AirSim/) neighborhood environment [(AirSimNH)](https://github.com/microsoft/AirSim/releases/tag/v1.8.1-windows) is utilized due to its simple structure, making it well-suited for path-tracking algorithms. The project has been tested on Windows 11 but not on Linux; therefore, running the scripts on a Windows operating system is recommended.
The simulation environment can be downloaded from the project's [release page](https://github.com/microsoft/AirSim/releases/tag/v1.8.1-windows). To configure the environment, move the `settings.json` file to the `%userprofile%\Documents\AirSim` and run the `AirSimNH\WindowsNoEditor\AirsimNH.exe` from the environment folder. Once the simulation is running, a Python script containing the following lines can connect to the environment for API calls:
```python
import airsim
client = airsim.CarClient()
client.confirmConnection()
client.enableApiControl(True)
car_controls = airsim.CarControls()
```
## Conda environment
Run the following commands to create and activate the conda environment with the required dependencies
```
conda env create -f environment.yml
conda activate airsim
```
## Usage
### Training
Once the simulation environment is running, execute the `train.py` script to connect to it. By default, the agent is trained for 1000 epochs, but this can be adjusted using the `episodes` parameter in the script.
```
python train.py
```
Since this is a path-tracking application, the reference route is specified by the `waypoints` list in the script.
```python
waypoints = [
# Starting Street
airsim.Vector3r(-3.8146975356312396e-08, 7.445784285664558e-05, -0.5857376456260681), # Starting point
airsim.Vector3r(21.836427688598633, -0.024445464834570885, -0.5837180614471436), # White car
airsim.Vector3r(51.68717575073242, -0.5642141103744507, -0.584981381893158), # Red car
airsim.Vector3r(80.388427734375, -1.1560953855514526, -0.5853434801101685), # Near end of the street
]
```
Run `custom_collect_poses.py` to control the car for position data collection. The collected position data will be stored in `%userprofile%\Documents\AirSim`.
```
python custom_collect_poses.py
```
### Transfer Learning
The network weights will be saved after training. Running `load_model.py` will load and execute the saved weights.
```
python load_model.py
```
## Path Evaluation
To evaluate the model's path-tracking performance, robotics navigation metrics (SR, OSR, NE, SDTW, NDTW, CLS) are provided in the `evaluation.py` script. Executing the script will generate a comparison plot of the resulting and reference trajectories.
See the [documentation](https://github.com/BashMocha/Reinforcement-Learning-in-AirSim/blob/master/docs/Reinforcement_Learning_on_Autonomous_Vehicles.pdf) for a detailed explanation of metrics.
```
python evaluation.py
```
## DQN Algorithm
A simple DQN is built to train the agents for the given path. RGB and GPS sensors are used as inputs to the network, and an action is obtained as the output. This approach enables the agent to track the path and avoid obstacles effectively.
See the [documentation](https://github.com/BashMocha/Reinforcement-Learning-in-AirSim/blob/master/docs/Reinforcement_Learning_on_Autonomous_Vehicles.pdf) for a detailed explanation of the utilized network.
