Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://microsoft.github.io/AirSim/

Open source simulator for autonomous vehicles built on Unreal Engine / Unity, from Microsoft AI & Research
https://microsoft.github.io/AirSim/

ai airsim artificial-intelligence autonomous-quadcoptor autonomous-vehicles computer-vision control-systems cross-platform deep-reinforcement-learning deeplearning drones pixhawk platform-independent research self-driving-car simulator unreal-engine

Last synced: about 1 month ago
JSON representation

Open source simulator for autonomous vehicles built on Unreal Engine / Unity, from Microsoft AI & Research

Lists

README 

        
## AirSim announcement: This repository will be archived in the coming year 



In 2017 Microsoft Research created AirSim as a simulation platform for AI research and experimentation. Over the span of five years, this research project has served its purpose—and gained a lot of ground—as a common way to share research code and test new ideas around aerial AI development and simulation. Additionally, time has yielded advancements in the way we apply technology to the real world, particularly through aerial mobility and autonomous systems. For example, drone delivery is no longer a sci-fi storyline—it’s a business reality, which means there are new needs to be met. We’ve learned a lot in the process, and we want to thank this community for your engagement along the way. 



In the spirit of forward momentum, we will be releasing a new simulation platform in the coming year and subsequently archiving the original 2017 AirSim. Users will still have access to the original AirSim code beyond that point, but no further updates will be made, effective immediately. Instead, we will focus our efforts on a new product, Microsoft Project AirSim, to meet the growing needs of the aerospace industry. Project AirSim will provide an end-to-end platform for safely developing and testing aerial autonomy through simulation. Users will benefit from the safety, code review, testing, advanced simulation, and AI capabilities that are uniquely available in a commercial product. As we get closer to the release of Project AirSim, there will be learning tools and features available to help you migrate to the new platform and to guide you through the product. To learn more about building aerial autonomy with the new Project AirSim, visit [https://aka.ms/projectairsim](https://aka.ms/projectairsim).



# Welcome to AirSim



AirSim is a simulator for drones, cars and more, built on [Unreal Engine](https://www.unrealengine.com/) (we now also have an experimental [Unity](https://unity3d.com/) release). It is open-source, cross platform, and supports software-in-the-loop simulation with popular flight controllers such as PX4 & ArduPilot and hardware-in-loop with PX4 for physically and visually realistic simulations. It is developed as an Unreal plugin that can simply be dropped into any Unreal environment. Similarly, we have an experimental release for a Unity plugin.



Our goal is to develop AirSim as a platform for AI research to experiment with deep learning, computer vision and reinforcement learning algorithms for autonomous vehicles. For this purpose, AirSim also exposes APIs to retrieve data and control vehicles in a platform independent way.



**Check out the quick 1.5 minute demo**



Drones in AirSim



[![AirSim Drone Demo Video](docs/images/demo_video.png)](https://youtu.be/-WfTr1-OBGQ)



Cars in AirSim



[![AirSim Car Demo Video](docs/images/car_demo_video.png)](https://youtu.be/gnz1X3UNM5Y)



## How to Get It



### Windows

[![Build Status](https://github.com/microsoft/AirSim/actions/workflows/test_windows.yml/badge.svg)](https://github.com/microsoft/AirSim/actions/workflows/test_windows.yml)

* [Download binaries](https://github.com/Microsoft/AirSim/releases)

* [Build it](https://microsoft.github.io/AirSim/build_windows)



### Linux

[![Build Status](https://github.com/microsoft/AirSim/actions/workflows/test_ubuntu.yml/badge.svg)](https://github.com/microsoft/AirSim/actions/workflows/test_ubuntu.yml)

* [Download binaries](https://github.com/Microsoft/AirSim/releases)

* [Build it](https://microsoft.github.io/AirSim/build_linux)



### macOS

[![Build Status](https://github.com/microsoft/AirSim/actions/workflows/test_macos.yml/badge.svg)](https://github.com/microsoft/AirSim/actions/workflows/test_macos.yml)

* [Build it](https://microsoft.github.io/AirSim/build_macos)



For more details, see the [use precompiled binaries](docs/use_precompiled.md) document. 



## How to Use It



### Documentation



View our [detailed documentation](https://microsoft.github.io/AirSim/) on all aspects of AirSim.



### Manual drive



If you have remote control (RC) as shown below, you can manually control the drone in the simulator. For cars, you can use arrow keys to drive manually.



[More details](https://microsoft.github.io/AirSim/remote_control)



![record screenshot](docs/images/AirSimDroneManual.gif)



![record screenshot](docs/images/AirSimCarManual.gif)



### Programmatic control



AirSim exposes APIs so you can interact with the vehicle in the simulation programmatically. You can use these APIs to retrieve images, get state, control the vehicle and so on. The APIs are exposed through the RPC, and are accessible via a variety of languages, including C++, Python, C# and Java.



These APIs are also available as part of a separate, independent cross-platform library, so you can deploy them on a companion computer on your vehicle. This way you can write and test your code in the simulator, and later execute it on the real vehicles. Transfer learning and related research is one of our focus areas.



Note that you can use [SimMode setting](https://microsoft.github.io/AirSim/settings#simmode) to specify the default vehicle or the new [ComputerVision mode](https://microsoft.github.io/AirSim/image_apis#computer-vision-mode-1) so you don't get prompted each time you start AirSim.



[More details](https://microsoft.github.io/AirSim/apis)



### Gathering training data



There are two ways you can generate training data from AirSim for deep learning. The easiest way is to simply press the record button in the lower right corner. This will start writing pose and images for each frame. The data logging code is pretty simple and you can modify it to your heart's content.



![record screenshot](docs/images/record_data.png)



A better way to generate training data exactly the way you want is by accessing the APIs. This allows you to be in full control of how, what, where and when you want to log data.



### Computer Vision mode



Yet another way to use AirSim is the so-called "Computer Vision" mode. In this mode, you don't have vehicles or physics. You can use the keyboard to move around the scene, or use APIs to position available cameras in any arbitrary pose, and collect images such as depth, disparity, surface normals or object segmentation.



[More details](https://microsoft.github.io/AirSim/image_apis)



### Weather Effects



Press F10 to see various options available for weather effects. You can also control the weather using [APIs](https://microsoft.github.io/AirSim/apis#weather-apis). Press F1 to see other options available.



![record screenshot](docs/images/weather_menu.png)



## Tutorials



- [Video - Setting up AirSim with Pixhawk Tutorial](https://youtu.be/1oY8Qu5maQQ) by Chris Lovett

- [Video - Using AirSim with Pixhawk Tutorial](https://youtu.be/HNWdYrtw3f0) by Chris Lovett

- [Video - Using off-the-self environments with AirSim](https://www.youtube.com/watch?v=y09VbdQWvQY) by Jim Piavis

- [Webinar - Harnessing high-fidelity simulation for autonomous systems](https://note.microsoft.com/MSR-Webinar-AirSim-Registration-On-Demand.html) by Sai Vemprala

- [Reinforcement Learning with AirSim](https://microsoft.github.io/AirSim/reinforcement_learning) by Ashish Kapoor

- [The Autonomous Driving Cookbook](https://aka.ms/AutonomousDrivingCookbook) by Microsoft Deep Learning and Robotics Garage Chapter

- [Using TensorFlow for simple collision avoidance](https://github.com/simondlevy/AirSimTensorFlow) by Simon Levy and WLU team



## Participate



### Paper



More technical details are available in [AirSim paper (FSR 2017 Conference)](https://arxiv.org/abs/1705.05065). Please cite this as:

```

@inproceedings{airsim2017fsr,

  author = {Shital Shah and Debadeepta Dey and Chris Lovett and Ashish Kapoor},

  title = {AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles},

  year = {2017},

  booktitle = {Field and Service Robotics},

  eprint = {arXiv:1705.05065},

  url = {https://arxiv.org/abs/1705.05065}

}

```



### Contribute



Please take a look at [open issues](https://github.com/microsoft/airsim/issues) if you are looking for areas to contribute to.



* [More on AirSim design](https://microsoft.github.io/AirSim/design)

* [More on code structure](https://microsoft.github.io/AirSim/code_structure)

* [Contribution Guidelines](CONTRIBUTING.md)



### Who is Using AirSim?



We are maintaining a [list](https://microsoft.github.io/AirSim/who_is_using) of a few projects, people and groups that we are aware of. If you would like to be featured in this list please [make a request here](https://github.com/microsoft/airsim/issues).



## Contact



Join our [GitHub Discussions group](https://github.com/microsoft/AirSim/discussions) to stay up to date or ask any questions.



We also have an AirSim group on [Facebook](https://www.facebook.com/groups/1225832467530667/). 



## What's New



* [Cinematographic Camera](https://github.com/microsoft/AirSim/pull/3949)

* [ROS2 wrapper](https://github.com/microsoft/AirSim/pull/3976)

* [API to list all assets](https://github.com/microsoft/AirSim/pull/3940)

* [movetoGPS API](https://github.com/microsoft/AirSim/pull/3746)

* [Optical flow camera](https://github.com/microsoft/AirSim/pull/3938)

* [simSetKinematics API](https://github.com/microsoft/AirSim/pull/4066)

* [Dynamically set object textures from existing UE material or texture PNG](https://github.com/microsoft/AirSim/pull/3992)

* [Ability to spawn/destroy lights and control light parameters](https://github.com/microsoft/AirSim/pull/3991)

* [Support for multiple drones in Unity](https://github.com/microsoft/AirSim/pull/3128)

* [Control manual camera speed through the keyboard](https://github.com/microsoft/AirSim/pulls?page=6&q=is%3Apr+is%3Aclosed+sort%3Aupdated-desc#:~:text=1-,Control%20manual%20camera%20speed%20through%20the%20keyboard,-%233221%20by%20saihv) 



For complete list of changes, view our [Changelog](docs/CHANGELOG.md)



## FAQ



If you run into problems, check the [FAQ](https://microsoft.github.io/AirSim/faq) and feel free to post issues in the  [AirSim](https://github.com/Microsoft/AirSim/issues) repository.



## Code of Conduct



This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [[email protected]](mailto:[email protected]) with any additional questions or comments.



## License



This project is released under the MIT License. Please review the [License file](LICENSE) for more details.