{"id":13442976,"url":"https://github.com/erdos-project/pylot","last_synced_at":"2025-04-04T18:08:29.270Z","repository":{"id":37663095,"uuid":"190252594","full_name":"erdos-project/pylot","owner":"erdos-project","description":"Modular autonomous driving platform running on the CARLA simulator and real-world vehicles.","archived":false,"fork":false,"pushed_at":"2023-03-24T22:58:22.000Z","size":10603,"stargazers_count":490,"open_issues_count":48,"forks_count":134,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-03-28T17:08:48.500Z","etag":null,"topics":["autonomous-driving","autonomous-vehicles","carla","carla-simulator","control","lane-detection","machine-learning","obstacle-tracking","perception","planning","prediction","self-driving-car","traffic-light-detection"],"latest_commit_sha":null,"homepage":"https://pylot.readthedocs.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/erdos-project.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-06-04T17:54:40.000Z","updated_at":"2025-03-25T01:16:06.000Z","dependencies_parsed_at":"2024-06-19T02:54:54.684Z","dependency_job_id":"023d929f-9e30-4cc6-8c2c-a92c475a94de","html_url":"https://github.com/erdos-project/pylot","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erdos-project%2Fpylot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erdos-project%2Fpylot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erdos-project%2Fpylot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erdos-project%2Fpylot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/erdos-project","download_url":"https://codeload.github.com/erdos-project/pylot/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247226215,"owners_count":20904465,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autonomous-driving","autonomous-vehicles","carla","carla-simulator","control","lane-detection","machine-learning","obstacle-tracking","perception","planning","prediction","self-driving-car","traffic-light-detection"],"created_at":"2024-07-31T03:01:54.128Z","updated_at":"2025-04-04T18:08:29.251Z","avatar_url":"https://github.com/erdos-project.png","language":"Python","funding_links":[],"categories":["Python","Topics","Simulation","Sample Codes / Projects \u003ca name=\"sample\" /\u003e 🎉🎉🎉"],"sub_categories":["OS","Version Control","Other \u003ca name=\"Other_Code\" /\u003e🚦"],"readme":"[![Build Status](https://github.com/erdos-project/pylot/workflows/CI/badge.svg)](https://github.com/erdos-project/pylot/actions)\n[![Documentation Status](https://readthedocs.org/projects/pylot/badge/?version=latest)](https://pylot.readthedocs.io/en/latest/?badge=latest)\n\nPylot is an autonomous vehicle platform for developing and testing autonomous\nvehicle components (e.g., perception, prediction, planning) on the\nCARLA simulator and real-world cars.\n\n* [**Setup instructions**](#setup-instructions)\n* [**Documentation**](https://pylot.readthedocs.io/en/latest/)\n* [**Pylot components**](#pylot-components)\n* [**Data collection**](#data-collection)\n* [**Build Docker image**](#build-your-own-docker-image)\n* [**CARLA autonomous driving challenge**](#carla-autonomous-driving-challenge)\n* [**Getting involved**](#getting-involved)\n\n# Setup instructions\n\n## Deploy using Docker\n\nThe easiest way to get Pylot running is to use our Docker image. Please ensure\nyou have `nvidia-docker` on your machine before you start installing Pylot.\nIn case you do not have `nvidia-docker` please\nrun ```./scripts/install-nvidia-docker.sh```\n\nWe provide a Docker image with both Pylot and CARLA already setup.\n\n```console\ndocker pull erdosproject/pylot\nnvidia-docker run -itd --name pylot -p 20022:22 erdosproject/pylot /bin/bash\n```\n\nFollowing, start the simulator in the container:\n\n```console\nnvidia-docker exec -i -t pylot /home/erdos/workspace/pylot/scripts/run_simulator.sh\n```\n\nFinally, start Pylot in the container:\n\n```console\nnvidia-docker exec -i -t pylot /bin/bash\ncd ~/workspace/pylot/\npython3 pylot.py --flagfile=configs/detection.conf\n```\n\n## Visualizing components\nIn case you desire to visualize outputs of different components (e.g., bounding boxes),\nyou have to forward X from the container. First, add your public ssh key to the\n`~/.ssh/authorized_keys` in the container:\n\n```console\nnvidia-docker cp ~/.ssh/id_rsa.pub pylot:/home/erdos/.ssh/authorized_keys\nnvidia-docker exec -i -t pylot sudo chown erdos /home/erdos/.ssh/authorized_keys\nnvidia-docker exec -i -t pylot sudo service ssh start\n```\n\nFinally, ssh into the container with X forwarding:\n```console\nssh -p 20022 -X erdos@localhost\ncd /home/erdos/workspace/pylot/\npython3 pylot.py --flagfile=configs/detection.conf --visualize_detected_obstacles\n```\n\nIf everything worked ok, you should be able to see a visualization like\nthe one below:\n\n![Pylot obstacle detection](/doc/source/images/pylot-obstacle-detection.png)\n\n## Manual installation instructions\nAlternatively, you can install Pylot on your base system by executing the\nfollowing steps:\n\n```console\n./install.sh\npip install -e ./\n```\n\nNext, start the simulator:\n```console\nexport CARLA_HOME=$PYLOT_HOME/dependencies/CARLA_0.9.10.1/\n./scripts/run_simulator.sh\n```\n\nIn a different terminal, setup the paths:\n```console\nexport CARLA_HOME=$PYLOT_HOME/dependencies/CARLA_0.9.10.1/\ncd $PYLOT_HOME/scripts/\nsource ./set_pythonpath.sh\n```\n\nFinally, run Pylot:\n```console\ncd  $PYLOT_HOME/\npython3 pylot.py --flagfile=configs/detection.conf\n```\n\n# Pylot components\n\nPylot comprises of several components: obstacle detection, traffic light\ndetection, lane detection, obstacle tracking, localization, segmentation,\nfusion, prediction, planners, and control.\nEach component is implemented using one or more ERDOS operators and can be\nexecuted in isolation or with the entire Pylot application. Please read\nthe [**Documentation**](https://pylot.readthedocs.io/en/latest/) for\na more in depth description.\n\n![Pylot pipeline](/doc/source/images/pylot.png)\n\nRun the following command to see a demo of all the components, and the Pylot\ndriving policy:\n\n```console\npython3 pylot.py --flagfile=configs/demo.conf\n```\n\nThe demo will execute: obstacle detection, traffic light detection,\nsegmentation, prediction, planning, and the driving policy.\n\n***\nYou can also run components in isolation:\n\n### Obstacle detection\nPylot supports three object detection models: `frcnn_resnet101`,\n`ssd-mobilenet-fpn-640` and `ssdlite-mobilenet-v2`. The following command runs\na detector in isolation:\n\n```console\npython3 pylot.py --flagfile=configs/detection.conf\n```\n\nIn case you want to evaluate the detector (i.e., compute mAP), you can run:\n```console\npython3 pylot.py --flagfile=configs/detection.conf --evaluate_obstacle_detection\n```\n\nIn case you are not satisfied with the accuracy of our obstacle detector, you\ncan run a perfect version of it:\n\n```console\npython3 pylot.py --flagfile=configs/perfect_detection.conf\n```\n\nIf the detector does not run at your desired frequency, or if you want to track\nobstacles across frames, you can use a mix of detector plus tracker by running:\n\n```console\npython3 pylot.py --flagfile=configs/tracking.conf\n```\n\n### Traffic light detection\nPylot has uses a separate component for traffic light detection and\nclassification. The following command runs the component in isolation:\n\n```console\npython3 pylot.py --flagfile=configs/traffic_light.conf\n```\n\nIn case you require higher accuracy, you can run perfect traffic light detection\nby passing the ```--perfect_traffic_light_detection``` flag.\n\n### Lane detection\n\n```console\npython3 pylot.py --flagfile=configs/lane_detection.conf\n```\n\n### Obstacle tracking\n\n```console\npython3 pylot.py --flagfile=configs/tracking.conf\n```\n\n### Segmentation\nIn order to run Pylot's segmentation component in isolation execute the\nfollowing command:\n\n```console\npython3 pylot.py --flagfile=configs/segmentation.conf\n```\n\nSimilarly, pass ```--perfect_segmentation``` if you desire ideal pixel semantic\nsegmentation.\n\n### Prediction\nPylot offers a simple linear prediction component:\n\n```console\npython3 pylot.py --flagfile=configs/prediction.conf\n```\n\n### Planning\nThe planning component provides two planning options, which can be specified\nusing the ```--planning_type``` flag:\n\n1. `waypoint`: a simple planner that follows predefined waypoints. These\nwaypoints can either be either pre-specified or computed using the A-star\nplanner part of the CARLA simulator map. The planner ensures that the\nego-vehicle respects traffic lights, stops whenever there are obstacles in its\npath, but does not implement obstacle avoidance.\n2. `frenet_optimal_trajectory`: a Frenet Optimal Trajectory planner.\n3. `rrt_star`: a Rapidly-explory Random Tree planner.\n4. `hybrid_astar`: a Hybrid A* planner.\n\n```console\n# To run the Frenet Optimal Trajectory planner.\npython3 pylot.py --flagfile=configs/frenet_optimal_trajectory_planner.conf\n\n# To run the RRT* planner.\npython3 pylot.py --flagfile=configs/rrt_star_planner.conf\n\n# To run the Hybrid A* planner.\npython3 pylot.py --flagfile=configs/hybrid_astar_planner.conf\n```\n\n### Control\nPylot supports three controllers, which can be specified using the\n```control``` flag:\n1. `pid`: follows the waypoints computed by the planning component using a PID\ncontroller.\n2. `mpc`: uses model predictive control for speed and waypoint following.\n3. `simulator_auto_pilot`: uses the simulator auto pilot to drive on predefined routes.\nThis controller drives independent of the output of the other components.\n\nYou can run all the components, together with one of the two policies by\nexecuting:\n\n```console\n# Runs all components using the algorithms we implemented and the models we trained:\npython3 pylot.py --flagfile=configs/e2e.conf\n# Runs the MPC\npython3 pylot.py --flagfile=configs/mpc.conf\n# Runs the simulator auto pilot.\npython3 pylot.py --control=simulator_auto_pilot\n```\n\n### Debug logs\nIn case you want to debug the application, you can active additional logging\nby passing: `--log_file_name=pylot.log --v=1` to your command.\n\n# Data collection\n\nPylot also provides a script for collecting CARLA data such as: RGB images,\nsegmented images, obstacle 2D bounding boxes, depth frames, point clouds,\ntraffic lights, obstacle trajectories, and data in Chauffeur format.\n\nRun ```python3 data_gatherer.py --help``` to see what data you can collect.\nAlternatively, you can inspect this\n[configuration](https://github.com/erdos-project/pylot/blob/master/configs/data_gatherer.conf)\nfor an example of a data collection setup.\n\n# Build your own Docker image\n\nIn case you want to build your own images from the latest code, you can execute:\n\n```console\ncd docker\n./build_images.sh\n```\n\nThe script creates two Docker images: one that contains the CARLA simulator and\nanother one that contains ERDOS and Pylot.\n\n# CARLA autonomous driving challenge\n\nPylot can also be used as a baseline for executing on the CARLA\n[**Leaderboard**](https://leaderboard.carla.org/) routes. We provide an  agent\nthat offers reference implementations for perception (i.e., detection,\ntracking), localization (Extended Kalman filter), prediction, planning\n(e.g., waypoint follower, Frenet optimal trajectory, RRT*, Hybrid A*),\nand control.\n\nTo test this agent you can pull our image which has all the necessary software\nalready installed.\n\n```console\ndocker pull erdosproject/pylot-carla-challenge\nnvidia-docker run -itd --name pylot-challenge -p 20022:22 erdosproject/pylot-carla-challenge /bin/bash\n```\n\nAlternatively, you can manually install the dependencies on your machine by\nfollowing the instructions provided below:\n\n```console\nmkdir challenge\nexport CHALLENGE_ROOT=`pwd`\n# Clone the challenge leaderboard repository.\ngit clone -b stable --single-branch https://github.com/carla-simulator/leaderboard.git\nexport LEADERBOARD_ROOT=${CHALLENGE_ROOT}/leaderboard/\ncd ${LEADERBOARD_ROOT} ; pip3 install -r requirements.txt ; cd ${CHALLENGE_ROOT}\n# Clone the CARLA scenario runner repository. This is used by the leaderboard.\ngit clone -b leaderboard --single-branch https://github.com/carla-simulator/scenario_runner.git\nexport SCENARIO_RUNNER_ROOT=${CHALLENGE_ROOT}/scenario_runner/\ncd ${SCENARIO_RUNNER_ROOT} ; pip3 install -r requirements.txt ; cd ${CHALLENGE_ROOT}\n# Checkout the CARLA challenge branch.\ncd ${PYLOT_HOME} ; git checkout -b challenge origin/challenge\nexport CARLA_ROOT=Path to CARLA 0.9.10.1.\ncd ${CHALLENGE_ROOT}\nexport TEAM_CODE_ROOT=${PYLOT_HOME} ; ${LEADERBOARD_ROOT}/scripts/make_docker.sh\n```\n\n## Notes on the Pylot CARLA challenge agent\nSimilar to regular Pylot, the [Challenge agent](https://github.com/erdos-project/pylot/blob/master/pylot/simulation/challenge/ERDOSAgent.py)\nnot only connects different reference implementation, but also provides the\noption of testing them in different configurations (e.g., test prediction,\nplanning and control with perfect perception). This can be done by changing the\nflags in the [challenge configuration](https://github.com/erdos-project/pylot/blob/master/pylot/simulation/challenge/challenge.conf)\naccording to the specification from the Pylot documentation.\n\n# More Information\n\nTo read more about the ideas behind Pylot, refer to our paper,\n*Pylot: A Modular Platform for Exploring Latency-Accuracy Tradeoffs in Autonomous Vehicles* ([IEEE](https://ieeexplore.ieee.org/document/9561747/)) ([arXiv](https://arxiv.org/abs/2104.07830)).\nIf you find Pylot useful to your work, please cite our paper as follows:\n```bibtex\n@inproceedings{gog2021pylot,\n  title={Pylot: A modular platform for exploring latency-accuracy tradeoffs in autonomous vehicles},\n  author={Gog, Ionel and Kalra, Sukrit and Schafhalter, Peter and Wright, Matthew A and Gonzalez, Joseph E and Stoica, Ion},\n  booktitle={2021 IEEE International Conference on Robotics and Automation (ICRA)},\n  pages={8806--8813},\n  year={2021},\n  organization={IEEE}\n}\n```\n\n# Getting Involved\n* [Community on Slack](https://forms.gle/KXwSrjM6ZqRi2MT18): Join our community\non Slack for discussions about development, questions about usage, and feature\nrequests.\n* [Github Issues](https://github.com/erdos-project/pylot/issues): For reporting\nbugs.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ferdos-project%2Fpylot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ferdos-project%2Fpylot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ferdos-project%2Fpylot/lists"}