{"id":21707773,"url":"https://github.com/flightaware/firestarter","last_synced_at":"2026-03-08T05:31:54.822Z","repository":{"id":42069623,"uuid":"256344519","full_name":"flightaware/firestarter","owner":"flightaware","description":"Getting started with FlightAware Firehose","archived":false,"fork":false,"pushed_at":"2024-10-25T21:13:22.000Z","size":6034,"stargazers_count":31,"open_issues_count":7,"forks_count":12,"subscribers_count":20,"default_branch":"master","last_synced_at":"2025-04-12T16:16:49.698Z","etag":null,"topics":["docker","docker-compose","flightaware-firehose-service","python3"],"latest_commit_sha":null,"homepage":"https://flightaware.com/commercial/firehose/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/flightaware.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-04-16T22:31:11.000Z","updated_at":"2025-03-02T06:58:38.000Z","dependencies_parsed_at":"2023-01-25T13:16:29.888Z","dependency_job_id":"d161ec52-210b-46e2-b95a-de4ca042146c","html_url":"https://github.com/flightaware/firestarter","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/flightaware/firestarter","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flightaware%2Ffirestarter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flightaware%2Ffirestarter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flightaware%2Ffirestarter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flightaware%2Ffirestarter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/flightaware","download_url":"https://codeload.github.com/flightaware/firestarter/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flightaware%2Ffirestarter/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30246724,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-08T00:58:18.660Z","status":"online","status_checked_at":"2026-03-08T02:00:06.215Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","docker-compose","flightaware-firehose-service","python3"],"created_at":"2024-11-25T22:19:14.117Z","updated_at":"2026-03-08T05:31:54.790Z","avatar_url":"https://github.com/flightaware.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Docker Image CI](https://github.com/flightaware/firestarter/workflows/Docker%20Image%20CI/badge.svg)](https://github.com/flightaware/firestarter/actions?query=workflow%3A%22Docker+Image+CI%22)\n# Firestarter - Getting started with FlightAware Firehose\nFirestarter is a small collection of services and sample applications to help\nyou get started using [FlightAware's Firehose Flight Data Feed](https://flightaware.com/commercial/firehose/documentation).\n\nFirestarter is structured as a group of Docker containers managed by\nDocker Compose. Currently, 3 core services and 2 sample applications are\nincluded, with more being developed (see [the roadmap](./ROADMAP.md) for\ndetails).\n\n## Quickstart\nWe suggest that you have at least 8GB of RAM and 30GB of available disk space \nto run the full Firestarter stack. In order to have more fine-grained control\nover memory usage, check the notes on how to configure the [timescaledb-tune](https://github.com/timescale/timescaledb-docker#notes-on-timescaledb-tune)\nparameters for TimescaleDB.\n\nYou must set the following variables (in your environment or a .env file)\nbefore you can start using Firestarter.\n* FH_USERNAME - Your FlightAware Firehose account username\n* FH_APIKEY - The key to your FlightAware Firehose account\n* INIT_CMD_ARGS - Firehose initiation command; more information about this is\navailable at https://flightaware.com/commercial/firehose/documentation/commands\nand in the env section of [docker-compose.yml](./docker-compose.yml). Its value\nwill vary based on your account configuration, but a very basic example that\nshould work for most users is `events \"flifo departure arrival cancellation\nposition\"`.\n\nThere are a number of other environment variables that can be set to tune the\nbehavior of Firestarter. They are documented in\n[docker-compose.yml](./docker-compose.yml).\n\nYou'll also need to install Docker (18.06+) and Docker Compose (1.22.0+)\\\nDetails available at Docker's site: https://docs.docker.com/get-docker/\n\nThe usual Docker Compose incantation run in the root of this repo will get you\nup and running:\n```\ndocker-compose pull \u0026\u0026 docker-compose up\n```\n\n`docker-compose pull` pulls prebuilt images from the Github Container Registry,\nand `docker-compose up` creates containers from those images and launches them.\nIf you'd like to build the images yourself, you can instead run\n`docker-compose up --build`.\n\nAfter running the above command, you should be greeted with log output from\neach container. The services will log periodically as Firehose messages are\nreceived, while the sample webapps will produce some initial log output and\nthen only log as requests are made to them.\n\nYou can test out the FIDS sample application by visiting http://localhost:8080\nin your web browser (if not running Docker locally, use the Docker host's\naddress). The map sample application can be accessed at http://localhost:5001.\n\nIf you are running on Mac OS Monterrey or later, disable \"AirPlay Receiver\"\nin System Preferences \u003e Sharing \u003e AirPlay Receiver. It runs on port 5000,\nwhich is the same port as the FIDS backend application.\n\n\n## Firestarter Components\n\n![](docs/architecture-diagram.png)\n\n### connector\nThe connector service handles connecting to Firehose over an SSL socket. This\ninvolves building and sending the initiation command, handling compression, and\nreconnecting to Firehose without data loss if the connection is interrupted.\nThe connector then forwards Firehose messages to its own clients.\n\n### kafka/zookeeper\nWe are using kafka as a message queue between the connector and the db-updater.\nKafka depends on zookeeper to coordinate some important tasks, so we included\nthat as well. We chose to pull existing docker containers for these pieces of\nsoftware.\nTheir documentation can be found here:\nhttps://hub.docker.com/r/bitnami/kafka/\nhttps://hub.docker.com/r/bitnami/zookeeper/\n\nIn this code, the connector is the kafka \"producer\" and the db-updater is the\nkafka \"consumer\". If db-updater stops running and restarts, kafka will ensure\nthat it starts reading from the queue where it left off. We recommend that\nyou let kafka take care of this offset reconnect logic.\n\nWe ensure that the kafka consumer will start where it left off with the\n\"enable_auto_commit\" and \"auto_commit_interval_ms\" parameters. We also need to\nbe sure to provide a group name to store the last offset. Consumers with\ndifferent group names will each consume all messages in a given topic, and\nconsumers with the same group name will split messages from that topic between\nthem. A single Kafka topic is used in Firestarter to stream all messages\npublished by the connector to all subscribers.\n\n### db-updater\nThe db-updater service receives Firehose messages from the queue and\nmaintains a database table based on their contents. The service is capable of\nhandling so-called \"flifo\" (flight info) messages and airborne position messages.\nTwo db-updater containers are configured by default - one handles flight info and\nupdates a \"flights\" table, and the other handles airborne positions and updates\na \"positions\" table. The flight info db-updater uses sqlite by default (but has been\ntested with PostgreSQL), and the position db-updater uses TimescaleDB which is\nbased on PostgreSQL. Other databases could potentially be supported with little\neffort. To prevent bloat, flights and positions older than 48 hours are\nautomatically dropped from the table.\n\n### fids\nThe fids sample application is a webapp backed by the flights and positions\ndatabases. You can use it to browse flight data by airport, presenting flights\nsimilarly to how you'd see them on a flight information display system (FIDS).\nDetailed information for individual flights can also be viewed.\n\nIf you've specified a Google Maps API key, a flight's actual route will be\ndisplayed as a static image on its information page.\nInstructions:\nhttps://developers.google.com/maps/documentation/maps-static/get-api-key\nOnce you get your API key, just specify it in your .env file as\nGOOGLE_MAPS_API_KEY. Then you will see static maps with a flight track on your\nflight info pages. Note that the Google Maps API is a paid service with a free\ntier, so you will need to provide payment information when signing up.\nPricing information:\nhttps://developers.google.com/maps/documentation/maps-static/usage-and-billing\nYou can see that you currently get \"a $200 USD Google Maps Platform credit\"\neach month, and each query 0-100,000 is 0.002 USD each. So that means that you\nwill get 100,000 free queries per month. Since this is a demo and not meant for\nproduction, that should be fine.\n\nWhile Firestarter's services are suited for use in a production environment,\nthis sample application should only be considered a demonstration of what can\nbe built using the data from Firehose. It should *not* be used in a production\nenvironment.\n\n### map\nThe map sample application demonstrates another potential application of\nFirehose data by plotting live airborne flight positions directly onto a\ndynamic Google Map. Rather than using Firestarter's databases, this web app's\nbackend connects directly to the queue service. This is a place where an API\nlike Firehose truly shines, as no polling is required by the client to update\nthe map. A Google Maps API key is required to run this application.\n\nCheck out [the roadmap](./ROADMAP.md) to see what components are coming in the\nfuture!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflightaware%2Ffirestarter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fflightaware%2Ffirestarter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflightaware%2Ffirestarter/lists"}