{"id":19463734,"url":"https://github.com/adamko-dev/kafkatorio","last_synced_at":"2025-04-25T08:32:33.230Z","repository":{"id":70873126,"uuid":"439994069","full_name":"adamko-dev/kafkatorio","owner":"adamko-dev","description":null,"archived":false,"fork":false,"pushed_at":"2022-12-28T21:20:59.000Z","size":3167,"stargazers_count":9,"open_issues_count":5,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-04-16T15:47:12.833Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adamko-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-19T23:53:41.000Z","updated_at":"2024-03-30T07:37:22.000Z","dependencies_parsed_at":"2023-02-22T00:30:42.564Z","dependency_job_id":null,"html_url":"https://github.com/adamko-dev/kafkatorio","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamko-dev%2Fkafkatorio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamko-dev%2Fkafkatorio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamko-dev%2Fkafkatorio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamko-dev%2Fkafkatorio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adamko-dev","download_url":"https://codeload.github.com/adamko-dev/kafkatorio/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250782299,"owners_count":21486431,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T18:12:00.994Z","updated_at":"2025-04-25T08:32:32.810Z","avatar_url":"https://github.com/adamko-dev.png","language":"Kotlin","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Mod Portal](https://img.shields.io/badge/dynamic/json?label=Mod%20Portal\u0026query=%24.releases%5B-1%3A%5D.version\u0026url=https%3A%2F%2Fmods.factorio.com%2Fapi%2Fmods%2Fkafkatorio-events\u0026style=flat-square\u0026color=%23600401)](https://mods.factorio.com/mod/kafkatorio-events)\n[![Factorio Version](https://img.shields.io/badge/dynamic/json?label=Factorio%20Version\u0026query=%24.releases%5B-1%3A%5D.info_json.factorio_version\u0026url=https%3A%2F%2Fmods.factorio.com%2Fapi%2Fmods%2Fkafkatorio-events\u0026style=flat-square\u0026color=%23d7cdf6)](https://mods.factorio.com/mod/kafkatorio-events)\n\n# Kafkatorio\n\n[Kafkatorio](https://github.com/adamko-dev/kafkatorio) is a platform used for creating\n[Factorio](https://www.factorio.com/) mods that require communication with an external server.\n\nDevelopment is ongoing.\n\nAt present Kafkatorio is used to create a [live-view web map](https://factorio.adamko.dev), that\nshows the current status of a Factorio server, and any connected players.\n\nHowever, Kafkatorio can be used for more than this. It has the potential to export metrics, and\nallow for inter-server communication.\n\nKafkatorio was created to explore the possibilities of using\n[Apache Kafka](https://kafka.apache.org/) to process updates from a Factorio server.\n\n### Running\n\n[Instructions for running a Kafkatorio instance are available in the docs](./docs/guide.md).\n\n### Overview\n\nThis is a brief overview of how Kafkatorio gathers data, processes it, and uses it to create the\nlive web-map.\n\nKafkatorio receives updates from a Factorio server, processes them, and exposes them over a REST\nand WebSocket APIs.\n\n1. The Kafkatorio game mod, [`events-mod`](./modules/events-mod), collects event data, de-bouncing\n   and grouping events, and converts them to JSON packets\n2. `events-mod` prints the packets to the Factorio game server logs\n3. The Factorio logs are forwarded to a Kafka cluster, either using\n    * [syslog](https://docs.docker.com/config/containers/logging/syslog/) (using Docker logging\n      plugin)\n    * or [Kafka Pipe](./modules/infra-kafka-pipe) (a script that reads the server logs and forwards\n      them to a Kafka topic)\n4. The packets are consumed by [`events-server`](./modules/events-server), which processes the\n   packets using Kafka Streams, creating and hosting map tiles.\n\n   `events-server` serves the map tiles via a REST API, and produces WebSocket messages regarding\n   updates (for example, when a player's location has changed).\n5. The `web-map` uses Leaflet/JS to display the map, using the tiles and WebSocket messages from\n   `events-server`\n\n```mermaid\nsequenceDiagram\n\nFactorio multiplayer server-\u003e\u003eevents-mod: in-game events\nNote over events-mod: converts events to packets\nevents-mod-\u003e\u003elogs processor: packets logged to console\nlogs processor-\u003e\u003eKafka: publishes packets\u003cbr/\u003e(Requires API key!)\n\nKafka-\u003e\u003eevents-server: \n\nNote over events-server: Packets are processed\n\nevents-server-\u003e\u003eweb-map: Web map tiles (REST API)\nevents-server-)web-map: WebSocket updates\u003cbr/\u003e(e.g. player movement)\n```\n\n## License\n\nBecause I care about sharing improvements, Kafkatorio is licensed under the  \n[**GNU General Public License v3.0**](https://choosealicense.com/licenses/gpl-3.0/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadamko-dev%2Fkafkatorio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadamko-dev%2Fkafkatorio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadamko-dev%2Fkafkatorio/lists"}