{"id":13475125,"url":"https://github.com/ArroyoSystems/arroyo","last_synced_at":"2025-03-26T23:30:31.717Z","repository":{"id":151021644,"uuid":"621929252","full_name":"ArroyoSystems/arroyo","owner":"ArroyoSystems","description":"Distributed stream processing engine in Rust","archived":false,"fork":false,"pushed_at":"2024-10-29T22:39:29.000Z","size":14223,"stargazers_count":3732,"open_issues_count":64,"forks_count":215,"subscribers_count":42,"default_branch":"master","last_synced_at":"2024-10-30T00:46:07.603Z","etag":null,"topics":["data","data-stream-processing","dev-tools","infrastructure","kafka","rust","sql","stream-processing","stream-processing-engine"],"latest_commit_sha":null,"homepage":"https://arroyo.dev","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ArroyoSystems.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-31T17:41:56.000Z","updated_at":"2024-10-29T06:45:02.000Z","dependencies_parsed_at":"2024-11-13T00:30:57.284Z","dependency_job_id":null,"html_url":"https://github.com/ArroyoSystems/arroyo","commit_stats":null,"previous_names":[],"tags_count":31,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArroyoSystems%2Farroyo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArroyoSystems%2Farroyo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArroyoSystems%2Farroyo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArroyoSystems%2Farroyo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ArroyoSystems","download_url":"https://codeload.github.com/ArroyoSystems/arroyo/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245753759,"owners_count":20666801,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data","data-stream-processing","dev-tools","infrastructure","kafka","rust","sql","stream-processing","stream-processing-engine"],"created_at":"2024-07-31T16:01:17.580Z","updated_at":"2025-03-26T23:30:30.498Z","avatar_url":"https://github.com/ArroyoSystems.png","language":"Rust","funding_links":[],"categories":["Rust","Libraries","Table of Contents","sql","data","Projects","\u003ca name=\"Rust\"\u003e\u003c/a\u003eRust","Stream Processing"],"sub_categories":["Data streaming","Streaming Engine","Stream Processing"],"readme":"\n\u003ch1 align=\"center\"\u003e\n    \u003cimg src=\"https://raw.githubusercontent.com/ArroyoSystems/arroyo/760aabdbdb019d95f0c5ebb60933233aa735f830/images/arroyo_logo.png\" width=\"400px\" alt=\"Arroyo\" /\u003e\n\u003c/h1\u003e\n\n\n\u003ch4 align=\"center\"\u003e\n  \u003ca href=\"https://arroyo.dev/\"\u003eArroyo Cloud\u003c/a\u003e |\n  \u003ca href=\"https://doc.arroyo.dev/getting-started\"\u003eGetting started\u003c/a\u003e |\n  \u003ca href=\"https://doc.arroyo.dev\"\u003eDocs\u003c/a\u003e |\n  \u003ca href=\"https://discord.gg/cjCr5rVmyR\"\u003eDiscord\u003c/a\u003e |\n  \u003ca href=\"https://arroyo.dev\"\u003eWebsite\u003c/a\u003e\n\u003c/h4\u003e\n\n\u003ch4 align=\"center\"\u003e\n  \u003ca href=\"https://github.com/ArroyoSystems/arroyo/blob/master/LICENSE-APACHE\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/license-MIT%2FApache--2.0-orange\" alt=\"Arroyo is dual-licensed under Apache 2 and MIT licenses.\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/ArroyoSystems/arroyo/blob/master/CONTRIBUTING.md\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/PRs-Welcome-brightgreen\" alt=\"PRs welcome!\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/ArroyoSystems/arroyo/commits\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/commit-activity/m/ArroyoSystems/arroyo\" alt=\"git commit activity\" /\u003e\n  \u003c/a\u003e\n  \u003cimg alt=\"CI\" src=\"https://github.com/ArroyoSystems/arroyo/actions/workflows/ci.yml/badge.svg\"\u003e\n\n  \u003ca href=\"https://github.com/ArroyoSystems/arroyo/releases\"\u003e\n    \u003cimg alt=\"GitHub release (latest by date)\" src=\"https://img.shields.io/github/v/release/ArroyoSystems/arroyo?display_name=release\"\u003e\n  \u003c/a\u003e\n\u003c/h4\u003e\n\n\n[Arroyo](https://arroyo.dev) is a distributed stream processing engine written in Rust, designed to efficiently\nperform stateful computations on streams of data. Unlike traditional batch processing, streaming engines can operate\non both bounded and unbounded sources, emitting results as soon as they are available.\n\nIn short: Arroyo lets you ask complex questions of high-volume real-time data with subsecond results.\n\n![running job](https://raw.githubusercontent.com/ArroyoSystems/arroyo/760aabdbdb019d95f0c5ebb60933233aa735f830/images/header_image.png)\n\n## Features\n\n🦀 SQL and Rust pipelines\n\n🚀 Scales up to millions of events per second\n\n🪟 Stateful operations like windows and joins\n\n🔥State checkpointing for fault-tolerance and recovery of pipelines\n\n🕒 Timely stream processing via the [Dataflow model](https://www.oreilly.com/radar/the-world-beyond-batch-streaming-101/)\n\n## Use cases\n\nSome example use cases include:\n\n* Detecting fraud and security incidents\n* Real-time product and business analytics\n* Real-time ingestion into your data warehouse or data lake\n* Real-time ML feature generation\n\n## Why Arroyo\n\nThere are already a number of existing streaming engines out there, including [Apache Flink](https://flink.apache.org),\n[Spark Streaming](https://spark.apache.org/docs/latest/streaming-programming-guide.html), and\n[Kafka Streams](https://kafka.apache.org/documentation/streams/). Why create a new one?\n\n* _Serverless operations_: Arroyo pipelines are designed to run in modern cloud environments, supporting seamless scaling,\n    recovery, and rescheduling\n* _High performance SQL_: SQL is a first-class concern, with consistently excellent performance\n* _Designed for non-experts_: Arroyo cleanly separates the pipeline APIs from its internal implementation. You don't\n    need to be a streaming expert to build real-time data pipelines.\n\n## Installing\n\nArroyo ships as a single binary. You can install it locally on MacOS using Homebrew\n\n```shellsession\nbrew install arroyosystems/tap/arroyo\n```\n\nor on MacOS or Linux with this script:\n\n```shellsession\ncurl -LsSf https://arroyo.dev/install.sh | sh\n```\n\nor you can download a binary for your platform from the [releases page](https://github.com/ArroyoSystems/arroyo/releases).\n\nOnce you have Arroyo installed, start a cluster with\n\n```shellsession\n$ arroyo cluster\n```\n\nYou can also run a cluster in Docker, with\n\n```shellsession\ndocker run -p 5115:5115 \\\n      ghcr.io/arroyosystems/arroyo:latest\n```\n\nThen, load the Web UI at http://localhost:5115.\n\nFor a more in-depth guide, see the [getting started guide](https://doc.arroyo.dev/getting-started).\n\nOnce you have Arroyo running, follow the [tutorial](https://doc.arroyo.dev/tutorial) to create your first real-time\npipeline.\n\n## Developing Arroyo\n\nWe love contributions from the community! See the [developer setup](https://doc.arroyo.dev/developing/dev-setup) guide\nto get started, and reach out to the team on [discord](https://discord.gg/cjCr5rVmyR) or create an issue.\n\n## Community\n\n* [Discord](https://discord.gg/cjCr5rVmyR) \u0026mdash; support and project discussion\n* [GitHub issues](https://github.com/ArroyoSystems/arroyo/issues) \u0026mdash; bugs and feature requests\n* [Arroyo Blog](https://arroyo.dev/blog) \u0026mdash; updates from the Arroyo team\n\n## Arroyo Enterprise\n\nRunning in production? Arroyo Systems provides [enterprise features and support](https://www.arroyo.dev/enterprise) for\nArroyo users. Get in touch at [support@arroyo.systems](mailto:support@arroyo.systems).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FArroyoSystems%2Farroyo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FArroyoSystems%2Farroyo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FArroyoSystems%2Farroyo/lists"}