{"id":14987979,"url":"https://github.com/apache/flink-statefun","last_synced_at":"2025-04-13T22:19:18.236Z","repository":{"id":37664605,"uuid":"225625782","full_name":"apache/flink-statefun","owner":"apache","description":"Apache Flink Stateful Functions","archived":false,"fork":false,"pushed_at":"2024-05-08T12:59:03.000Z","size":5662,"stargazers_count":517,"open_issues_count":22,"forks_count":182,"subscribers_count":49,"default_branch":"master","last_synced_at":"2025-04-06T20:01:36.029Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://flink.apache.org/","language":"Java","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apache.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-12-03T13:24:13.000Z","updated_at":"2025-04-02T22:16:50.000Z","dependencies_parsed_at":"2024-01-25T23:04:21.932Z","dependency_job_id":"ab3f21dc-958c-4d97-91d5-25567722949b","html_url":"https://github.com/apache/flink-statefun","commit_stats":{"total_commits":823,"total_committers":49,"mean_commits":16.79591836734694,"dds":0.5650060753341434,"last_synced_commit":"accd75ea0109845c4b4c0ddd74021147af1439d4"},"previous_names":[],"tags_count":34,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fflink-statefun","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fflink-statefun/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fflink-statefun/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fflink-statefun/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apache","download_url":"https://codeload.github.com/apache/flink-statefun/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247721888,"owners_count":20985085,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-24T14:15:53.981Z","updated_at":"2025-04-13T22:19:18.193Z","avatar_url":"https://github.com/apache.png","language":"Java","readme":"\u003cimg alt=\"Stateful Functions\" src=\"docs/static/navbar-brand-logo.jpg\" width=\"350px\"/\u003e\n\nStateful Functions is an API that simplifies the building of **distributed stateful applications** with a **runtime built for serverless architectures**.\nIt brings together the benefits of stateful stream processing - the processing of large datasets with low latency and bounded resource constraints -\nalong with a runtime for modeling stateful entities that supports location transparency, concurrency, scaling, and resiliency. \n\n\u003cimg alt=\"Stateful Functions Architecture\" width=\"80%\" src=\"docs/static/fig/concepts/arch_overview.svg\"\u003e\n\nIt is designed to work with modern architectures, like cloud-native deployments and popular event-driven FaaS platforms \nlike AWS Lambda and KNative, and to provide out-of-the-box consistent state and messaging while preserving the serverless\nexperience and elasticity of these platforms.\n\nStateful Functions is developed under the umbrella of [Apache Flink](https://flink.apache.org/).\n\nThis README is meant as a brief walkthrough on the core concepts and how to set things up\nto get yourself started with Stateful Functions.\n\nFor a fully detailed documentation, please visit the [official docs](https://ci.apache.org/projects/flink/flink-statefun-docs-master).\n\n![Java 8 Build](https://github.com/apache/flink-statefun/workflows/Java%208%20Build/badge.svg)\n\n## Table of Contents\n\n- [Core Concepts](#core-concepts)\n   * [Abstraction](#abstraction)\n   * [Function modules and extensibility](#modules)\n   * [Runtime](#runtime)\n- [Getting Started](#getting-started)\n   * [New Java Users](#java)\n   * [New Python Users](#python)\n- [Building the Project](#build)\n   * [Prerequisites](#build-prerequisites)\n   * [Procedures](#build-procedures)\n- [Contributing](#contributing)\n- [License](#license)\n\n## \u003ca name=\"core-concepts\"\u003e\u003c/a\u003eCore Concepts\n\n### \u003ca name=\"abstraction\"\u003e\u003c/a\u003eAbstraction\n\nA Stateful Functions application consists of the following primitives: stateful functions, ingresses,\nrouters and egresses.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/static/fig/concepts/statefun-app.svg\" width=\"650px\"/\u003e\n\u003c/p\u003e\n\n#### Stateful functions\n\n* A _stateful function_ is a small piece of logic/code that is invoked through a message. Each stateful function \nexists as a uniquely invokable _virtual instance_ of a _function type_. Each instance is addressed by its ``type``, as well as an unique ``ID`` (a string) within its type.\n\n* Stateful functions may be invoked from ingresses or any other stateful function (including itself).\nThe caller simply needs to know the address of the target function.\n\n* Function instances are _virtual_, because they are not all active in memory at the same time.\nAt any point in time, only a small set of functions and their state exists as actual objects. When a\nvirtual instance receives a message, one of the objects is configured and loaded with the state of that virtual\ninstance and then processes the message. Similar to virtual memory, the state of many functions might be “swapped out”\nat any point in time.\n\n* Each virtual instance of a function has its own state, which can be accessed in local variables.\nThat state is private and local to that instance.\n\nIf you know Apache Flink’s DataStream API, you can think of stateful functions a bit like a lightweight\n`KeyedProcessFunction`. The function ``type`` is the process function transformation, while the ``ID`` is the key. The difference\nis that functions are not assembled in a Directed Acyclic Graph (DAG) that defines the flow of data (the streaming topology),\nbut rather send events arbitrarily to all other functions using addresses.\n\n#### Ingresses and Egresses\n\n* _Ingresses_ are the way that events initially arrive in a Stateful Functions application.\nIngresses can be message queues, logs or HTTP servers — anything that produces an event to be\nhandled by the application.\n\n* _Routers_ are attached to ingresses to determine which function instance should handle an event initially.\n\n* _Egresses_ are a way to send events out from the application in a standardized way.\nEgresses are optional; it is also possible that no events leave the application and functions sink events or\ndirectly make calls to other applications.\n\n### \u003ca name=\"modules\"\u003e\u003c/a\u003eModules\n\nA _module_ is the entry point for adding the core building block primitives to a Stateful Functions\napplication, i.e. ingresses, egresses, routers and stateful functions.\n\nA single application may be a combination of multiple modules, each contributing a part of the whole application.\nThis allows different parts of the application to be contributed by different modules; for example,\none module may provide ingresses and egresses, while other modules may individually contribute specific parts of the\nbusiness logic as stateful functions. This facilitates working in independent teams, but still deploying\ninto the same larger application.\n\n## \u003ca name=\"runtime\"\u003eRuntime\n\nThe Stateful Functions runtime is designed to provide a set of properties similar to what characterizes [serverless functions](https://martinfowler.com/articles/serverless.html), but applied to stateful problems.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/static/fig/concepts/arch_overview.svg\" width=\"600px\"/\u003e\n\u003c/p\u003e\n\nThe runtime is built on Apache Flink\u003csup\u003e®\u003c/sup\u003e, with the following design principles:\n\n* **Logical Compute/State Co-location:** Messaging, state access/updates and function invocations are managed tightly together. This ensures a high-level of consistency out-of-the-box.\n\n* **Physical Compute/State Separation:** Functions can be executed remotely, with message and state access provided as part of the invocation request. This way, functions can be managed like stateless processes and support rapid scaling, rolling upgrades and other common operational patterns.\n\n* **Language Independence:** Function invocations use a simple HTTP/gRPC-based protocol so that Functions can be easily implemented in various languages.\n\nThis makes it possible to execute functions on a Kubernetes deployment, a FaaS platform or behind a (micro)service, while providing consistent state and lightweight messaging between functions.\n\n## \u003ca name=\"getting-started\"\u003e\u003c/a\u003eGetting Started\n\nWe highly recommend starting from the the tutorials and examples that can be found in the [Stateful Functions Playground repository](https://github.com/apache/flink-statefun-playground). There you will find material that serve as a great starting point if you've just started with getting to know the project, or looking for specific examples of common usage patterns.\n\n### \u003ca name=\"java\"\u003eNew Java Users\n\nHead over to the [Java SDK Showcase](https://github.com/apache/flink-statefun-playground/tree/release-3.2/java/showcase), or the [Java Greeter Example](https://github.com/apache/flink-statefun-playground/tree/release-3.2/java/greeter) in the playground repository.\n\nAlternatively, execute the following commands to clone the playground repository locally:\n```\n$ git clone -b release-3.2 https://github.com/apache/flink-statefun-playground.git\n$ cd java/showcase # or, java/greeter\n```\n\n### \u003ca name=\"python\"\u003eNew Python Users\n\nFor new Python users, head over to the [Python SDK Showcase](https://github.com/apache/flink-statefun-playground/tree/release-3.2/python/showcase), or the [Python Greeter Example](https://github.com/apache/flink-statefun-playground/tree/release-3.2/python/greeter) in the playground repository.\n\nAlternatively, execute the following commands to clone the playground repository locally:\n```\n$ git clone -b release-3.2 https://github.com/apache/flink-statefun-playground.git\n$ cd python/showcase # or, python/greeter\n```\n\nEach tutorial or example in the playground repository will have a `README` that contains details on building and running the code. Take a look and try it out yourself!\n\n## \u003ca name=\"build\"\u003e\u003c/a\u003eBuilding the Project\n\nThis section contains information for building this project.\n\n### \u003ca name=\"build-prerequisites\"\u003ePrerequisites\n\n- Maven v3.5+\n- Java 8\n- Docker v20.10+ (if you'd like to run the end-to-end tests as part of the build)\n\n### \u003ca name=\"build-procedures\"\u003eProcedures\n\n1. Build source code:\n  ```\n  $ mvn clean install\n  $ # or, alternatively, run the end-to-end tests as well (requires Docker)\n  $ mvn clean install -Prun-e2e-tests\n  ```\n\n2. Build Stateful Functions Docker image: This step requires that you've already compiled artifacts from the source code.\n  ```\n  $ ./tools/docker/build-stateful-functions.sh\n  ```\n  This builds a local Docker image tagged as `flink-statefun:\u003cversion_of_current_source_version\u003e`.\n\n## \u003ca name=\"code-of-conduct\"\u003e\u003c/a\u003eCode of Conduct\n\nApache Flink, Stateful Functions, and all its associated repositories follow the [Code of Conduct of the Apache Software Foundation](https://www.apache.org/foundation/policies/conduct).\n\n## \u003ca name=\"license\"\u003e\u003c/a\u003eLicense\n\nThe code in this repository is licensed under the [Apache Software License 2.0](LICENSE).\n","funding_links":[],"categories":["Java","Serverless"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Fflink-statefun","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapache%2Fflink-statefun","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Fflink-statefun/lists"}