Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/fristonio/xene
A distributed workflow runner focusing on performance and simplicity.
https://github.com/fristonio/xene
airflow cloud dag distributed-systems docker golang kubernetes workflow workflow-engine
Last synced: 7 days ago
JSON representation
A distributed workflow runner focusing on performance and simplicity.
- Host: GitHub
- URL: https://github.com/fristonio/xene
- Owner: fristonio
- License: mit
- Created: 2019-12-17T17:38:08.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2023-05-05T02:24:44.000Z (over 1 year ago)
- Last Synced: 2024-06-19T06:00:36.592Z (5 months ago)
- Topics: airflow, cloud, dag, distributed-systems, docker, golang, kubernetes, workflow, workflow-engine
- Language: Go
- Homepage: https://xene-api-docs.netlify.app
- Size: 1.11 MB
- Stars: 65
- Watchers: 4
- Forks: 2
- Open Issues: 15
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
- Roadmap: ROADMAP.md
Awesome Lists containing this project
README
# Xene
> A distributed workflow runner with focus on performance and simplicity.
[](https://app.netlify.com/sites/sad-thompson-bcaa9a/deploys)  [](https://opensource.org/licenses/MIT) [](https://goreportcard.com/report/github.com/fristonio/xene) [](https://godoc.org/github.com/fristonio/xene)
Xene is a high performance, distributed workflow execution platform. It lets you create custom workflows in the form of a Directed Acyclic Graph, which can then be executed based on a trigger configured with the workflow.
Xene is built with Golang with a focus on performance, resiliency, and simplicity. Xene internally runs in a way similar to Kubernetes by getting triggered on a level and then reconciling the desired state with the current state. Currently, xene supports [Badger](https://github.com/dgraph-io/badger) as the storage engines, which is pluggable and can be replaced with a similar key-value store.
## Features
Xene aims to tackle two main problems in any workflow execution system:
1. Low resource footprint
2. Flexibility and Simplicity
Xene exploits a lot of small enhancement to reduce the network and memory footprint of the executors and APIServer.
APIServer acts as the brains of the whole ecosystem by directing the decisions of scheduling workloads and managing coordination
between all the moving components. All the communication between APIServer and Agent is encrypted using mTLS and xene doesn't require
Agents to communicate with each other reducing network footprint.
Xene derives a lot of its design decisions from Kubernetes, like:
- Like Kubernetes, xene is also level triggered and does the reconciliation of state described by the user with the actual state.
- It also uses a KeyValue store to store all the states and objects. To reduce the memory footprint, xene supports the use of [Badger](https://github.com/dgraph-io/badger) as a key-value store.
- Objects are reconciled using controllers configured to watch changes on keys in the KVStore.
Some of the main features of Xene are:
- Locally running workflow from provided specs.
- Command line tool for xene(xenectl) to interact with apiserver.
- mTLS based authentication between agent and apiserver.
- JWT based authentication for API server.
- Google OAuth based authentication integration.
- [Beta UI](https://github.com/fristonio/xene-ui) based on React and Typescript
- Workflow creation and running on top of Docker based container runtime executor on the agent.
- Secrets management for agents.
- Workflow pipelines run live log streaming and status feed.
- Google based OAuth for APIServer with domain filtered authorization.
## Contributing
If you'd like to contribute to this project, refer to the [contributing guide](Contributing.md).
You can start with setting up Xene on your system and try solving a few bugs listed here: https://github.com/fristonio/xene/issues
## Documentation
- [API Docs](https://xene-api-docs.netlify.app/apidocs.html)
- [Roadmap](/ROADMAP.md)
- [Xene Documentation](https://xene-api-docs.netlify.app/)
- [Go Doc](https://pkg.go.dev/github.com/fristonio/xene)
- [Getting Started](/docs/GettingStarted.md)
## License
Xene is licensed under [MIT License](https://github.com/fristonio/xene/blob/master/LICENSE.md).
## Contact
If you have any queries regarding the project or just want to say hello, feel free to drop a mail at [email protected].