{"id":19577129,"url":"https://github.com/tbobm/machinery","last_synced_at":"2025-04-27T06:32:02.063Z","repository":{"id":43341128,"uuid":"429833000","full_name":"tbobm/machinery","owner":"tbobm","description":"Workflow processing inspired by AWS State Machines. Define workflows, register services and start processing events.","archived":false,"fork":false,"pushed_at":"2022-03-07T08:54:18.000Z","size":117,"stargazers_count":4,"open_issues_count":8,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-08-10T09:18:23.314Z","etag":null,"topics":["api","event-driven","flask","python","state-machine"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tbobm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/contributing.rst","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null}},"created_at":"2021-11-19T14:48:40.000Z","updated_at":"2022-01-07T11:58:35.000Z","dependencies_parsed_at":"2022-09-02T18:21:59.202Z","dependency_job_id":null,"html_url":"https://github.com/tbobm/machinery","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tbobm%2Fmachinery","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tbobm%2Fmachinery/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tbobm%2Fmachinery/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tbobm%2Fmachinery/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tbobm","download_url":"https://codeload.github.com/tbobm/machinery/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224062795,"owners_count":17249291,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","event-driven","flask","python","state-machine"],"created_at":"2024-11-11T07:04:39.592Z","updated_at":"2024-11-11T07:04:40.418Z","avatar_url":"https://github.com/tbobm.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# machinery\n\n[![CI](https://github.com/tbobm/machinery/actions/workflows/ci.yml/badge.svg)](https://github.com/tbobm/machinery/actions/workflows/ci.yml)\n\nState-aware workflow orchestrator.\n\nThis project aims to bring stateful workflows, by defining a list of actions\nrepresenting multiple microservices.\n\nDefine a suite of actions to process events, abstract intelligence for your microservices.\n\nOnce defined, use your workflows to process your Events.\n\n## Example\n\nWe have 3 microservices:\n- The `upper` microservice that transforms texts into uppercase\n- The `reverse` microservice that reverses a text\n- The `space-counter` microservice that returns the number of space in a string\n\nSuppose we want to process some text by making it uppercase and reversing it.\n\nWe will define the following definitions:\n- [`examples/definitions/service-upper.json`](./examples/definitions/service-upper.json)\n- [`examples/definitions/service-reverse.json`](./examples/definitions/service-reverse.json)\n- [`examples/definitions/service-space-counter.json`](./examples/definitions/service-space-counter.json)\n\nThen, we can define a Workflow in [`examples/definitions/workflow-process-text.json`](./examples/definitions/workflow-process-text.json).\n\nAfter having created our Workflow, we can send our first Event !\n\nWe will send the content of [`examples/definitions/event-text.json`](./examples/definitions/event-text.json)\n\n## Goals\n\n- Register services (ex: a new API)\n- Define a simple Workflow (subsequent calls to different services)\n- Allow async (expose endpoint for webhook?)\n- Visualisation\n\n## Development\n\nThe recommended approach is to use `virtualenv` to install the API's dependencies.\n\n```console\n$ pip install virtualenv\n$ python -m virtualenv venv\n$ source venv/bin/activate\n$ pip install -r requirements_dev.txt\n$ pip install -e .\n```\n\nIn order to ease iterations, some helpers scripts are available.\n\n**start the database**\n\nStarts a background MongoDB container in host networking mode.\n\n```console\n$ ./run-background-service.sh\n```\n\n**run the api in development mode**\n\nStart the Flask Application in development + debug mode.\n\n```console\n$ ./run_app \n * Serving Flask app 'machinery.api:create_app' (lazy loading)\n * Environment: development\n * Debug mode: on\n * Running on all addresses.\n   WARNING: This is a development server. Do not use it in a production deployment.\n```\n\n## Testing\n\nEvery testing-related dependency is listed in the `requirements_dev.txt` file.\n\n```console\n$ pip install -r requirements_dev.txt\n```\n\n### Unit tests\n\nThis project implements unittests using `pytest`.\n\nIn a configured environment, run the following command:\n```console\n$ pytest\n```\n\n### Linting\n\nThis project is linted using `pylint`\n\nIn a configured environment, run the following command:\n```console\n$ pylint machinery\n```\n\n### Functional tests\n\nThis API is tested using Postman/Newman. ([newman: Getting Started][newman])\n\nAssets are available in the `./postman` directory and expect the API to be accessible at `http://localhost:5000`.\nThis can be achieved using the `./run_app` script.\n\n**Example:**\n```console\n$ npm install newman\n$ ./node_modules/newman/bin/newman.js run ./postman/Machinery.postman_collection.json -e postman/Machinery\\ -\\ Local.postman_env.json \nnewman\n\nMachinery\n\n→ Create a Service\n  POST http://localhost:5000/s [201 CREATED, 197B, 38ms]\n  ✓  Status code is 201\n  ✓  Response contains a service_id\n\n┌─────────────────────────┬──────────────────┬──────────────────┐\n│                         │         executed │           failed │\n├─────────────────────────┼──────────────────┼──────────────────┤\n│              iterations │                1 │                0 │\n├─────────────────────────┼──────────────────┼──────────────────┤\n│                requests │                1 │                0 │\n├─────────────────────────┼──────────────────┼──────────────────┤\n│            test-scripts │                1 │                0 │\n├─────────────────────────┼──────────────────┼──────────────────┤\n│      prerequest-scripts │                0 │                0 │\n├─────────────────────────┼──────────────────┼──────────────────┤\n│              assertions │                2 │                0 │\n├─────────────────────────┴──────────────────┴──────────────────┤\n│ total run duration: 70ms                                      │\n├───────────────────────────────────────────────────────────────┤\n│ total data received: 47B (approx)                             │\n├───────────────────────────────────────────────────────────────┤\n│ average response time: 38ms [min: 38ms, max: 38ms, s.d.: 0µs] │\n└───────────────────────────────────────────────────────────────┘\n```\n\n[newman]: https://learning.postman.com/docs/running-collections/using-newman-cli/command-line-integration-with-newman/\n\n#### A Note on functional tests\n\nAs this Python project is to be considered to be a POC to define the required\nfeature set for the \"production-grade\" version of Machinery, this testing part\nis the most important than any other tests.\n\nThe next version _probably_ will not be written in Python and this contract\nwill allow to quickly develop a stable version.\n\nBeing able to define a usable API is the current goal for Machinery.\n\n## Components\n\n### API (WIP)\n\n_Could be divided into Consumer API and Management API._\n\n- Register services\n- Define Workflows (json with arbitrary service definition)\n- Treat Event\n\n## Data structures\n\n### Service\n\n```json\n{\n    \"name\": \"upper\",\n    \"address\": \"http://upper.local:5000/\",\n    \"inputs\": [\n         {\n            \"name\": \"message\",\n            \"type\": \"string\",\n            \"description\": \"The text to transform\"\n        }\n    ],\n    \"outputs\": [\n         {\n            \"name\": \"message\",\n            \"type\": \"string\",\n            \"description\": \"The transformed text, in uppercase\"\n        }\n    ]\n}\n```\n\n\ninput:\n- `name`: The name of the Service\n- `address`: The URL of the Service\n- `inputs`: Each input with its type and description\n- `outputs`: Each output with its type and description\n\noutput: TBD\n(service-id, ack date)\n\n### Event\n\nPOST `/e/\u003cworkflow-id\u003e`\n\n```json\n{\n    \"data\": {\n        \"message\": \"The Sun will rise in the morning !\"\n    }\n}\n```\n\nPOSTing an Event will return an `event-id`, an ack date and a status.\ninput:\n- `data`: The Event's data that **must** match the Workflow's inputs.\n\n### Workflow\n\n```json\n{\n    \"name\": \"process-text\",\n    \"services\": [\"upper\", \"reverse\", \"space-counter\"],\n    \"inputs\": [\n         {\n            \"name\": \"message\",\n            \"type\": \"string\",\n            \"description\": \"The text to process\"\n        }\n    ],\n    \"outputs\": [\n         {\n            \"name\": \"message\",\n            \"type\": \"string\",\n            \"description\": \"The processed text\"\n        },\n         {\n            \"name\": \"space_count\",\n            \"type\": \"int\",\n            \"description\": \"The number of spaces\"\n        }\n    ],\n    \"operations\": [ ]\n}\n```\n\ninput:\n- `name`: The name of the Workflow\n- `services`: The Services to be called, in order\n- `inputs`: The inputs of this Workflow\n- `outputs`: The outputs of this workflow\n\noutput: TBD\n(workflow-id, ack date)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftbobm%2Fmachinery","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftbobm%2Fmachinery","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftbobm%2Fmachinery/lists"}