{"id":28602056,"url":"https://github.com/masumi-network/kodosumi","last_synced_at":"2025-06-18T21:44:06.463Z","repository":{"id":279565043,"uuid":"939179736","full_name":"masumi-network/kodosumi","owner":"masumi-network","description":"runtime environment to manage and execute agentic services at scale","archived":false,"fork":false,"pushed_at":"2025-06-04T04:44:39.000Z","size":5995,"stargazers_count":7,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-09T14:29:22.807Z","etag":null,"topics":["agentic-ai","fastapi","litestar","ray"],"latest_commit_sha":null,"homepage":"https://www.kodosumi.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/masumi-network.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-02-26T05:54:24.000Z","updated_at":"2025-06-02T21:11:52.000Z","dependencies_parsed_at":null,"dependency_job_id":"a84e69e6-2eb0-47c6-9e36-4245eb0687ea","html_url":"https://github.com/masumi-network/kodosumi","commit_stats":null,"previous_names":["masumi-network/kodosumi"],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/masumi-network%2Fkodosumi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/masumi-network%2Fkodosumi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/masumi-network%2Fkodosumi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/masumi-network%2Fkodosumi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/masumi-network","download_url":"https://codeload.github.com/masumi-network/kodosumi/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/masumi-network%2Fkodosumi/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259274085,"owners_count":22832466,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agentic-ai","fastapi","litestar","ray"],"created_at":"2025-06-11T16:01:00.210Z","updated_at":"2025-06-11T16:02:44.802Z","avatar_url":"https://github.com/masumi-network.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kodosumi\n\nkodosumi is the runtime environment to manage and execute agentic services at scale. The system is based on [Ray](https://ray.io) - a distributed computing framework - and a combination of [litestar](https://litestar.dev/) and [fastapi](https://fastapi.tiangolo.com/) to deliver agentic services to users or other agents. Similar to Ray, kodosumi follows a _Python first_ agenda.\n\nkodosumi is one component of a larger eco system with [masumi and sokosumi](https://www.masumi.network/).\n\n![Eco System](./docs/assets/ecosystem.png)\n\n\n# introduction\n\nkodosumi consists of three main building blocks. First, a _Ray cluster_ to execute agentic services at scale. kodosumi builds on top of Ray and actively manages the lifecycle and events of service executions from _starting_ to _finished_ or _error_. No matter you name your code an application, flow, service or script: The third building block is _your application_ which runs on top of kodosumi.\n\nThe following architecture shows the relation between the three building blocks: 1) your service on top of 2) kodosumi which operates 3) a distributed compute cluster with Ray secure and at scale.\n\n[![kodosumi overview](./docs/assets/thumb/architecture.png)](./docs/assets/architecture.png)\n\nYou build and deploy your [Flow](./docs/concepts.md#flow) by providing an [endpoint](./docs/concepts.md#endpoint) (HTTP route) and an [entrypoint](./docs/concepts.md#entrypoint) (Python callable) to kodosumi (left bottom box in the diagram). kodosumi delivers features for [access control](./docs/api.md#authentication), [flow control](./docs/api.md#flow-control) and manages [flow execution](./docs/api.md#execution-control) with Ray [head node](./docs/concepts.md#ray-head) and [worker nodes](./docs/concepts.md#ray-worker). [kodosumi spooler](./docs/concepts.md#spooler) gathers flow execution results and outputs into the [event stream](./docs/concepts.md#event-stream).\n\nDeep-dive into [endpoints](./docs/concepts.md#endpoint) and how these translate into [entrypoints](./docs/concepts.md#entrypoint) of [flows](#flows) which operationalize the business logic of [agentic services](#agentic-service) or [agents](#agents) in the broader sense.\n\nIf you need further background information read [why kodosumi](./docs/why.md)\n\n\n# installation\n\nThe following quick guide\n\n1. installs kodosumi and all prerequisites along the kodosumi example flows\n2. starts Ray and kodosumi on your localhost\n3. deploys example flows\n\nThis installation has been tested with versions `ray==2.46.0` and `python==3.12.6`.\n\nIf you want to skip the examples then continue with the [kodosumi development workflow](./docs/develop.md) and start implementing your custom agentic service with the kodosumi framework.\n\n## install and run examples\n\n### STEP 1 - clone and install `kodosumi-examples`\n\nClone and install the `kodosumi-examples` into your Python Virtual Environment. The kodosumi and Ray packages are automatically installed as a dependency.\n\n```bash\ngit clone https://github.com/masumi-network/kodosumi-examples.git\ncd ./kodosumi-examples\ngit checkout dev\npip install .\n```\n\nSince some of the examples utilize additional frameworks like `CrewAI` and `langchain` the installation of the kodosumi examples takes a while. All dependencies of the examples are installed. Please note that these dependencies are managed by Ray in production. See [deployment](./docs/deploy.md). \n\n\n### STEP 2 - prepare runtime environment\n\nYou need an OpenAI API key to run some of the examples. Specify the API key in `.env`.\n\n    # ./env\n    OPENAI_API_KEY=\u003c-- enter-your-key --\u003e\n\n\n### STEP 3 - start ray\n\nStart Ray _head_ node on your localhost. Load environment variables with `dotenv` before starting ray:\n\n```bash\ndotenv run -- ray start --head\n```\n\nCheck `ray status` and visit ray dashboard at [http://localhost:8265](http://localhost:8265). For more information about ray visit [ray's documentation](https://docs.ray.io/en/latest).\n\n\n### STEP 4 - deploy\n\nYou have various options to deploy and run the example services. _kodosumi-examples_ repository ships with the following examples in `agentic.examples`:\n\n* **hymn** - creates a hymn based on a given topic. The example demonstrates the use of [CrewAI](https://www.crewai.com/) and [OpenAI](https://openai.com/)\n* **prime** - calculates prime number gaps. Distributes the tasks across the Ray cluster and demonstrates performance benefits.\n* **throughput** - real-time experience of different event stream pressures with parameterized BPMs (beats per minute).\n* **form** - demonstrates form elements supported by kodosumi.\n\nYou can run any of these examples. The next steps focus on `agentic.examples.hymn`.\n\n\n#### alternative 1: run with uvicorn\n\nYou can launch each example service as a python module.\n\n    uvicorn agentic.examples.hymn.app:app --port 8011\n\nThis starts a uvicorn (Asynchronous Server Gateway Interface) server at http://localhost:8011. All HTTP endpoints of `app` are available at URL http://localhost:8011/openapi.json. Launch another terminal session, source the Python Virtual Environment and register this URL with kodosumi panel:\n\n    koco start --register http://localhost:8011/openapi.json\n\nVisit kodosumi **[admin panel](http://localhost:3370)** at [http://localhost:3370](http://localhost:3370). The default user is defined in `config.py` and reads `name=admin` and `password=admin`. Launch the **[Hymn Creator](http://localhost:3370/inputs/-/localhost/8001/hymn/-/)** from the **[service screen](http://localhost:3370/admin/flow)** and revisit results at the **[timeline screen](http://localhost:3370/timeline/view)**.\n\nYou can start another service `prime` in a new terminal with\n\n    uvicorn agentic.examples.prime.app:app --port 8012\n\nRegister this service with [kodosumi panel config](http://localhost:3370/admin/routes) with both service endpoints\n\n    http://localhost:8011/openapi.json\n    http://localhost:8012/openapi.json\n\nYou can specify multiple _registers_ at `koco start`\n\n    koco start --register http://localhost:8011/openapi.json --register http://localhost:8012/openapi.json\n\nRunning your services as standalone uvicorn applicaitons is best practice to facilitate debugging. \n\n#### alternative 2: deploy and run with Ray serve\n\nRun your services as Ray serve deployments. This is the preferred approach to deploy services in production. The downside of this approach is that you have to use remote debugging tools and attach to session breakpoints for debugging (see [Using the Ray Debugger](https://docs.ray.io/en/latest/ray-observability/user-guides/debug-apps/ray-debugging.html)). \n\nRay Serve is built on top of Ray, so it easily scales to many machines and offers flexible scheduling support such as fractional GPUs so you can share resources and serve many applications at low cost.\n\nWith Ray _serve_ you either run or deploy your services. Instead of the mechanics with uvicorn which refers the `app` application object, Ray serve demands the bound `fast_app` object. To test and improve your service run it with\n\n    serve run agentic.examples.hymn.app:fast_app\n\nIn contrast to the previous command a `serve deploy` command is used to deploy your Serve application to the Ray cluster. It sends a deploy request to the cluster and the application is deployed asynchronously. This command is typically used for deploying applications in a production environment. \n\n    serve deploy agentic.examples.hymn.app:fast_app\n\nUsing Ray _serve run_ or _deploy_ the `--register` must connect to Ray's proxy URL `/-/routes`. With `serve run` or `deploy` the port defaults to `8000` and you start `koco start` with the Ray serve endpoint http://localhost:8000/-/routes.\n\n    koco start --register http://localhost:8000/-/routes\n\n\n#### multi-service setup with Serve config files\n\n`serve run` and `serve deploy` feature single services. Running multiple uvicorn services is possible but soon gets dity and quirky. For multi-service deployments use Ray serve _config files_.\n\nIn directory `./data/config` create a file `config.yaml` with _serve's_ overarching configuration, for example\n\n```yaml\nproxy_location: EveryNode\n\nhttp_options:\n  host: 127.0.0.1\n  port: 8001\n\ngrpc_options:\n  port: 9001\n  grpc_servicer_functions: []\n\nlogging_config:\n  encoding: TEXT\n  log_level: DEBUG\n  logs_dir: null\n  enable_access_log: true\n```\n\nAlongside this file `config.yaml` create service configuration files. For each service deployment create a dedicated configuration file:\n\n##### `hymn.yaml`\n\n```yaml\nname: hymn\nroute_prefix: /hymn\nimport_path: agentic.examples.hymn.app:fast_app\nruntime_env: \n  pip:\n  - crewai\n  - crewai_tools\n  env_vars:\n    OTEL_SDK_DISABLED: \"true\"\n    OPENAI_API_KEY: \u003c-- enter-you-API-key --\u003e\n```\n\n##### `prime.yaml`\n\n```yaml\nname: prime\nroute_prefix: /prime\nimport_path: agentic.examples.prime.app:fast_app\n```\n\n##### `throughput.yaml`\n\n```yaml\nname: throughput\nroute_prefix: /throughput\nimport_path: agentic.examples.throughput.app:fast_app\nruntime_env: \n  pip:\n  - lorem-text\n```\n\nTest this deployment set with\n\n    koco deploy ./data/config/config.yaml --test\n\n\nWith success, stop `ray serve` and perform a Ray serve deployment\n\n    serve shutdown --yes\n    koco deploy ./data/config/config.yaml\n\n\nRestart `koco start` with the Ray serve endpoint http://localhost:8001/-/routes as configured in `config.yaml`.\n\n    koco start --register http://localhost:8001/-/routes\n\n\nIf one or more Ray serve applications are not yet available when kodosumi starts, you need to refresh the list of registered flows. Visit **[control screen](http://localhost:3370/admin/routes)** in the **[admin panel](http://localhost:3370/)** and click **RECONNECT**. \n\nAdding and removing deployments is operationalized with config files in `./data/config`. All files alongside `config.yaml` are deployed. You can test your deployment setup with `koco deploy ./data/config/config.yaml --test`.\n\n\n**Where to get from here?**\n\n* continue with [kodosumi development workflow](./docs/develop.md) \n* see the admin panel [screenshots](./panel.md)\n* read about [basic concepts and terminology](./concepts.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmasumi-network%2Fkodosumi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmasumi-network%2Fkodosumi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmasumi-network%2Fkodosumi/lists"}