{"id":19440750,"url":"https://github.com/streamingflow/dispel4py-execution","last_synced_at":"2026-06-08T21:31:02.198Z","repository":{"id":249834123,"uuid":"832697996","full_name":"StreamingFlow/dispel4py-execution","owner":"StreamingFlow","description":"Execution engine for Laminar (serverless framework)","archived":false,"fork":false,"pushed_at":"2024-08-27T10:25:29.000Z","size":263,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-07T21:47:38.116Z","etag":null,"topics":["dispel4py","execution","execution-environment","serverless-framework","stream-based"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/StreamingFlow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-23T14:40:26.000Z","updated_at":"2024-08-27T10:09:59.000Z","dependencies_parsed_at":"2024-08-27T11:29:20.123Z","dependency_job_id":"aa4a5097-26d8-4a1c-8286-a486dc39811a","html_url":"https://github.com/StreamingFlow/dispel4py-execution","commit_stats":null,"previous_names":["streamingflow/dispel4py-execution"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StreamingFlow%2Fdispel4py-execution","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StreamingFlow%2Fdispel4py-execution/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StreamingFlow%2Fdispel4py-execution/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StreamingFlow%2Fdispel4py-execution/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/StreamingFlow","download_url":"https://codeload.github.com/StreamingFlow/dispel4py-execution/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240621571,"owners_count":19830569,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dispel4py","execution","execution-environment","serverless-framework","stream-based"],"created_at":"2024-11-10T15:30:42.228Z","updated_at":"2026-06-08T21:30:57.172Z","avatar_url":"https://github.com/StreamingFlow.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Laminar Logo](logo.webp)\n\n# Introduction\n\nThe Laminar Execution Engine is a crucial component of the Laminar framework, designed to execute dispel4py workflows. This is the only part of the system where [dispel4py](https://github.com/StreamingFlow/d4py), along with Redis, is installed and configured. The execution engine is built as a Flask application that handles and manages workflow execution requests from multiple users.\n\nWhen a user submits a workflow execution request, the engine processes it through a queue system. This ensures that multiple workflows can be handled efficiently and concurrently. The engine supports various processing modes, including simple sequential execution, multiprocessing, and dynamic processing using Redis, making it flexible and scalable to meet different computational needs.\n\nThe engine communicates with the [Laminar Server](https://github.com/StreamingFlow/dispel4py-server) to retrieve and execute workflows on behalf of the [client](https://github.com/StreamingFlow/dispel4py-client), ensuring that all necessary resources are managed effectively. The results of an execution are directly sent back to the client.  Whether running locally or within a Docker container, the execution engine is responsible for the core computational tasks of the Laminar framework.\n\nLaminar takes full advantage of dispel4py’s capabilities, including automatic parallelization of workflows and dynamic resource provisioning when running workflows in parallel or dynamically. These features ensure efficient and scalable execution of data-intensive applications.\n\n# Laminar Execution Instructions\n\nThe following instructions will allow you to run the Flask application which executes dispel4py workflows \n\n# Docker Installation\nClone repository \n```\ngit clone https://github.com/StreamingFlow/dispel4py-execution.git\n```\nThen enter directory by\n```\ncd dispel4py-execution \n```\n\nRun docker compose to load up the execution engine and redis server. The first time we recommend to use --build flag.\n```\ndocker compose up --build\n```\nNext time you could use:\n```\ndocker compose up\n```\n\nIf you need to rebuild the Docker containers (for instance, after making configuration changes), you can do so by following these steps:\n\nFirst, bring down the running containers:\n```\ndocker-compose down\n```\nThen, rebuild and start the containers:\n```\ndocker-compose up --build\n```\nBy following these steps, you can ensure that the execution engine is properly configured and running efficiently.\n\n\n# Non-Docker Installation\n\nClone repository \n```\ngit clone https://github.com/Laminar-2/dispel4py-execution.git\n```\nThen enter directory by\n```\ncd dispel4py-execution \n```\nIn order to run the application you need to create a new Python 3.10 enviroment\n```\n--note conda must be installed beforehand, go to https://conda.io/projects/conda/en/stable/user-guide/install/linux.html\nconda create --name py10 python=3.10\nconda activate py10\n```\nInstall app modules\n```\npip install -r requirements.txt\n```\nRun application\n```\npython app.py\n```\n\n## Other Laminar components\n\nThe [laminar client](https://github.com/StreamingFlow/dispel4py-client) offers a user-friendly interface for registering and managing Processing Elements (PEs) as well as stream-based dispel4py workflows. For detailed guidance on how to interact with the system, please refer to the [user manual](https://github.com/StreamingFlow/dispel4py-client/wiki) available on the project's wiki.\n\nIn addition, [laminar server](https://github.com/StreamingFlow/dispel4py-server) is another essential component that must be installed and running, either locally or remotely. The server acts as the central hub for managing communication and coordination between the client and the execution engine. It also communicates with the registry, which stores, workflows, PEs, and other essential metadata.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstreamingflow%2Fdispel4py-execution","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstreamingflow%2Fdispel4py-execution","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstreamingflow%2Fdispel4py-execution/lists"}