{"id":20534332,"url":"https://github.com/pyapp-org/pyapp-flow","last_synced_at":"2025-04-14T06:53:49.978Z","repository":{"id":38493903,"uuid":"496823063","full_name":"pyapp-org/pyapp-flow","owner":"pyapp-org","description":"A simple application level workflow.","archived":false,"fork":false,"pushed_at":"2025-03-24T03:52:46.000Z","size":641,"stargazers_count":1,"open_issues_count":4,"forks_count":0,"subscribers_count":1,"default_branch":"develop","last_synced_at":"2025-03-24T04:01:33.675Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://pyapp-flow.readthedocs.io/en/latest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pyapp-org.png","metadata":{"files":{"readme":"README.md","changelog":"HISTORY","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-27T01:22:59.000Z","updated_at":"2025-03-24T03:38:20.000Z","dependencies_parsed_at":"2024-11-16T00:36:15.930Z","dependency_job_id":null,"html_url":"https://github.com/pyapp-org/pyapp-flow","commit_stats":null,"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyapp-org%2Fpyapp-flow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyapp-org%2Fpyapp-flow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyapp-org%2Fpyapp-flow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyapp-org%2Fpyapp-flow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pyapp-org","download_url":"https://codeload.github.com/pyapp-org/pyapp-flow/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248837281,"owners_count":21169374,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T00:26:12.365Z","updated_at":"2025-04-14T06:53:49.952Z","avatar_url":"https://github.com/pyapp-org.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pyapp-flow\nA simple application level workflow library.\n\nAllows complex processes to be broken into smaller specific steps, greatly \nsimplifying testing and re-use.\n\n[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=pyapp-org_pyapp-flow\u0026metric=alert_status)](https://sonarcloud.io/summary/new_code?id=pyapp-org_pyapp-flow)\n[![Security Rating](https://sonarcloud.io/api/project_badges/measure?project=pyapp-org_pyapp-flow\u0026metric=security_rating)](https://sonarcloud.io/summary/new_code?id=pyapp-org_pyapp-flow)\n[![Coverage](https://sonarcloud.io/api/project_badges/measure?project=pyapp-org_pyapp-flow\u0026metric=coverage)](https://sonarcloud.io/summary/new_code?id=pyapp-org_pyapp-flow)\n[![Once you go Black...](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black)\n\n\n## Installation\n\n```shell\npip install pyapp-flow\n```\n\n\n## Usage\n\n```python\nfrom pathlib import Path\nfrom typing import Sequence\nimport pyapp_flow as flow\n\n# Define steps:\n\n@flow.step(name=\"Load Names\", output=\"names\")\ndef load_names(root_path: Path) -\u003e Sequence[str]:\n    \"\"\"\n    Read a sequence of names from a file\n    \"\"\"\n    with (root_path / \"names.txt\").open() as f_in:\n        return [name.strip() for name in f_in.readlines()]\n\n@flow.step(name=\"Say hello\")\ndef say_hi(name: str):\n    print(f\"Hello {name}\")\n\n# Define a workflow:\n\ngreat_everybody = (\n    flow.Workflow(name=\"Great everybody in names file\")\n    .nodes(\n      load_names,\n      flow.ForEach(\"name\", in_var=\"names\").loop(say_hi)\n    )\n)\n\n# Execute workflow:\n\ncontext = flow.WorkflowContext(root_path=Path())\ngreat_everybody(context)\n```\n\nAll nodes within the workflow follow a simple interface of:\n```python\ndef node_function(context: flow.WorkflowContext):\n    ...\n```\nor using typing\n```python\nNodeFunction = Callable[[flow.WorkflowContext], Any]\n```\n\nThe `step` decorator simplifies definition of a step by handling loading and saving \nof state variables from the `WorkflowContext`.\n\n\n## Reference\n\n### Workflow\n\nAt the basic level a workflow is an object that holds a series of nodes to be called \nin sequence. The workflow object also includes helper methods to generate and append\nthe nodes defined in the *Builtin Nodes* section of the documentation. \n\nJust like every node in pyApp-Flow a workflow is called with an `WorkflowContext` \nobject, this means workflows can be nested in workflows, or called from a for-each \nnode.\n\nThe one key aspect with a workflow object is related to context variable scope. \nWhen a workflow is triggered the context scope is copied and any changes made \nto the variables are discarded when the workflow ends. However, just like Python \nscoping only the reference to the variable is copied meaning mutable objects can \nbe modified (eg list/dicts).\n\n```python\nworkflow = (\n    flow.Workflow(name=\"My Workflow\")\n    .nodes(...)\n)\n```\n\n### WorkflowContext\n\nThe workflow context object holds the state of the workflow including handling \nvariable scoping and helper methods for logging progress.\n\n**Properties**\n\n- `state` \n\n  Direct access to state variables in the current scope.\n\n- `depth` \n \n  Current scope depth\n\n- `indent` \n\n  Helper that returns a string indent for use formatting messages\n\n**Methods**\n\n- `format`\n\n  Format a string using values from the context state. Most *name*\n  values for nodes/workflows use this method to allow values to be included\n  from scope eg:\n\n  ```python\n  context.format(\"Current path {working_path}\")\n  ```\n\n- `push_state`/`pop_state`\n\n  Used to step into or out of a lower state scope. Typically these methods are\n  not called directly but are called via using a with block eg:\n  \n  ```python\n  with context:\n      pass  # Separate variable scope \n  ```\n\n- Logging wrappers\n\n  Wrappers around an internal workflow logger that handle indentation to make\n  reading the log easier.\n  \n  - log\n  - debug\n  - info\n  - warning\n  - error\n  - exception\n\n\n\n### Builtin Nodes\n\n**Modify context variables**\n\n- `SetVar`\n  \n    Set one or more variables into the context\n\n    ```python\n    SetVar(my_var=\"foo\")\n    ```\n\n- `Append`\n\n    Append a value to a list in the context object (will create the list if it \n    does not exist).\n\n    ```python\n    Append(\"messages\", \"Operation failed to add {my_var}\")\n    ```\n  \n- `CaptureErrors`\n\n    Capture and store any errors raised by node(s) within the capture block to a \n    variable within the context.\n\n    ```python\n    CaptureErrors(\"errors\").nodes(my_flaky_step)\n    ```\n  \n    This node also has a `try_all` argument that controls the behaviour when an  \n    error is captured, if `True` every node is called even if they all raise errors,\n    this is useful for running a number of separate tests that may fail.\n\n    ```python\n    CaptureErrors(\n        \"errors\", \n        try_all=True\n    ).nodes(\n        my_first_check, \n        my_second_check, \n    )\n    ```\n\n**Provide feedback**\n\n- `LogMessage`\n    \n    Insert a message within optional values from the context into the runtime \n    log with an optional level.\n    \n    ```python\n    LogMessage(\"State of my_var is {my_var}\", level=logging.INFO)\n    ```\n\n\n**Branching**\n\nBranching nodes utilise a fluent interface for defining the nodes within each \nbranch. \n\n- `Conditional` / `If`\n    \n    Analogous with an `if` statement, it can accept either a context variable \n    that can be interpreted as a `bool` or a function/lamba that accepts a \n    `WorkflowContext` object and returns a `bool`.\n\n    ```python \n    # With context variable\n    (\n        If(\"is_successful\")\n        .true(log_message(\"Process successful :)\"))\n        .false(log_message(\"Process failed :(\"))\n    )\n  \n    # With Lambda\n    (\n        If(lambda context: len(context.state.errors) == 0)\n        .true(log_message(\"Process successful :)\"))\n        .false(log_message(\"Process failed :(\"))\n    )\n    ```\n  \n- `Switch`\n\n    Analogous with a `switch` statement found in many languages or with Python \n    a `dict` lookup with a default fallback.\n\n    Like the conditional node switch can accept a context variable or a \n    function/lambda that accepts a `WorkflowContext`, except returns any *hashable*\n    object.\n\n    ```python\n    # With context variable\n    (\n        Switch(\"my_var\")\n        .case(\"foo\", log_message(\"Found foo!\"))\n        .case(\"bar\", log_message(\"Found bar!\"))\n        .default(log_message(\"Found neither.\"))\n    )\n  \n    # With Lambda\n    (\n        Switch(lambda context: context.state[\"my_var\"])\n        .case(\"foo\", log_message(\"Found foo!\"))\n        .case(\"bar\", log_message(\"Found bar!\"))\n    )\n    ```\n  \n\n**Iteration**\n\n- `ForEach`\n    \n    Analogous with a `for` loop this node will iterate through a sequence and \n    call each of the child nodes.\n\n    All nodes within a for-each loop are in a nested context scope.\n    \n    ```python\n    # With a single target variable\n    (\n        ForEach(\"message\", in_var=\"messages\")\n        .loop(log_message(\"- {message}\"))\n    )\n  \n    # With multiple target variables\n    (\n        ForEach(\"name, age\", in_var=\"students\")\n        .loop(log_message(\"- {name} is {age} years old.\"))\n    )\n    ```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpyapp-org%2Fpyapp-flow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpyapp-org%2Fpyapp-flow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpyapp-org%2Fpyapp-flow/lists"}