{"id":41082243,"url":"https://github.com/0hsn/hence","last_synced_at":"2026-01-22T13:42:00.315Z","repository":{"id":222234610,"uuid":"756663460","full_name":"0hsn/hence","owner":"0hsn","description":"Hence, a simple but powerful framework designed to streamline data pipeline, scraping, automation workflow orchestration.","archived":false,"fork":false,"pushed_at":"2025-04-07T06:07:08.000Z","size":256,"stargazers_count":0,"open_issues_count":8,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-11-27T17:26:24.235Z","etag":null,"topics":["dataloading","python-workflow","webscraping","workflow-automation","workflow-engine"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/0hsn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-13T03:55:54.000Z","updated_at":"2025-04-07T06:07:11.000Z","dependencies_parsed_at":"2024-06-15T15:53:38.728Z","dependency_job_id":"c961e6eb-0bcf-4356-ab44-22fb1943ffdd","html_url":"https://github.com/0hsn/hence","commit_stats":null,"previous_names":["chkware/hench","chkware/hence"],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/0hsn/hence","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0hsn%2Fhence","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0hsn%2Fhence/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0hsn%2Fhence/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0hsn%2Fhence/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/0hsn","download_url":"https://codeload.github.com/0hsn/hence/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0hsn%2Fhence/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28663854,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-22T01:17:37.254Z","status":"online","status_checked_at":"2026-01-22T02:00:07.137Z","response_time":144,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dataloading","python-workflow","webscraping","workflow-automation","workflow-engine"],"created_at":"2026-01-22T13:41:59.549Z","updated_at":"2026-01-22T13:42:00.295Z","avatar_url":"https://github.com/0hsn.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Hence - A minimal python workflow engine\n\n## Introduction\n\nWelcome to _Hence_, a powerful framework designed to streamline your workflow orchestration process. \n\nWhether you're involved in web scraping, data loading, fetching, or any other repetitive task, _Hence_ offers a comprehensive solution to break down these tasks into manageable units of work.\n\nBy orchestrating these units sequentially, _Hence_ empowers you to focus on the big picture without the hassle of manually ensuring the success of each step.\n\n## Features\n\n- **Task Breakdown** – Hence breaks complex tasks into smaller, manageable units for better organization and execution.\n- **Workflow Orchestration** – Automate workflows with Hence to ensure smooth, sequential execution without manual effort.\n- **Error Handling** – Hence manages errors gracefully, preventing workflow interruptions and ensuring seamless execution.\n- **Scalability** – Whether small or large-scale, Hence adapts effortlessly to your needs for optimal performance.\n\n## Use-cases\n\n- **Web Scraping** – Hence automates web scraping by breaking tasks into fetching, extracting, and storing data.\n- **Data Loading/Fetching** – Hence streamlines fetching from APIs and loading data into databases effortlessly.\n- **Repetitive Tasks** – Automate reports, file processing, and data transformations with Hence to save time and effort.\n\n## Setup / Installation\n\n### Use as library\n\n#### Install from Pypi\n\n```shell\npip install -U hence\n```\n\n#### Install from Github\n\n```shell\npip install -U git+https://github.com/0hsn/hence.git@main\n```\n\nor a specific tag\n\n```shell\npip install -U git+https://github.com/0hsn/hence.git@v0.12.1\n```\n\n### Development setup\n\n#### Prerequisite\n\n- [Poetry](https://python-poetry.org/docs/#installation)\n\n#### Local installation steps\n\n- Firstly, clone the repository\n- Setup with development tools\n\n  ```shell\n  pipenv install --dev\n  ```\n\n### Testing\n\n```shell\npoetry run py.test -s\n```\n\n### Samples\n\n```shell\npoetry run python -m samples.web_scraping\n```\n\n## API\n\n- [Pipeline](#pipeline)\n\n  - [Pipeline.add_task](#pipelineadd_task)\n  - [Pipeline.re_add_task](#pipelinere_add_task)\n  - [Pipeline.parameter](#pipelineparameter)\n  - [Pipeline.run](#pipelinerun)\n\n- [PipelineContext](#pipelinecontext)\n\n### Pipeline\n\n#### Pipeline.add_task\n\nAdd a task to pipeline using decorator. This decorator is useful, when you want to define a function and make it pipeline task at the same time.\n\n##### Signature\n\n```python\ndef add_task(uid: typing.Optional[str] = None, pass_ctx: bool = False) -\u003e typing.Any\n```\n\n##### Parameters\n\n`uid: str | None` Optional. Default: `None`. A unique name for a task function in a pipeline. If same id passed, should replace older assignment.\n\n`pass_ctx: bool` Optional. Default: `False`. Pass [PipelineContext](#pipelinecontext) as 1st parameter to task function. If true, the 1st parameter to the function\n\n##### Example\n\n```python\n@pipeline.add_task(pass_ctx=True)\ndef function_1(ctx: PipelineContext, a: str):\n    return a\n```\n\n#### Pipeline.re_add_task\n\nAdd a task to pipeline. This function is useful, when you want to define a function early and make it pipeline task later.\n\n##### Signature\n\n```python\ndef re_add_task(function: typing.Callable, uid: typing.Optional[str] = None, pass_ctx: bool = False) -\u003e None\n```\n\n##### Parameters\n\n`function: typing.Callable` Required. A function to act as a pipeline task.\n\n`uid: str | None` Optional. Default: `None`. A unique name for a task function in a pipeline. If same id passed, should replace older assignment.\n\n`pass_ctx: bool` Optional. Default: `False`. Pass [PipelineContext](#pipelinecontext) as 1st parameter to task function. If true, the 1st parameter to the function\n\n##### Example\n\n```python\ndef function_1(ctx: PipelineContext, a: str):\n    return a\n\npipeline.re_add_task(function_1, pass_ctx=True)\n```\n\n#### Pipeline.parameter\n\nAdd parameters before [Pipeline.run](#pipelinerun). This function passes parameters when running the task.\n\n##### Signature\n\n```python\ndef parameter(self, **kwargs) -\u003e typing.Self\n```\n\n##### Parameters\n\npass the function name or registered uid for the function as parameter.\n\n##### Example\n\n```python\ndef function_1(ctx: PipelineContext, a: str):\n    return a\n\ndef function_2(ctx: PipelineContext, a: str):\n    return a\n\npipeline.re_add_task(function_1, pass_ctx=True)\npipeline.re_add_task(function_2, uid=\"r_func\")\n\npipeline\n    .parameter(function_1={\"a\": \"Some string\"})\n    .parameter(r_func={\"a\": \"Some string\"})\n```\n\n#### Pipeline.run\n\nRun the pipeline.\n\n##### Signature\n\n```python\ndef run(self, is_parallel: bool = False) -\u003e dict[str, typing.Any]:\n```\n##### Parameters\n\n`is_parallel: bool` Optional. To run added tasks in parallel.\n\n##### Example\n\n```python\ndef function_1(ctx: PipelineContext, a: str):\n    return a\n\ndef function_2(ctx: PipelineContext, a: str):\n    return a\n\npipeline.re_add_task(function_1, pass_ctx=True)\npipeline.re_add_task(function_2, uid=\"r_func\")\n\noutput = pipeline.run()\n\n# or in parallel, since these tasks are not dependent\noutput = pipeline.run(True)\n```\n\nThis function outputs a dictionary containing all function returns, by function name or uid (if used).\n\n### PipelineContext\n\nPipelineContext is a class that holds all the operation data for a certain [Pipeline](#pipeline).\n\n- PipelineContext is passed when `.add_task(pass_ctx=True, ..` or `.re_add_task(.., pass_ctx=True, ..`.\n- remember to add a variable as 1st parameter to function when `pass_ctx` is `True`.\n\n##### Members\n\n`result: dict[str, typing.Any]`. A dictionary containing returns from the executed functions in a certain pipeline.\n\n`parameters: dict[str, dict[str, typing.Any]]` A dictionary containing all the parameters passed using [Pipeline.parameter](#pipelineparameter).\n\n`sequence: list[str]` A list containing all the functions added as task to a certain pipeline.\n\n`functions: dict[str, typing.Callable]` A dictionary containing all the functions added as task via [Pipeline.add_task](#pipelineadd_task) and [Pipeline.re_add_task](#pipelinere_add_task).\n\n## Contributions\n\n- Read [CONTRIBUTING](./CONTRIBUTING) document before you contribute.\n- [Create issues](https://github.com/0hsn/hence/issues) for any questions or request\n---\n\nLicensed under [AGPL-3.0](./LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F0hsn%2Fhence","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F0hsn%2Fhence","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F0hsn%2Fhence/lists"}