{"id":20636737,"url":"https://github.com/libcommon/task-py","last_synced_at":"2025-10-26T10:40:32.942Z","repository":{"id":62575372,"uuid":"220917568","full_name":"libcommon/task-py","owner":"libcommon","description":"Python library for implementing arbitrary tasks with support for ArgumentParser.","archived":false,"fork":false,"pushed_at":"2024-04-12T04:23:42.000Z","size":2440,"stargazers_count":1,"open_issues_count":7,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-18T09:39:22.332Z","etag":null,"topics":["library","python","task"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/libcommon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-11T06:41:07.000Z","updated_at":"2024-01-02T21:06:08.000Z","dependencies_parsed_at":"2024-11-16T15:12:05.704Z","dependency_job_id":"34f2b4a0-fd46-4ffc-815c-09d8fa6f8aa7","html_url":"https://github.com/libcommon/task-py","commit_stats":{"total_commits":22,"total_committers":1,"mean_commits":22.0,"dds":0.0,"last_synced_commit":"7f8b65f342a07e30fdeec25cba7f829aa0523e34"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/libcommon%2Ftask-py","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/libcommon%2Ftask-py/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/libcommon%2Ftask-py/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/libcommon%2Ftask-py/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/libcommon","download_url":"https://codeload.github.com/libcommon/task-py/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242656036,"owners_count":20164431,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["library","python","task"],"created_at":"2024-11-16T15:12:01.152Z","updated_at":"2025-10-26T10:40:27.917Z","avatar_url":"https://github.com/libcommon.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# task-py: Python Task Execution Framework\n\n## Overview\n\nOne day you need to write a script to process some data, so you sit down and write it. The filepath and other parameters\nare hard-coded, but it works. A few days later, you need to do the same thing, but need to accept a filepath on the\ncommand line and maybe one more parameter, so you refactor some functionality into functions and create a simple CLI\nusing `sys.argv`.  Then a coworker asks if he/she/they could use the script, so you finally break down and use `argparse`\nto write a better CLI.  If this cycle sounds familiar, `task-py` can help by allowing you to skip steps one and two, while\nwriting interoperable, composable tools.\n\n## Installation\n\n### Install from PyPI (preferred method)\n\n```bash\npip install lc_task\n```\n\n### Install Directly with Pip and Git\n\n```bash\npip install git+https://github.com/libcommon/task-py.git@vx.x.x#egg=lc_task\n```\n\nwhere `x.x.x` is the version you want to download.\n\n### Install from Cloned Repo\n\n```bash\ngit clone ssh://git@github.com/libcommon/task-py.git \u0026\u0026 \\\n    cd task-py \u0026\u0026 \\\n    pip install .\n```\n\n## Dependencies\n\nThe `tool.poetry.*dependencies` sections in [pyproject.toml](pyproject.toml) contain the app and dev dependencies.\nSee [DEVELOPMENT.md](DEVELOPMENT.md) for local development and build dependencies.\n\n## Getting Started\n\n### Creating a Task\n\nCreating a `Task` is simple: specify some (optional) attributes and implement `Task._perform_task`, which performs the primary unit of work.\nFor example, you could implement a task that takes a file path and prints the data of that file to stdout:\n\n```python\nimport typing\nfrom pathlib import Path\n\nfrom lc_task import Task, taskclass\n\n\n@taskclass\nclass CatFileTask(Task):\n    \"\"\"Print contents of a file to stdout.\"\"\"\n    input_path: typing.Optional[Path] = None\n\n    def _perform_task(self) -\u003e None:\n        if self.input_path is None:\n            raise ValueError(\"No input path provided\")\n        if not self.input_path.is_file():\n            raise FileNotFoundError(self.input_path)\n        print(self.input_path.read_text())\n\nif __name__ == \"__main__\":\n    import sys\n    # Example of how to use CatFileTask.run() to run the task\n    CatFileTask(input_path=Path(sys.argv[0])).run()\n```\n\nIn the example above, we define an attribute `input_path` on `CatFileTask` to store the input file path.\nFor tasks that require some setup and teardown steps, use the `Task._preamble` and `Task._postamble` functions,\nwhich get called before and after the primary unit of work is completed.\nTo run this task explicitly, use the `CatFileTask.run` method.\n\n### Accepting Command Line Input\n\nTo create a task that takes command line input (using `argparse`), create a child class of `CliTask` that defines\nthe name of the command, a brief description, and the command line arguments (plus optional command aliases). Taking the example above:\n\n```python\nimport typing\nfrom pathlib import Path\n\nfrom lc_task import CliTask, Task, taskclass\n\n\n@taskclass\nclass CatFileTask(Task):\n    \"\"\"Print contents of a file to stdout.\"\"\"\n    input_path: typing.Optional[Path] = None\n\n    def _perform_task(self) -\u003e None:\n        if self.input_path is None:\n            raise ValueError(\"No input path provided\")\n        if not self.input_path.is_file():\n            raise FileNotFoundError(self.input_path)\n        print(self.input_path.read_text())\n\n\n@taskclass\nclass CatFileCliTask(CliTask):\n    _task_cls = CatFileTask\n    command = \"cat\"\n    aliases = [\"c\"]\n    description = \"Print contents of a file to stdout.\"\n\n    @classmethod\n    def gen_command_parser(cls, parser: typing.Optional[ArgumentParser] = None) -\u003e ArgumentParser:\n        parser = super(CatFileCliTask, cls).gen_command_parser()\n        parser.add_argument(\"input_path\", type=Path, help=\"Path to input file\")\n        return parser\n\n\nif __name__ == \"__main__\":\n    CatFileTask.run_command()\n```\n\nThis example shows a few key features of the library:\n\n1. The command line parser for a CLI task is defined using the `CliTask.gen_command_parser` function.\n2. Command aliases can be added using the `CliTask.aiases` attribute.\n3. `Task`s and `CLiTask`s are easily composable using the `_task_cls` attribute of `CliTask`.\nBy default, `CliTask` merges the command line arguments with an instance of `_task_cls` and runs the task.\nThis behavior can be customized by overriding `Clitask._gen_task` and `Clitask._perform_task`.\n\nFor defining more complex, hierarchical command line interfaces with subcommands, look at the documentation for `cli.gen_cli_parser`.\nIt allows you to define your command line app with a mapping of commands to actions, then generates the CLI for you.\n\n### Pipelines and Message Passing\n\nAnyone who has used Bash, PowerShell, or other scripting languages is familiar with the idea of composability and pipes:\nwriting simple commands that return structured data recognized by other commands can be very powerful.\nPython does not support this style of programming out of the box, but it does support overloading operators for custom types,such as the bitwise or operator (`|`).\nIn Python, the [`__ror__` builtin method](https://docs.python.org/3/reference/datamodel.html#object.__or__)\nis called when using the bitwise or operator on two objects where the first (left side) doesn't implement `__or__`.\n\n`task-py` takes advantage of this flexibility to allow piping `Task`s together to create pipelines. For example, suppose\nyou were writing a CSV handing toolkit and wanted to create a command line app that reads data from a CSV and removes some columns.\nThere are two clear steps in this pipeline:\n\n1. Read data from the CSV into some data structure\n2. Remove specified columns (and write to stdout)\n\n```python\nimport csv\nimport typing\nfrom pathlib import Path\n\nfrom lc_task import CliTask, Task, TaskResult, taskclass\n\n\n@taskclass\nclass CsvColumnRemovalTask(Task):\n    \"\"\"Remove specified columns from frows in a CSV.\"\"\"\n    __slots__ = (\"columns\", \"input_file\", \"reader\")\n    columns_to_remove: typing.Optional[typing.List[str]] = None\n    records: typing.Optional[typing.List[typing.Dict[str, typing.Any]]] = None\n\n    def _perform_task(self) -\u003e None:\n        if self.columns and self.records:\n            # Print the header row\n            print(\",\".join(records[0].keys()))\n            for record in records:\n                for column in self.columns:\n                    del record[column]\n                # Print the record\n                print(\", \".join(record.values()))\n\n\n@taskclass\nclass CsvReaderTaskResult(TaskResult):\n    records: typing.Optional[typing.List[typing.Dict[str, typing.Any]]] = None\n\n\n@taskclass\nclass CsvReaderTask(Task):\n    \"\"\"Read data from a CSV into a `csv.DictReader` object.\"\"\"\n    _result_cls = CsvReaderTaskResult\n\n    input_path: typing.Optional[Path] = None\n\n    def _perform_task(self) -\u003e None:\n        if self.input_path is None:\n            raise ValueError(\"Must provide path to or file handle of CSV file\")\n        if not self.input_path.is_file():\n            raise FileNotFoundError(self.input_file)\n\n        with self.input_path.open(newline=\"\") as input_file:\n            self.result.records = [record for record in csv.DictReader(input_file, csv.unix_dialect)]\n\n\n@taskclass\nclass RemoveColumnsCliTask(CliTask):\n    command = \"remove-columns\"\n    description = \"Remove columns from a CSV and write to disk.\"\n\n    @classmethod\n    def gen_command_parser(cls, parser: Optional[ArgumentParser] = None) -\u003e ArgumentParser:\n        parser = super(RemoveColumnsCliTask, cls).gen_command_parser()\n        parser.add_argument(\"input_path\", type=Path, help=\"Path to input file\")\n        parser.add_argument(\"columns\", nargs=+, help=\"Column names to remove\")\n        return parser\n\n    def _perform_task(self) -\u003e None:\n        columns_to_remove: typing.List[str] = self.args.columns\n        input_path: Path = self.args.path = self.args.input_path\n        # Pipe the output of CSVReaderTask to CSVColumnRemovalTask\n        (CSVReaderTask(input_path=input_path).run() |\n         CSVColumnRemovalTask(columns_to_remove=columns_to_remove))\n\n\nif __name__ == \"__main__\":\n    RemoveColumnsCLITask.run_command()\n```\n\n## Development\n\nSee [DEVELOPMENT.md](DEVELOPMENT.md) for development instructions.\n\n## Contributing/Suggestions\n\nContributions and suggestions are welcome! To make a feature request, report a bug, or otherwise comment on existing functionality, please file an issue.\nFor contributions please submit a PR, but make sure to lint, type-check, and test your code before doing so. Thanks in advance!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flibcommon%2Ftask-py","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flibcommon%2Ftask-py","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flibcommon%2Ftask-py/lists"}