{"id":30288342,"url":"https://github.com/soren-n/tickle","last_synced_at":"2025-08-16T22:04:15.616Z","repository":{"id":57475382,"uuid":"442445523","full_name":"soren-n/tickle","owner":"soren-n","description":"Command line workflow automation tool which performs task graph scheduling and concurrent task evaluation.","archived":false,"fork":false,"pushed_at":"2022-02-21T15:25:48.000Z","size":313,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-07-28T07:47:29.052Z","etag":null,"topics":["command-line-tool","library","python3","task-automation","task-runner","task-scheduler","workflow-automation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/soren-n.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-12-28T11:43:06.000Z","updated_at":"2023-11-20T18:04:49.000Z","dependencies_parsed_at":"2022-09-07T17:13:01.408Z","dependency_job_id":null,"html_url":"https://github.com/soren-n/tickle","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/soren-n/tickle","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soren-n%2Ftickle","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soren-n%2Ftickle/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soren-n%2Ftickle/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soren-n%2Ftickle/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/soren-n","download_url":"https://codeload.github.com/soren-n/tickle/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soren-n%2Ftickle/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270775881,"owners_count":24642964,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-16T02:00:11.002Z","response_time":91,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["command-line-tool","library","python3","task-automation","task-runner","task-scheduler","workflow-automation"],"created_at":"2025-08-16T22:03:37.545Z","updated_at":"2025-08-16T22:04:15.594Z","avatar_url":"https://github.com/soren-n.png","language":"Python","funding_links":["https://github.com/sponsors/soren-n"],"categories":[],"sub_categories":[],"readme":"[![GitHub](https://img.shields.io/github/license/soren-n/tickle)](https://github.com/soren-n/tickle/blob/main/LICENSE)\n[![PyPI](https://img.shields.io/pypi/v/tickle-soren-n)](https://pypi.org/project/tickle-soren-n/)\n[![PyPI - Downloads](https://img.shields.io/pypi/dm/tickle-soren-n)](https://pypi.org/project/tickle-soren-n/)\n[![Discord](https://img.shields.io/discord/931473325543268373?label=discord)](https://discord.gg/bddF43Vk2q)\n[![GitHub Sponsors](https://img.shields.io/github/sponsors/soren-n)](https://github.com/sponsors/soren-n)\n\n# Tickle\nA command line workflow automation tool which performs task graph scheduling and concurrent task evaluation.\n\nThink of tickle as a generalised version of [ninja](https://github.com/ninja-build/ninja), i.e. not just for compiling native code projects, but for arbitrary concurrent evaluation of command line tasks. Tickle was initially conceived as a general backend for build systems, but can be used as a backend for any system that needs to schedule and evaluate command line tasks that produce and consume files concurrently.\n\nTickle takes as input a description of tasks to be performed and their dependencies; this is in the form of two files: _agenda_ and _depend_. It then compiles an evaluation schedule for these tasks; checking their input and output files for changes against a persistent cache, as well as checking status of task dependencies. I.e. only tasks that need to or can be re/evaluated will be scheduled.\n\n# Install\nTickle is currently only supported for Python \u003e=3.9, although it might work with older versions. It is distributed with pip and can be installed with the following example command:\n```\n$ python3 -m pip install tickle-soren-n\n```\n\nThe pip install above will also install the following project dependencies:\n\n- [PyYAML](https://github.com/yaml/pyyaml)\n- [Watchdog](https://github.com/gorakhargosh/watchdog)\n\nIf you love Tickle, then please go give the author/maintainers of these projects some love too!\n\n# Modes\nTickle has two evaluation modes:\n\n- The __offline mode__ builds the task graph once and will schedule and generally evaluate once from there. However it will watch the _depend_ file for dynamic dependency changes, and reschedule as necessary.\n- The __online mode__ will watch the _agenda_ file and the initial input files for changes, as well as the _depend_ file for dynamic dependency changes. It will then dynamically schedule and evaluate as the task graph changes.\n\nIn build system terms, offline mode is like a regular build, and online mode is like a watch/dev/live build. Both are incremental.\n\n# Assumptions\nTickle assumes that it has full control over the input and output files described in the agenda. The only files that tickle supports modification to during runtime are: the agenda file, the depend file and the initial input files.\n\nAs such if you need to interface/overlap tickle with other systems in a workflow, it is safest that you do so modally; i.e. before tickle is run, and after it has terminated. However it should be safe to overlap your system's runtime with tickle's, iff your system only reads the generated files.\n\n# Usage\nTickle can be used through two interfaces; the CLI and the API.\n\n## CLI\n```\nusage: tickle [-h] [--debug] [-w WORKERS] [-a AGENDA] [-d DEPEND] [-c CACHE] [-l LOG]\n              {offline,online,clean,version}\n\nTask graph scheduling with asynchronous evaluation.\n\npositional arguments:\n  {offline,online,clean,version}\n                        Offline mode for an inattentive evaluation mode where file\n                        modifications are ignored once tasks have been scheduled. Online\n                        mode for an attentive evaluation mode where file creations or\n                        modifications trigger a rescheduling of the task graph. Clean mode\n                        will delete all files and folders generated during offline or\n                        online evaluation. Version mode will print the tool version.\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --debug               Sets debug logging level for tool messages (default: False)\n  -w WORKERS, --workers WORKERS\n                        The number of concurrent workers; defaults to the number of\n                        logical cores minus one for the main thread (default: e.g. 3)\n  -a AGENDA, --agenda AGENDA\n                        Agenda YAML file location; contains the procedure and task\n                        definitions, file path must be relative to current working\n                        directory (default: agenda.yaml)\n  -d DEPEND, --depend DEPEND\n                        Depend YAML file location; contains a map of dynamic task\n                        dependencies, this file is optional, file path must be relative to\n                        current working directory (default: depend.yaml)\n  -c CACHE, --cache CACHE\n                        Binary cache file location; contains inter-run persistent data,\n                        file path must be relative to current working directory (default:\n                        tickle.cache)\n  -l LOG, --log LOG     Log file location; contains runtime messages, file path must be\n                        relative to current working directory (default: tickle.log)\n```\nThe CLI has two additional modes to offline/online:\n\n- The __clean mode__ will delete any files or folders generated during a previous offline or online evaluation.\n- The __version mode__ will print the installed version of tickle.\n\nThe clean mode will only delete generated folders if they are empty after generated files are deleted; i.e. if there are leftover files in the folders, e.g. generated from other processes not within the control of tickle; these files and the host folders are then left untouched.\n\nIf you stick to the default paths and file names, then running tickle should be as simple as:\n```\n$ cd my_workflow\n$ tickle MODE\n```\nWhere `MODE` is one of offline, online, clean or version.\n\n## API\nThe API is accessible in case you wish to run one of the evaluation modes as part of your workflow scripts; rather than spawning a subprocess for tickle.\nThe basic setup is that your script generates an agenda, stores it, and then runs tickle through the api.\n\nFor example running tickle in offline mode could look something like this:\n```Python\nimport tickle.api as tickle_api\nfrom tickle import agenda\n\ndef _my_workflow(target_dir):\n\n    # Paths\n    agenda_path = tickle_api.default_agenda_path(target_dir)\n    depend_path = tickle_api.default_depend_path(target_dir)\n    cache_path = tickle_api.default_cache_path(target_dir)\n    log_path = tickle_api.default_log_path(target_dir)\n\n    # Make and store agenda\n    agenda_data = _make_agenda(target_dir)\n    agenda.store(agenda_path, agenda_data)\n\n    # Run tickle offline\n    success = tickle_api.offline(\n        target_dir,\n        agenda_path,\n        depend_path,\n        cache_path,\n        log_path\n    )\n\n    # Done\n    return success\n```\n\nIn the case of running tickle in online mode, you will need to do so async or concurrently, could look something like this:\n```Python\nimport tickle.api as tickle_api\nfrom tickle import agenda\n\nclass Runner(Thread):\n    def __init__(self):\n        super().__init__()\n        self._func = None\n        self._args = None\n        self._result = None\n\n    def run(self):\n        self._result = self._func(\n            *self._args,\n            **self._kargs\n        )\n\n    def start(self, func, *args, **kargs):\n        self._func = func\n        self._args = args\n        self._kargs = kargs\n        super().start()\n\n    def join(self):\n        super().join()\n        return self._result\n\ndef _my_workflow(target_dir):\n\n    # Paths\n    agenda_path = tickle_api.default_agenda_path(target_dir)\n    depend_path = tickle_api.default_depend_path(target_dir)\n    cache_path = tickle_api.default_cache_path(target_dir)\n    log_path = tickle_api.default_log_path(target_dir)\n\n    # Make and store agenda\n    agenda_data = _make_agenda(target_dir)\n    agenda.store(agenda_path, agenda_data)\n\n    # Run tickle online\n    runner = Runner()\n    runner.start(\n        tickle_api.online,\n        target_dir,\n        agenda_path,\n        depend_path,\n        cache_path,\n        log_path\n    )\n\n    # Do other stuff, e.g. modify agenda\n    ...\n\n    # Setop the runner\n    runner.stop()\n    success = runner.join()\n\n    # Done\n    return success\n```\n\nTODO: Make full documentation for API\n\n## The agenda file\nThe agenda file is a YAML file with the follow grammar:\n```\nprocs:\n  \u003cproc name\u003e:\n    - \u003ccommand word 1\u003e\n    - \u003ccommand word 2\u003e\n    ...\n  ...\nflows:\n  \u003cflow name\u003e:\n    - [ \u003cproc name 1\u003e, \u003cproc name 2\u003e, ... ]\n    ...\n  ...\ntasks:\n  - desc: \u003ctask description\u003e\n    proc: \u003cproc name\u003e\n    args:\n      \u003cparam name\u003e:\n        - \u003carg value 1\u003e\n        - \u003carg value 2\u003e\n        ...\n      ...\n    flows: [ \u003cflow name\u003e, ... ]\n    inputs:\n      - \u003cfile path 1\u003e\n      - \u003cfile path 2\u003e\n      ...\n    outputs:\n      - \u003cfile path 1\u003e\n      - \u003cfile path 2\u003e\n      ...\n  ...\n```\nThe procs section defines a dictionary of procedures.\nA proc is defined with a name and it's implementation is a command as a list of string words.\nA command word is a parameter if it is prefixed with $.\n\nThe flows section defines a list of workflows.\nA workflow is a sequence of stages. A stage is a list of procs, defining which procs are allowed to be evaluated in parallel.\nThis is useful when you have a clear separation in the evaluation order between groups of tasks; you could achieve the same ordering without stages, by having a many-to-many dependency between the task groups that need separating, which however would be costly on the scheduler. So stages were added as both a semantic convenience as well as an optimisation.\n\nMost likely you will only need to define one workflow; however there are cases where you would wish to evaluate two agendas in parallel, or with some overlap. In this case it is more beneficial to keep these two agenda in the same evaluation context; i.e. defining multiple workflows allows you to do that.\n\nThe tasks section defines a list of tasks.\nA task is an instantiation of a proc.\nThe task description is set with the desc field.\nThe proc is selected with the proc field.\nThe proc parameters are given arguments via the args field.\nAn arg is a list of string values to be inserted into the proc's command.\nThe input and output files that the task ranges over are set with the inputs and outputs fields.\n\nYou should think of the agenda file as describing the explicit dependencies between files for a task, e.g. source file to object file in a code project build workflow.\n\nFor more context please check out the examples directory.\n\n## The depend file\nThe depend file is a YAML file with the following grammar:\n```\n\u003cfile path\u003e:\n  - \u003cfile path 1\u003e\n  - \u003cfile path 2\u003e\n  ...\n...\n```\nThe file defines a dictionary of file path to list of file paths, i.e. a file path dependency graph.\n\nYou should think of the depend file as describing the implicit dependencies between files for a task, i.e. uncovered by scanning the content of the files; e.g. source file to header file in a code project build workflow; the source file to header file dependencies change more often during development than the task dependencies mentioned earlier, and as such should be defined in the depend file and not the agenda file.\n\nFor more context please check out the examples directory.\n\n# Examples\nThe example project named hello_world is a simple C++ build example. To build the project in watch/dev mode; run the following command line:\n```\n$ cd tickle/examples/hello_world\n$ tickle online\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoren-n%2Ftickle","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsoren-n%2Ftickle","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoren-n%2Ftickle/lists"}