{"id":16480670,"url":"https://github.com/cdluminate/tasque","last_synced_at":"2025-03-23T11:33:06.374Z","repository":{"id":57476812,"uuid":"129078926","full_name":"cdluminate/tasque","owner":"cdluminate","description":"Zero-Config Single-Node Workload Manager","archived":false,"fork":false,"pushed_at":"2021-01-05T13:39:19.000Z","size":256,"stargazers_count":17,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-18T20:13:16.374Z","etag":null,"topics":["atd","condor","job-control","pbs","priority","pypi","python","python3","schedule","torque"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cdluminate.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-04-11T10:49:10.000Z","updated_at":"2025-01-15T08:22:09.000Z","dependencies_parsed_at":"2022-09-12T14:33:03.406Z","dependency_job_id":null,"html_url":"https://github.com/cdluminate/tasque","commit_stats":null,"previous_names":["cdluminate/tq"],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cdluminate%2Ftasque","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cdluminate%2Ftasque/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cdluminate%2Ftasque/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cdluminate%2Ftasque/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cdluminate","download_url":"https://codeload.github.com/cdluminate/tasque/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245097159,"owners_count":20560311,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["atd","condor","job-control","pbs","priority","pypi","python","python3","schedule","torque"],"created_at":"2024-10-11T13:04:57.825Z","updated_at":"2025-03-23T11:33:05.983Z","avatar_url":"https://github.com/cdluminate.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Tasque -- Zero-Config Single-Node Workload Manager (WORK IN PROGRESS)\n===\n\n[![Latest Version](https://pypip.in/version/tq1/badge.svg)](https://pypi.python.org/pypi/tq1/)\n\n\u003c!-- ![tqls1](tqls1.png) --\u003e\n\nTasque (Task Queue) is a simple workload manager for single-node usage that can\nbe used out-of-the-box. It is resource-aware (e.g. CPU, Memory, GPU, Video Memory),\nand can automatically schedule the submitted jobs in a sensible way. It is much\nmore light-weight compared to cluster workload managers such as Slurm and PBS,\nwhile being much smarter than a casually rushed script using tmux, screen, or\nalike. Tasque is written by the author for scheduling batches of machine\nlearning experiments (e.g. caffe, pytorch, tensorflow).\n\nTasque has the following characteristics:\n1. Submitted jobs (command lines) will be automatically scheduled to run when\nthere is enough resource to do so.\n2. The daemon is resource-aware, namely it is able to plan the usage of either\nCPU, Memory, GPU, or Video memory. When resource is aboundant, tasks may be\nscheduled to run in parallel.\n3. The default behavior is to execute the given commands in the FIFO order.\n4. Tasks with high priority values will be scheduled to run prior to the rest.\n5. The queue is stored in an SQLite3 database, and will not be lost in case of\npowerloss.\n6. Users can assign text annotations with the tasks in database.\n7. Requires no configuration and can be used out-of-the-box.\n\n## Example\n\n\u003c!--\nTQ can be used to deal with some commands in an async manner. e.g.\n```\n$ tq -- git push  # Doesn't block. Have it done in async.\n$ vim mycode.py\n```\n\nTQ can be used to manage a series of computation experiments, such as\ndeep learning experiments, e.g.\n```\n$ tq r5 -- caffe train -solver net1forfun.prototxt\n$ tq r5 -- caffe train -solver net2forfun.prototxt\n$ tq 1 5 -- python3 train.py --lr 1e-2\n$ tq 1 5 -- python3 train.py --lr 1e-3\n$ tq 1 5 -- python3 train.py --lr 1e-4\n$ tq 1 5 -- python3 train.py --lr 1e-5\n$ tq p10 -- python3 important_train.py\n```\nOne can just put many computation tasks in the queue, and TQ will smartly\nschedule these experiments according to the given priority and resource\noccupancy parameters.\n--\u003e\n\n## Installation\n\nThis tools is available on Pypi. Just issue the following command:\n```\npip3 install tq1\n```\nNote that some new language features may be used in the code.\nThere is no plan to support older versions of python3.\nHence `python3 \u003e= 3.8` is recommended.\n\nIn case you want to install it from source:\n```\npython3 setup.py install\n```\n\n## Usage\n\n\u003c!--\n```\nUsage: tq ACTION [COMMAND_ARGS]\n       tq [P R] -- TASK\n\nAvailable Actions:\n    start      start TQ's daemon\n    stop       stop TQ's daemon\n    log        dump log to screen\n    ls         fancy print of task queue\n    db         print database content to screen\n    rm \u003cID\u003e    remove task with specified id, see ID with tq ls\n    clean      remove finished tasks from queue\n    purge      remove log file and sqlite3 db file\n\nApending Task:\n    -- TASK        append TASK to the queue\n    p\u003cP\u003e -- TASK   append TASK with priority P to the queue\n    r\u003cR\u003e -- TASK   append TASK with resource occupancy R to the queue\n    P R -- TASK    append TASK with priority P and estimated occupancy R\n                   int P default  0 range [INT_MIN, INT_MAX], large=important\n                   int R detault 10 range [1,       10],      large=consuming\n```\n--\u003e\n\n## Copyright and License\n\n```\nCopyright (C) 2016-2021 Mo Zhou \u003clumin@debian.org\u003e\nReleased under the MIT/Expat License.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcdluminate%2Ftasque","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcdluminate%2Ftasque","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcdluminate%2Ftasque/lists"}