{"id":23670999,"url":"https://github.com/wickedshell/clj-taskpool","last_synced_at":"2025-06-17T23:03:07.977Z","repository":{"id":62431511,"uuid":"75519910","full_name":"WickedShell/clj-taskpool","owner":"WickedShell","description":"Simple parallel task pool","archived":false,"fork":false,"pushed_at":"2017-03-10T07:31:01.000Z","size":13,"stargazers_count":1,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-17T02:09:54.887Z","etag":null,"topics":["clojure"],"latest_commit_sha":null,"homepage":null,"language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"epl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/WickedShell.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-12-04T05:45:32.000Z","updated_at":"2017-03-10T07:32:29.000Z","dependencies_parsed_at":"2022-11-01T20:46:52.942Z","dependency_job_id":null,"html_url":"https://github.com/WickedShell/clj-taskpool","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/WickedShell/clj-taskpool","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WickedShell%2Fclj-taskpool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WickedShell%2Fclj-taskpool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WickedShell%2Fclj-taskpool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WickedShell%2Fclj-taskpool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/WickedShell","download_url":"https://codeload.github.com/WickedShell/clj-taskpool/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WickedShell%2Fclj-taskpool/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260453714,"owners_count":23011572,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clojure"],"created_at":"2024-12-29T09:45:48.258Z","updated_at":"2025-06-17T23:03:02.964Z","avatar_url":"https://github.com/WickedShell.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# clj-taskpool\n\n[![Build Status](https://semaphoreci.com/api/v1/wickedshell/clj-taskpool/branches/master/badge.svg)](https://semaphoreci.com/wickedshell/clj-taskpool)\n\nA Clojure library designed to create a simple to use task pool. The level of parallelism specified at the creation of the pool controls how many threads are used to process tasks.\n\n## Installation\n\n`clj-taskpool` is available from [Clojars](https://clojars.org/clj-taskpool):\n\n```\n[clj-taskpool \"0.1.0-SNAPSHOT\"]\n```\n\n## Usage\n\nInclude the namespace with something like:\n\n```clojure\n=\u003e (use 'taskpool.taskpool)\n```\n\n### Create a pool\n\nCreating a pool to submit tasks to is simple, just provide the name of the pool and the number of threads you want to consume tasks. The name is optional and can be omitted, but is helpful when debugging.\n\n```clojure\n    =\u003e (def pool (create-pool \"my-pool\" 10))\n```\n\n### Submitting tasks\n\nTo submit a task, simply call `add-task` with your task pool and the task to be added. Note that tasks are not guaranteed to run in the order you add them to the pool. For example the doseq below adds 20 simple tasks to the pool:\n\n```clojure\n=\u003e (doseq [i (range 0 20)]\n     (add-task pool\n               (fn []\n                 (Thread/sleep (+ (rand-int 1000) 1000))\n                 (println (str (.getName (Thread/currentThread)) \" finished \" i)))))\nmy-pool-8 finished 14\nmy-pool-6 finished 1\nmy-pool-1 finished 6\nmy-pool-9 finished 13\nmy-pool-0 finished 5\nmy-pool-7 finished 0\nmy-pool-5 finished 3\nmy-pool-4 finished 7\nmy-pool-3 finished 2\nmy-pool-2 finished 4\nmy-pool-8 finished 19\nmy-pool-1 finished 17\nmy-pool-9 finished 16\nmy-pool-0 finished 15\nmy-pool-6 finished 18\nmy-pool-2 finished 8\nmy-pool-5 finished 11\nmy-pool-4 finished 10\nmy-pool-7 finished 12\nmy-pool-3 finished 9\n```\n\nNotice that tasks are run as soon as they are submitted, you don't need to prompt the pool to check for any new tasks. Duplicate tasks (either not yet begun, or currently being processed) will be discarded. Once a task has completed it could be added to the pool again.\n\nRather then submitting tasks one at a time, a `set` of additional tasks can be added instead.\n\n### Data processing\n\nA pool can also be made that will always run the same function over the provided tasks. (Basically `pmap` but for a user controllable number of threads, that allows adding more tasks to the pool later)\n\n```\n=\u003e (def worker-pool (create-worker-pool \"my-worker-pool\" 5 #(println (str (.getName (Thread/currentThread)) \" finished \" %))))\n=\u003e (add-task worker-pool (into #{} (range 10)))\nmy-worker-pool-0 finished 0\nmy-worker-pool-2 finished 1\nmy-worker-pool-1 finished 7\nmy-worker-pool-1 finished 2\nmy-worker-pool-1 finished 9\nmy-worker-pool-1 finished 5\nmy-worker-pool-1 finished 8\nmy-worker-pool-3 finished 4\nmy-worker-pool-4 finished 6\nmy-worker-pool-0 finished 3\n\n```\n\n### Remove pending tasks\n\nTasks that haven't been started can be removed from the pool. To remove a single task call `remove-pending-task` with your task pool and the task to be removed. It will return a boolean for if it succeeded in removing the task from the pool without running the task.\n\n```\n=\u003e (remove-pending-task pool my-task)\ntrue\n```\n\nClearing all pending tasks is also supported:\n\n```\n=\u003e (remove-all-pending-tasks pool)\n```\n\n### Terminating the pool\n\nTo shutdown the thread pool you simply call:\n\n```\n=\u003e (terminate-pool pool)\n```\n\nThis will automatically remove all pending tasks, and stop all the threads in the pool after they have completed their current task.\n\n## License\n\nCopyright © 2017 Michael du Breuil\n\nDistributed under the Eclipse Public License either version 1.0 or (at your option) any later version.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwickedshell%2Fclj-taskpool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwickedshell%2Fclj-taskpool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwickedshell%2Fclj-taskpool/lists"}