{"id":27283933,"url":"https://github.com/zjiayao/pamuretas","last_synced_at":"2025-04-11T18:47:09.283Z","repository":{"id":91150094,"uuid":"113332412","full_name":"zjiayao/Pamuretas","owner":"zjiayao","description":"Parallel Multiple-Dependency Resource-Critical Task Scheduling","archived":false,"fork":false,"pushed_at":"2018-09-17T07:50:55.000Z","size":12530,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2023-02-28T08:31:00.519Z","etag":null,"topics":["multithreading","parallel-computing","pthread","scheduling-algorithms","semaphore"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zjiayao.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-12-06T15:21:46.000Z","updated_at":"2020-09-07T07:06:23.000Z","dependencies_parsed_at":"2023-07-24T00:45:58.205Z","dependency_job_id":null,"html_url":"https://github.com/zjiayao/Pamuretas","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjiayao%2FPamuretas","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjiayao%2FPamuretas/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjiayao%2FPamuretas/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zjiayao%2FPamuretas/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zjiayao","download_url":"https://codeload.github.com/zjiayao/Pamuretas/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248462256,"owners_count":21107834,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["multithreading","parallel-computing","pthread","scheduling-algorithms","semaphore"],"created_at":"2025-04-11T18:47:04.216Z","updated_at":"2025-04-11T18:47:09.268Z","avatar_url":"https://github.com/zjiayao.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pamuretas\n\n[![License](https://img.shields.io/badge/License-GPL%20v3-blue.svg)](https://github.com/zjiayao/Pamuretas/blob/master/LICENSE)\n[![Travis\nStatus](https://travis-ci.org/zjiayao/Pamuretas.svg?branch=master)](https://travis-ci.org/zjiayao/Pamuretas/)\n[![Chat on\nGitter](https://badges.gitter.im/zjiayao/Pamuretas.svg)](https://gitter.im/zjiayao/Pamuretas/)\n\n\n![demo](figs/demo.gif)\n\n*100 workers produce 10 cars given a reasonable amount of storage spaces in ~17.14 seconds.*\n\n**This project has a companion technical report, [learn\nmore](https://jiayao-zhang.com/doc/pamuretas.pdf).**\n\n## Introduction\n\n\n*Pamuretas* (**Pa**rallel **Mu**lti-Dependency **Re**source-Critical **Ta**sk **S**cheduler)\nis a dynamic job allocation and dispatch system that is designed to be\nrobust, flexible and efficient. Currently, it can only solve one particular\ninstance of this class of problems, but it does it very well.\n\n\n### Problem Formulation\n\nWe formulate our problem as a variant to the classical costumer-producer problem\nwith Resource Constrained Project Scheduling Problem (RCPSP) flavour.\nConcretely, given the number of workers and the goal of production, we wish to\ncomplete production with a pre-specified amount of *storage space* such that\nany component listed below occupies one space once produced; the space is\nfreed when it is fed into another production task as raw material.\nWe further assume the goal of production (i.e., cars) does not\noccupy any spaces upon completion.\n\n\u003ctable\u003e\n\u003ctr\u003e\u003cth\u003eDependency DAG\u003c/th\u003e\u003cth\u003eConstraints\u003c/th\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003ctd\u003e\n\n\n![DAG](figs/dag.png)\n\n\n\u003c/td\u003e\u003ctd\u003e\n\n| Task     | Time/sec | Amount |\n| :-:      | :-:      | :-:    |\n| Skeleton | 5        | 1      |\n| Engine   | 4        | 1      |\n| Chassis  | 3        | 1      |\n| Body     | 4        | 1      |\n| Window   | 1        | 7      |\n| Tire     | 2        | 4      |\n| Battery  | 3        | 1      |\n| Car      | 6        | --     |\n\n\u003c/td\u003e\u003c/tr\u003e \u003c/table\u003e\n\n### Features\n\nPamuretas has the following charming characteristics:\n\n#### Parallelism\n\nPamuretas assumes a customized input of number of maximum threads\nthat may run concurrently. It leverages the abundant working resource if and when\npossible but contemplate discreetly when facing limited resource to\nguarantee a stable performance.\n\n#### Scalability\n\nPamuretas is shipped with a dedicated scheduler and two dispatchers, through\nwhose delegation and feedback, Pamuretas is able to perform decently well\nno matter what task it is facing. Under most scenarios, the dynamic task\nscheduling results a nearly optimal solution.\n\n#### Robustness\n\nPamuretas has two built-in systems for deadlock prevention and\nhandling/recovery;\nits built-in signaling mechanism enables the feedback between dispatchers\nsuch that both competition and hungers between threads are minimized.\n    \n\n### Architecture\n\nPamuretas adapts the master-slave paradigm, and models the sets of individual tasks\nas two FIFO queues, where they are enqueued in topological order of the DAG.\n\n\n\u003cp align=\"center\"\u003e\n\t\u003cimg alt=\"System Architecture\" src=\"figs/arch.png\" width=auto\u003e\n \u003cbr\u003eSystem Architecture\n\u003c/p\u003e\n\u003cbr\u003e\n\n\u003cp align=\"center\"\u003e\n\t\u003cimg alt=\"Work Flow\" src=\"figs/loop.png\" width=auto\u003e\n \u003cbr\u003eWork Flow\n\u003c/p\u003e\n\nThe core of Pamuretas lies on its ability of delegating between the key\ncomponents, i.e., one scheduler and two dispatchers. The main working\nflow is outlined above. As we shall see in the sequel, these are vital\nto Pamuretas' performance and robustness.\n\n## Usage\n\n### Dependencies\n\n**Important Note for MacOS Users:** the dependency `semaphore` has\ndeprecated in the `clang` distribution. Please do NOT run it, otherwise\nunexpected error may occur.\n\nPamuretas is written in pure `c`, with `pthread`, `semaphore` and `omp`\nlibraries linked.\n\n### Installation\n\nFirst, clone this repo via `git`:\n\n    $ git clone https://github.com/zjiayao/Pamuretas\n\nThen we may build from the source:\n\n    $ cd pamuretas \u0026\u0026 make\n\nBy default, we turn off the `debug` mode, which makes Pamuretas\nless verbose while working, to turn it on, add the following\nargument when `make`:\n\n    $ make debug=on\n\nThis would build an executable `pamuretas`.\n\n### Invoking from CLI\n\nPamuretas takes three parameters:\n\n    $ ./pamuretas [goals] [spaces] [workers]\n\nwhere all of them should be positive integers. Pamuretas\ndoes not allow input configurations which are insolvable:\n\n    $ ./pamuretas 1 8 1\n    [Fatal] Single worker requires at least 13 space but only 8 given, abort\n\n### Utilities\n\nWe also provide a handful of utilities for easy-benchmarking. These\nare included in the `benchmarking` and `results` folder:\na `python` plotter, a `c` test-case enumerator and several `shell`\nscripts for prettifying the result. Please refer to the comments therein\nfor usage.\n\n## Adversarial Examples (Defended)\n\nHere we attach a series of screencasts exhibiting\nthe robustness of Pamuretas under critical scenarios.\n\n\n### Single Worker, Thirteen Space\n\nThis is a very critical scenario, a slight misgiving would\ntrap a less-carefully implemented algorithm.\n\n![1-13-1](figs/screencasts/1-13-1.gif)\n\n*1 worker produces 1 car given critical storage spaces in ~40.02 seconds.*\n\n\n### Single Space, Huge Working Force\n\nThis posts two difficulties: how to optimally use\nthe only one space; and how to avoid workers from\ncompetiting with each other.\n\n![2-1-100](figs/screencasts/2-1-100.gif)\n\n*100 workers produce 2 cars given only one storage spaces in ~38.20 seconds.*\n\n\n## Benchmarking Results\n\nPamuretas has been thoroughly tested against over five\nthousand configurations. This indeed yields some interesting\nbenchmarking results.\n\n\n### Baseline\n\nWe first find out the time needed for one production goal\nwith sufficient resource:\n\n![1-15-15](figs/screencasts/1-15-15.gif)\n\nwhich is about 15 seconds.\n\n### Single/Multiple Production Goals\n\n\u003ctable\u003e\n\u003ctr\u003e\u003cth colspan=\"2\"\u003eSurface of Production Time against Worker-Space\u003c/th\u003e\u003c/tr\u003e\n\n\u003ctr\u003e\u003ctd\u003e\n\n![single_car_surface](figs/single_car_surface.png)\n\n\u003c/td\u003e\u003ctd\u003e\n\n![eight_cars_surface](figs/eight_cars_surface.png)\n\n\u003c/td\u003e\u003c/tr\u003e \u003c/table\u003e\n\nWe note Pamuretas is able to utilize the abundant\nworking force and storage space rather quickly;\nit also does not suffer from deadlock and alike\nwhen very limited spaces are provided. The results\nare nearly-optimal compared to the baseline computed\nfrom last section.\n\nWhen we scale up the production task, we note\nthe contours are nearly identical -- with\none peak appearing at low space region. This\nsuggested limited space posts more threats to Pamuretas.\n\nFurthermore, at abundant resource region, the running\ntime of Pamuretas is near optimal and remains stable\n(the plateau near the reader).\n\n\n### Optimal Performance Analysis\n\nWe proceed to consider the throughput and scalability of Pamuretas.\n\n\n\u003ctable\u003e\n\u003ctr\u003e\u003cth\u003eMaximum Throughput (cars/sec)\u003c/th\u003e\u003cth\u003eProduction Time with Unlimited\nResource\u003c/th\u003e\u003c/tr\u003e\n\n\u003ctr\u003e\u003ctd\u003e\n\n![throughput](figs/throughput.png)\n\n\u003c/td\u003e\u003ctd\u003e\n\n![scalability](figs/scalability.png)\n\n\u003c/td\u003e\u003c/tr\u003e \u003c/table\u003e\n\nFirst noted the plotter automatically extrapolates\nthe values at the boundary, hence resulting the slope\nnear reader. This should be ignored though. The increasing\nalong the diagonal of worker-space plane implies a good\nscalability. Under the scenario where sufficient resources are available,\nPamuretas is able to finish the production near the optimal\ntime, i.e., around 15 seconds.\n\n## Limitation and Future Work\n\nCurrently, the Worker Pool is implemented as a\nvannila array. This may become the bottleneck of\nthe scalability, if we have tens of or thousands of\nworkers, say. This may be solved by implementing\na tree structure for fast indexing.\n\n## Acknowledgment and Disclaimer\n\nPart of this project has been submitted in partial fulfillment\nfor the course *Operating Systems* offered by HKU, 2017-18.\nWe thank the teaching staff for providing\nthis motivating problem.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzjiayao%2Fpamuretas","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzjiayao%2Fpamuretas","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzjiayao%2Fpamuretas/lists"}