https://github.com/feldroop/static_task_scheduling
Implementations of different static task scheduling algorithms to evaluate a proposed MILP based model for this problem.
https://github.com/feldroop/static_task_scheduling
scheduling-algorithms static-scheduling workflows
Last synced: 2 months ago
JSON representation
Implementations of different static task scheduling algorithms to evaluate a proposed MILP based model for this problem.
- Host: GitHub
- URL: https://github.com/feldroop/static_task_scheduling
- Owner: feldroop
- License: mit
- Created: 2022-03-12T17:25:19.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2022-06-08T06:38:07.000Z (about 4 years ago)
- Last Synced: 2026-01-02T13:54:16.144Z (6 months ago)
- Topics: scheduling-algorithms, static-scheduling, workflows
- Language: C++
- Homepage:
- Size: 219 KB
- Stars: 3
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# static_task_scheduling
[build_badge]: https://img.shields.io/github/workflow/status/Felix-Droop/static_task_scheduling/CMake?style=flat-square
[build_workflow]: https://github.com/Felix-Droop/static_task_scheduling/actions/workflows/cmake.yml
[license_badge]: https://img.shields.io/github/license/Felix-Droop/static_task_scheduling?style=flat-square
[license_workflow]: https://github.com/Felix-Droop/static_task_scheduling/blob/main/LICENSE
[![Build][build_badge]][build_workflow]
[![License][license_badge]][license_workflow]
Implementations of different static task scheduling algorithms to internally evaluate a proposed MILP based model for this problem.
This is NOT intended for general purpose usage and there is no guarantee for
robustness or any expected program behavior.
## Implemented algorithms
* HEFT and CPOP ([publication](https://ieeexplore.ieee.org/document/993206))
* Modifications of the RBCA/DBCA algorithms ([publication](https://www.sciencedirect.com/science/article/abs/pii/S095070512030263X))
* TDCA ([publication](https://ieeexplore.ieee.org/document/8399533))
## Setup
* Requires CMake and a C++20 capable compiler with ranges support (e.g. `g++-10` or newer)
* Example of build and run on linux:
```
git clone --recurse-submodules https://github.com/Felix-Droop/static_task_scheduling
mkdir build && cd build
cmake ../static_task_scheduling -DCMAKE_BUILD_TYPE=Release
make
./bin/static_task_scheduling --help
```
## Usage
Look at the help menu for a detailed description of all possible flags. The main
differences are in the ways how task dependencies are read or generated for the
input workflow.
```
SYNOPSIS
static_task_scheduling -c -t [-p ] [-d
] [-a ] [-s ] [-o
] [-v] [-m]
OPTIONS
Input
-c, --cluster
File in .csv format that describes the cluster architecture. It should contain
exactly the fields bandwidth, performance, memory and num_cores.
-t, --tasks
File in .csv format that describes the tasks of the workflow. It should contain
exactly the fields workload, input_data_size, output_data_size, memory and
cardinality.
-p, --topology
Desired topology of the workflow. If no dependency file is given, the
dependencies will be inferred from the task bags using this configuration. Must
be one of: epigenome, cybershake, ligo or montage. For the montage workflow
topology, a dependency file must be given.
-d, --dependencies
File that contains the dependencies for the workflow tasks. Can either be in csv
format or in xml format. A csv file should contain exactly the fields from_id
and to_id. An xml file should model the schema at
https://pegasus.isi.edu/schema/dax-2.1.xsd. For files in xml format it is
assumed that the jobs in the file are specified in a level order of the DAG
implied by the task bags.
-a, --assignment
File in .csv format that describes an assignment of tasks to nodes. It should
contain exactly the fields task_number, node_number and is_assigned. *_number
fields are 1-based while *_id fields are 0-based.
-s, --select-algorithm
If this is given, only the selected algorithm is executed. Must be one of: heft,
cpop, rbca, dbca or none. If 'none' is given, no algorithm is executed.
Output
-o, --output
If given, the verbose output of this program is written to this file as plain
text.
-v, --verbose
If given, all metrics and the full solution are printed to the command line.
-m, --use-memory-requirements
If given, tasks are only scheduled onto cluster nodes with sufficient memory.
This is not part of the original HEFT and CPOP and is deactivated by default.
```
### Examples
* Get dependencies from a .csv file with `-d` flag:
```
./static_task_scheduling -c cluster.csv -t task_bags.csv -d dependencies.csv
```
* Get dependencies from an .xml file with `-d` flag:
```
./static_task_scheduling -c cluster.csv -t task_bags.csv -d workflow_spec.xml
```
* Infer dependencies for a given workflow topology with the `-p` flag:
```
./static_task_scheduling -c cluster.csv -t epigenome_bags.csv -p epigenome
```
This works for `epigenome`, `cybershake` and `ligo` workflows.
* Get complex dependencies from an .xml file with additional processing with `-d` and `-p`:
```
./static_task_scheduling -c cluster.csv -t epigenome_bags.csv -d montage.xml -p montage
```
This is currently only needed for `montage` workflows.
* Write verbose output to command line with `-v` and write the same verbose output to a file with `-o`:
```
./static_task_scheduling -c cluster.csv -t task_bags.csv -d dependencies.csv -v -o output.txt
```
* Add the schedule for a precomputed assignment to the output with `-a`:
```
./static_task_scheduling -c cluster.csv -t task_bags.csv -d dependencies.csv -a assignment.csv
```
* Only execute the HEFT algorithm with `-s`:
```
./static_task_scheduling -c cluster.csv -t task_bags.csv -d dependencies.csv -s heft
```
* Only create the schedule for a precomputed assignment with `-s` and `-a`:
```
./static_task_scheduling -c cluster.csv -t task_bags.csv -d dependencies.csv -a assignment.csv -s none
```