https://github.com/skypilot-org/mock-train-workflow
Mock training workflow defined as SkyPilot YAMLs
https://github.com/skypilot-org/mock-train-workflow
Last synced: 4 months ago
JSON representation
Mock training workflow defined as SkyPilot YAMLs
- Host: GitHub
- URL: https://github.com/skypilot-org/mock-train-workflow
- Owner: skypilot-org
- License: mit
- Created: 2025-02-25T06:14:24.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-07-23T19:19:29.000Z (11 months ago)
- Last Synced: 2025-07-23T21:53:51.323Z (11 months ago)
- Size: 7.81 KB
- Stars: 0
- Watchers: 3
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Example training workflow on SkyPilot
This repo contains an example training workflow consisting of three simple SkyPilot tasks:
1. `data_preprocessing`: Ingests data and writes it to a bucket.
2. `train`: Trains a model on the data.
3. `eval`: Evaluates the model. Can be optionally run on the same cluster as `train`.
These tasks don't actually run any code, but they demonstrate how to structure a training workflow on SkyPilot.
## Usage
When developing the workflow, you can run the tasks independently using `sky launch`:
```bash
$ sky launch -c data data_preprocessing.yaml
...
```
The train and eval step can be run in a similar way, and the eval step can be run on the same cluster as the train step by setting the `--cluster` flag:
```bash
$ sky launch -c train train.yaml
...
# Run eval on the same cluster as train
$ sky exec train eval.yaml
```