Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/twitter/scoot
Scoot is a distributed task runner, supporting both a proprietary API and Bazel's Remote Execution.
https://github.com/twitter/scoot
Last synced: about 2 months ago
JSON representation
Scoot is a distributed task runner, supporting both a proprietary API and Bazel's Remote Execution.
- Host: GitHub
- URL: https://github.com/twitter/scoot
- Owner: twitter
- License: apache-2.0
- Created: 2015-12-21T22:14:54.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2023-04-10T11:28:45.000Z (over 1 year ago)
- Last Synced: 2024-07-21T17:53:22.258Z (about 2 months ago)
- Language: Go
- Homepage:
- Size: 9.45 MB
- Stars: 349
- Watchers: 14
- Forks: 47
- Open Issues: 8
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-repositories - twitter/scoot - Scoot is a distributed task runner, supporting both a proprietary API and Bazel's Remote Execution. (Go)
README
# Scoot
[](https://godoc.org/github.com/twitter/scoot)
[](https://github.com/twitter/scoot/actions)
[](https://goreportcard.com/report/github.com/twitter/scoot)
[](https://codecov.io/gh/twitter/scoot?branch=master)
Scoot is a distributed task runner.
The core concept of Scoot is the Snapshot, which represents an immutable filesystem state.
Scoot allows for execution of commands against input Snapshots to create new output Snapshots.
### Scoot Components
###### Scheduler
The Scheduler receives and distributes jobs to workers, and maintains state. It is also responsible for serving the Cloud Scoot API for clients.
###### Worker
The Worker (workerserver) receives information about jobs and runs them, and is responsible for all Snapshot-related functionality.
###### Client APIs
* Cloud Scoot API
* Worker API
###### Jobs and Tasks
Scoot work is broken down into independent jobs, which can consist of one or more tasks, which are executed sequentially.
#### Go Version
Scoot is intended to work with go modules. If this repository is on your GOPATH, please set **GO111MODULE=on** when
working with go tools in this repository, and preferably use the Go distribution pinned in the .github/workflows/ci.yml file.
### Example
Setup a scheduler and worker nodes locally:
```sh
go run ./setup-cloud-scoot/main.go --strategy local.local
```
Run a series of randomly generated tests against the local scheduler and workers:
```sh
go run ./scheduler/client/scootcl/main.go smoke_test
```
## Scoot Integration Tests
Scoot has a few tests that exercise varying levels of common usages and workflows.
### Smoketest
Invokes a scoot client directly to run jobs against a local cluster and waits for the
scheduled jobs to complete.
(./scheduler/client/cli/smoketest.go)
### Recoverytest
Invokes a scoot client directly to run jobs against a local cluster, kills the cluster,
attempts to spin up a new one, and waits for the originally scheduled jobs to complete.
(./binaries/recoverytest/main.go)
### Integration
Invokes a scoot and scoot-snapshot-db client via CLI to run a job against a local cluster
and waits for the job to complete
(./binaries/scoot-integration/main.go)
## Scoot Thrift Code
__Thrift Prerequisites__
Install the Thrift tool and golang thrift repository locally using the following section.
__Generating thrift files__
See documentation in thrift definition files for specific generation instructions, or `make thrift`.
# Development Installation Instructions
## Install 3rd party tools:
### Thrift (version == 0.9.3)
macOS:
```sh
brew install [email protected]
```
### Thrift for go:
go modules will take care of this
## Install/Access Scoot Executables and libraries
Install binaries to $GOPATH/bin:
```sh
make install
```