Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/vitorenesduarte/tricks
Run experiments effortlessly on top of Kubernetes
https://github.com/vitorenesduarte/tricks
experiments kubernetes kubernetes-deployment tricks
Last synced: about 2 months ago
JSON representation
Run experiments effortlessly on top of Kubernetes
- Host: GitHub
- URL: https://github.com/vitorenesduarte/tricks
- Owner: vitorenesduarte
- License: apache-2.0
- Created: 2017-12-29T11:15:41.000Z (almost 7 years ago)
- Default Branch: main
- Last Pushed: 2022-05-20T20:52:07.000Z (over 2 years ago)
- Last Synced: 2024-10-13T18:44:20.100Z (2 months ago)
- Topics: experiments, kubernetes, kubernetes-deployment, tricks
- Language: Erlang
- Homepage:
- Size: 2.18 MB
- Stars: 23
- Watchers: 4
- Forks: 2
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://travis-ci.org/vitorenesduarte/tricks)
[](https://coveralls.io/github/vitorenesduarte/tricks?branch=master)
# Tricks
__Running an experiment should be as easy as describing it in an YAML file.__
----------
### Tricks presentation
[Here](https://docs.google.com/presentation/d/1uP6eM9eA7_g_B6Ap_1SLPsBOAPypLmLIX-TPf7GbOX4/edit#slide=id.gc6f980f91_0_0)'s a presentation about Tricks.
### About Tricks
A Tricks entry (TE) is a set of replicas/pods.
An experiment is one or more TE's.
#### Features
- [x] Register occurrence of events. Each event has a counter associated.
- [x] Implicit events (`start` and `stop` of each pod in a TE)
- [x] Subscribe to events
- can be used to implement synchronization barrier
(e.g. make sure all clients start at the same time)
- [x] Workflow (e.g. only start a given TE once event X counter is Y)
- [ ] Sequences of experiments
- [x] Pod discovery
- [ ] Log aggregation
- [ ] Plotting from logs (e.g. latency/throughput, CDF, bar, line)
- [ ] Detect coordination omission from logs
- [ ] Fault injection
- (when) could be defined in the same way workflow is
- (what) maybe provide some sort of pod selector for process faults,
and link selector for network faults
- [ ] Federation support (run across multiple Kubernetes clusters)
- [ ] Spot/preemptible instances support (if an instance is killed, the experiment is restarted)
### Example
In this example, event `client1_stop` and `client2_stop`
are implicit events,
while `server-ready` is an event registered by replicas
in the `server` TE.
```yaml
apiVersion: v1
experiment:
- tag: server
image: vitorenesduarte/tricks-example
replicas: 3
env:
- name: TYPE
value: loop
workflow:
stop:
name: client2_stop
value: 6
- tag: client1
image: vitorenesduarte/tricks-example
replicas: 3
env:
- name: TYPE
value: loop
- name: SECONDS
value: 5
workflow:
start:
name: server-ready
value: 3
- tag: client2
image: vitorenesduarte/tricks-example
replicas: 6
env:
- name: TYPE
value: loop
- name: SECONDS
value: 10
workflow:
start:
name: client1_stop
value: 3
```
#### Other examples
- [examples/hello-world.yaml](examples/hello-world.yaml)
- [examples/implicit-events.yaml](examples/implicit-events.yaml)
- [examples/explicit-events.yaml](examples/explicit-events.yaml)
### Running Tricks
Assuming there's a Kubernetes cluster running:
```bash
$ tricks start
$ tricks logs
```
And then in another terminal, run one of the examples:
```bash
$ tricks exp examples/explicit-events.yaml
```
### Environment variables
Configuration of pods is achieved through environment variables.
Some variable names are reserved and always defined in every pod:
- [x] `TAG`: from configuration file
- [x] `REPLICAS`: from configuration file
- [x] `EXP_ID`: an experiment identifier generated by Tricks
- [x] `POD_ID`: unique (no other pod with the same `TAG` and `EXP_ID`
has the same id)
- [x] `POD_IP`: pod IP from Kubernetes
- [x] `TRICKS_IP`: pod IP of Tricks
- [x] `TRICKS_PORT`: port of Tricks
### Drivers API
Replicas in experiments can be written in any language,
as long as there's a driver available.
Drivers open a socket (`TRICKS_IP` and `TRICKS_PORT`)
and talk with Tricks using the following API
(__[DT]__ is used if it's a message from a Driver to Tricks,
or with __[TD]__ otherwise).
- [x] Register events __[DT]__
```json
{
"expId": "123456",
"type": "event",
"eventName": "connected",
}
```
- [x] Subscription of events __[DT]__
```json
{
"expId": "123456",
"type": "subscription",
"eventName": "connected",
"value": 10
}
```
- [x] Notification of events __[TD]__
```json
{
"expId": "123456",
"type": "notification",
"eventName": "connected",
"value": 10
}
```
- [x] Pod discovery __[DT]__
```json
{
"expId": "123456",
"type": "discovery",
"tag": "server",
"min": 2
}
```
The argument `min` in the previous
message is optional
(default value is 0).
Tricks will only reply when
it has at least `min` pods
(sending all, even if more than `min`).
- [x] Pod discovery __[TD]__
```json
{
"expId": "123456",
"type": "pods",
"tag": "server",
"pods": [
{
"id": 1,
"ip": "10.12.13.15"
},
{
"id": 2,
"ip": "10.12.13.16"
},
{
"id": 3,
"ip": "10.12.13.17"
}
]
}
```
### Sequences of experiments
Typically we want, not to run a single experiment,
but several, and in the end compare metrics
collected.
In order to support sequences of experiments,
configuration values in the YAML file
can be replaced by variables
(e.g. `$var`),
that are defined in a list of
configurations.
Each configuration will be used
to create an experiment.
An example
with two variables (`$op_number` and `$client_number`)
that
are used to define number of replicas,
workflow configuration and
environment variables:
```bash
apiVersion: v1
config:
- $op_number: 100
$client_number: 3
- $op_number: 200
$client_number: 3
- $op_number: 100
$client_number: 6
- $op_number: 200
$client_number: 6
experiment:
- tag: server
image: vitorenesduarte/tricks-example
replicas: 3
env:
- name: TYPE
value: server
workflow:
stop:
name: client_stop
value: $client_number
- tag: client
image: vitorenesduarte/tricks-example
replicas: $client_number
env:
- name: TYPE
value: client
- name: OP_NUMBER
value: $op_number
workflow:
start:
name: server_start
value: 3
```