Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/shopify/goqueuesim
User queueing simulator supporting diverse client config behaviours via Go coroutines
https://github.com/shopify/goqueuesim
go queue-simulator queues redis redis-lua-script simulation
Last synced: about 2 months ago
JSON representation
User queueing simulator supporting diverse client config behaviours via Go coroutines
- Host: GitHub
- URL: https://github.com/shopify/goqueuesim
- Owner: Shopify
- License: mit
- Archived: true
- Created: 2022-09-22T20:32:19.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-03-21T23:03:37.000Z (almost 2 years ago)
- Last Synced: 2024-11-27T17:59:16.292Z (about 2 months ago)
- Topics: go, queue-simulator, queues, redis, redis-lua-script, simulation
- Language: Go
- Homepage:
- Size: 67.4 KB
- Stars: 9
- Watchers: 158
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: .github/CODEOWNERS
- Authors: AUTHORS
Awesome Lists containing this project
README
# Queue Simulator
# goqueuesim
## Table of Contents
- [Introduction](#introduction)
- [Usage](#-usage)
- [Notable Feature Gaps](#notable-feature-gaps)
- [Contributing](#contributing)
## Introduction
Simulator built at Shopify originally purposed to prototype different Checkout Queue algorithms.
Open-sourced at similar low-fidelity to its prototypical origins but readily-actionable for improved usability.
### 🔗  Requirements
First, install project Go dependencies to your local environment:
```bash
go mod download
```
If you plan to run with [lua-driven](https://redis.io/commands/evalsha/) queues in Redis (ie. `queue_type = "lua_driven_bins_queue"` in [config.go](cmd/config.go)), you'll need to [install Redis](https://redis.io/docs/getting-started/installation/) and [start a server](https://redis.io/docs/getting-started/#exploring-redis-with-the-cli) in the background.
The above step is not necessary for queue types other than `lua_driven_bins_queue`.
#### **Dashboards**
For dashboard statistics, this project hopes to eventually leverage dockerized [Prometheus](https://prometheus.io/) and [Grafana](https://grafana.com/).
However, for the moment, its statistics are built around [Datadog](https://www.datadoghq.com/) so you'll need a [Datadog API Key](https://docs.datadoghq.com/account_management/api-app-keys/).
Once you have one ready, create a `.env` file with the following:
```
DD_API_KEY=XXXX
DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true
```
and then use [Docker](https://www.docker.com/) to kickoff the Datadog agent:
```bash
docker-compose up
make
```
## 📖 Usage
For now, project usage is primarily driven by a [Makefile](https://www.gnu.org/software/make/manual/make.html) with the following commands:
| Make Task | Description |
| :-------- | :--------------------------------------------------------------------------------- |
| `default` | Compile the project binary executable & run it |
| `run` | Run the last compiled project binary executable |
| `clean` | Remove the last compiled project binary executable |
| `test` | Run tests |
Running a simulation is thus as simple as executing `make` (build & execute) or `make run` (execute last build).
#### **Experiment Configuration**
Experiment configuration (e.g. algorithm used, client behaviour, etc.) can be modified under [cmd/goqueuesim/config.go](cmd/goqueuesim/config.go). Available parameters are documented in that file.
Additionally, client behaviour can be specified via JSON under [config/simulation/client_distributions/](config/simulation/client_distributions/).
A simple example config with 3 different client types might look like:
```json
{
"representation_percent": 0.20,
"client_type": "routinely_polling_client",
"humanized_label": "fast_poller",
"obeys_server_poll_after": false,
"max_initial_delay_ms": 20000,
"max_network_jitter_ms": 100,
"custom_int_properties": {
"dflt_poll_interval_seconds": 1
}
},
{
"representation_percent": 0.60,
"client_type": "routinely_polling_client",
"humanized_label": "strictly_obedient_poller",
"obeys_server_poll_after": true,
"max_initial_delay_ms": 20000,
"max_network_jitter_ms": 450
},
{
"representation_percent": 0.20,
"client_type": "fully_disappearing_client",
"humanized_label": "immediately_exiting_client",
"obeys_server_poll_after": false,
"max_initial_delay_ms": 20000,
"max_network_jitter_ms": 1,
"custom_int_properties": {
"polls_before_disappearing": 1
}
}
```
See [internal/client/impl/](internal/client/impl/) for already-implemented example `client_type`s.
## Notable Feature Gaps
The main notable feature/usability gaps that have yet to be implemented are:
- Migrating our experiment statistics to leverage dockerized [Prometheus](https://prometheus.io/) and [Grafana](https://grafana.com/) rather than [Datadog](https://www.datadoghq.com/)
- Simplifying our build dependencies (incl. [Redis](https://redis.com/)) to a single containerized [Docker](https://www.docker.com/) image
## Contributing
Please refer to the [Contributing document](CONTRIBUTING.md) if you are interested in contributing to goqueuesim!