Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/mozilla/stmocli
St. Mocli is a CLI for linking and deploying queries to STMO
https://github.com/mozilla/stmocli
Last synced: 8 days ago
JSON representation
St. Mocli is a CLI for linking and deploying queries to STMO
- Host: GitHub
- URL: https://github.com/mozilla/stmocli
- Owner: mozilla
- License: mpl-2.0
- Created: 2017-11-15T20:55:23.000Z (almost 7 years ago)
- Default Branch: main
- Last Pushed: 2023-04-10T13:05:08.000Z (over 1 year ago)
- Last Synced: 2024-10-30T00:42:32.749Z (10 days ago)
- Language: Python
- Size: 55.7 KB
- Stars: 19
- Watchers: 43
- Forks: 14
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Introduction
St. Mocli allows you to keep SQL queries in github repositories
and easily deploy to
[re:dash](https://redash.io/).
[Re:dash](https://redash.io/)
is great for quickly creating and sharing simple analyses.
However, sometimes we want to **treat our queries like code**.
In re:dash, it's difficult to get review, track revisions, or collaborate on queries.
# Workflow
St. Mocli is still in early alpha,
so this workflow is going to change.
## Preliminaries
You should have a Redash API key to perform most operations.
You can get one from your
[Redash user settings page](https://sql.telemetry.mozilla.org/users/me).
Then, add something like:
```bash
export REDASH_API_KEY="Tua1aith1ay9roh5thuGhoh6sa3raene"
```
to your ~/.bash_profile, or pass the key to stmocli on the command line like:
```bash
stmocli --redash-api-key Tua1aith1ay9roh5thuGhoh6sa3raene view query.sql
```
Note that `--redash-api-key` has to come before the verb on the command line.
## `init` a directory
**Implemented**!
`stmocli init`
Creates an empty `.stmocli.conf` file in the current directory.
## `track` an existing query
**Implemented**!
`stmocli track `
This command downloads the SQL statements associated with the given redash_id
and saves it in a file with the given name.
The necessary metadata is then added to the config file.
For example, calling
`stmocli track 49741 poc.sql`
would create a file in the current directory called `poc.sql`,
with the following content:
```sql
SELECT
normalized_channel
FROM longitudinal
LIMIT 10
```
Assuming this is the first query being tracked, `.stmocli.conf` would look like this:
```json
{
"poc.sql": {
"query_id": 49741,
"data_source_id": ,
"name": ,
"description": ,
"schedule": ,
"options":
}
}
```
## `pull` a linked query
**Implemented!**
`stmocli pull [...]`
Pulls the current SQL statements and metadata from re:dash for the given query files.
If no file name is specified, pull data for all queries.
This will **overwrite local data**.
Be sure to use version control.
`` must be a key in the dictionary stored in `.stmocli.conf`
## `push` a query
**Implemented!**
`stmocli push [...]`
Pushes the current SQL statements and metadata to re:dash for the given query file.
You can specify one or more query files to be pushed, these must be keys in the
dictionary stored in `.stmocli.conf`. If no file names are specified, all SQL
statements are pushed.
# Roadmap
## Push-only and Automatic deploys
This tool assumes no edits happen in re:dash, which is a bad assumption.
Edits made in re:dash get overwritten if you `push` without `pull`ing first.
Ideally, there would be a Mocli-user in re:dash that owns all Mocli queries.
This would ensure all queries controlled by Mocli cannot be edited in re:dash.
We could then remove the `pull` command, and this tool becomes `push`-only.
From there we can have a scheduled job (hourly?) that pushes master to STMO.
Maybe we add a git-hook that pushes master on commit. Seamless.
## `preview` a query
Users will need to upload queries to a temporary re:dash query to preview the results.
This should be easy to do with a `preview` command.
It may also be useful to execute queries against presto directly.
## `start` a new query
Currently, St. Mocli can only track existing queries.
We should add a `start` command that will make it easy to start queries from the cli.