https://github.com/streamdal/schema-publisher
https://github.com/streamdal/schema-publisher
Last synced: about 1 year ago
JSON representation
- Host: GitHub
- URL: https://github.com/streamdal/schema-publisher
- Owner: streamdal
- Created: 2021-06-28T03:01:45.000Z (almost 5 years ago)
- Default Branch: main
- Last Pushed: 2023-02-08T19:50:59.000Z (about 3 years ago)
- Last Synced: 2024-03-15T00:46:10.492Z (about 2 years ago)
- Language: Go
- Size: 30.6 MB
- Stars: 8
- Watchers: 5
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# schema-publisher
`schema-publisher` is a GitHub action for publishing schema updates to
[Streamdal](https://streamdal.com).
This action will allow you to **avoid** having to manually update the schemas
in the Streamdal console every time your schemas change.
**NOTE**: While this repo is intended for use with Github Actions, you can
use the `schema-publisher` tool to perform the schema update in any CI system.
## Example Flow
1. Your schemas are in a Github repo
2. Upon creating a PR in your schema repo, CI runs against the PR and tests the schema
3. Upon merging the PR, CI compiles the schemas and generates a _descriptor set_ file
1. `$ ./protoc --descriptor_set_out=FILE ...`
4. The `schema-publisher` workflow is configured to point to the _descriptor set_
artifact that was created in the previous step
5. `schema-publisher` workflow updates the existing schema in Streamdal with the
new schema and includes the new schema version in the name
**NOTE**: If using `protobuf` with complex schemas, it is best to use
`descriptor_set` for the `input_type`. While Streamdal and the `schema-publisher`
tool support directory-based protobuf schemas, it is possible to run into errors
during schema updates if the source schemas have sophisticated directory
structure.
## Config
| Flag | Required | Description |
|---------------|----------|---------------------------------------------------------------------|
| `api_token` | **YES** | API token used for Streamdal API (dashboard -> account -> security) |
| `schema_id` | **YES** | Schema ID in Streamdal (dashboard -> collection -> schema |
| `input` | **YES** | File or directory with schema |
| `input_type` | No | `descriptor_set` (default) OR `dir` |
| `schema_name` | No | New "friendly" schema name (tip: set to current schema version) |
| `schema_type` | No | `protobuf` (default) OR `avro` |
| `api_address` | No | Override the Streamdal API endpoint |
| `output` | No | Optionally write `dir` artifact (zip file) to a file |
| `debug` | No | Display additional debug output when running workflow |
## Example Github Workflow
Your `.github/workflows/foo.yaml` should look something like this:
```yaml
name: Bump version
on:
push:
branches:
- master
jobs:
build:
runs-on: ubuntu-latest
outputs:
new_tag: ${{ steps.create_tag.outputs.new_tag }}
steps:
- uses: actions/checkout@master
- name: Bump version and push tag
uses: mathieudutour/github-tag-action@v4.5
id: create_tag
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
streamdal_push:
runs-on: ubuntu-latest
name: Update protobuf schema in Streamdal
needs: build
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Push new schemas
uses: batchcorp/schema-publisher@latest
id: publish
with:
api_token: '${{ secrets.STREAMDAL_API_TOKEN }}'
schema_id: '${{ secrets.STREAMDAL_SCHEMA_ID }}'
schema_name: '${{ github.repository }}: ${{ needs.build.outputs.new_tag }}'
input_type: descriptor_set
input: protoset.fds
debug: true
```