Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/sbdchd/squawk

🐘 linter for PostgreSQL, focused on migrations
https://github.com/sbdchd/squawk

linter postgres postgresql rust sql

Last synced: 6 days ago
JSON representation

🐘 linter for PostgreSQL, focused on migrations

Awesome Lists containing this project

README 

        
# squawk [![npm](https://img.shields.io/npm/v/squawk-cli)](https://www.npmjs.com/package/squawk-cli)



> linter for Postgres migrations



[quick start](https://squawkhq.com/docs/) | [rules documentation](https://squawkhq.com/docs/rules) | [github action](https://github.com/sbdchd/squawk-action) | [diy github integration](https://squawkhq.com/docs/github_app)



## Why?



Prevent unexpected downtime caused by database migrations and encourage best

practices around Postgres schemas and SQL.



Also it seemed like a nice project to spend more time with Rust.



## Install



```shell

npm install -g squawk-cli



# or via PYPI

pip install squawk-cli



# or install binaries directly via the releases page

https://github.com/sbdchd/squawk/releases

```



## Usage



```shell

❯ squawk example.sql

example.sql:2:1: warning: prefer-text-field



   2 | --

   3 | -- Create model Bar

   4 | --

   5 | CREATE TABLE "core_bar" (

   6 |     "id" serial NOT NULL PRIMARY KEY,

   7 |     "alpha" varchar(100) NOT NULL

   8 | );



  note: Changing the size of a varchar field requires an ACCESS EXCLUSIVE lock.

  help: Use a text field with a check constraint.



example.sql:9:2: warning: require-concurrent-index-creation



   9 |

  10 | CREATE INDEX "field_name_idx" ON "table_name" ("field_name");



  note: Creating an index blocks writes.

  note: Create the index CONCURRENTLY.



example.sql:11:2: warning: disallowed-unique-constraint



  11 |

  12 | ALTER TABLE table_name ADD CONSTRAINT field_name_constraint UNIQUE (field_name);



  note: Adding a UNIQUE constraint requires an ACCESS EXCLUSIVE lock which blocks reads.

  help: Create an index CONCURRENTLY and create the constraint using the index.

```



### `squawk --help`



```

squawk

Find problems in your SQL



USAGE:

    squawk [FLAGS] [OPTIONS] [path]... [SUBCOMMAND]



FLAGS:

        --assume-in-transaction

            Assume that a transaction will wrap each SQL file when run by a migration tool



            Use --no-assume-in-transaction to override this setting in any config file that exists

    -h, --help

            Prints help information



        --list-rules

            List all available rules



    -V, --version

            Prints version information



        --verbose

            Enable debug logging output



OPTIONS:

    -c, --config 

            Path to the squawk config file (.squawk.toml)



        --dump-ast 

            Output AST in JSON [possible values: Raw, Parsed, Debug]



        --exclude-path ...

            Paths to exclude



            For example: --exclude-path=005_user_ids.sql --exclude-path=009_account_emails.sql



            --exclude-path='*user_ids.sql'



    -e, --exclude ...

            Exclude specific warnings



            For example: --exclude=require-concurrent-index-creation,ban-drop-database

        --explain 

            Provide documentation on the given rule



        --pg-version 

            Specify postgres version



            For example: --pg-version=13.0

        --reporter 

            Style of error reporting [possible values: Tty, Gcc, Json]



        --stdin-filepath 

            Path to use in reporting for stdin



ARGS:

    ...

            Paths to search



SUBCOMMANDS:

    help                Prints this message or the help of the given subcommand(s)

    upload-to-github    Comment on a PR with Squawk's results

```



## Rules



Individual rules can be disabled via the `--exclude` flag



```shell

squawk --exclude=adding-field-with-default,disallowed-unique-constraint example.sql

```



### Configuration file



Rules can also be disabled with a configuration file.



By default, Squawk will traverse up from the current directory to find a `.squawk.toml` configuration file. You may specify a custom path with the `-c` or `--config` flag.



```shell

squawk --config=~/.squawk.toml example.sql

```



The `--exclude` flag will always be prioritized over the configuration file.



**Example `.squawk.toml`**



```toml

excluded_rules = [

    "require-concurrent-index-creation",

    "require-concurrent-index-deletion",

]

```



See the [Squawk website](https://squawkhq.com/docs/rules) for documentation on each rule with examples and reasoning.



## Bot Setup



Squawk works as a CLI tool but can also create comments on GitHub Pull

Requests using the `upload-to-github` subcommand.



Here's an example comment created by `squawk` using the `example.sql` in the repo:



See the ["GitHub Integration" docs](https://squawkhq.com/docs/github_app) for more information.



## `pre-commit` hook



Integrate Squawk into Git workflow with [pre-commit](https://pre-commit.com/). Add the following

to your project's `.pre-commit-config.yaml`:



```

repos:

  - repo: https://github.com/sbdchd/squawk

    rev: v0.10.0

    hooks:

     - id: squawk

       files: path/to/postres/migrations/written/in/sql

```



Note the `files` parameter as it specifies the location of the files to be linted.



## prior art



- 



### related tools



- 

- 

- 

- 

- 

- 

- 



## related blog posts / SE Posts / PG Docs



- 

- 

- 

- 

- 

- 

- 

- 

- 



## dev



```shell

cargo install

cargo run

./s/test

./s/lint

./s/fmt

```



... or with nix:



```

$ nix develop

[nix-shell]$ cargo run

[nix-shell]$ cargo insta review

[nix-shell]$ ./s/test

[nix-shell]$ ./s/lint

[nix-shell]$ ./s/fmt

```



### adding a new rule



When adding a new rule, the `s/new-rule` script will create stubs for your rule in Rust and in Documentation site.



```bash

s/new-rule 'prefer big serial'

```



### releasing a new version



1. update the CHANGELOG.md and bump version in the cli `Cargo.toml`, ensure the

   lock file is updated, and update `package.json` and commit the changes



   ```bash

   # update version in Cargo.toml files and package.json to 4.5.3

   s/update-version 4.5.3

   ```



2. create a new release on github - CI will attach the binaries automatically

3. wait for build artifacts to be attached to release.

4. login to `npm` and publish new version.



   ```bash

   npm login

   npm publish

   ```



### algolia



The squawkhq.com Algolia index can be found on [the crawler website](https://crawler.algolia.com/admin/crawlers/9bf0dffb-bc5a-4d46-9b8d-2f1197285213/overview). Algolia reindexes the site every day at 5:30 (UTC).



## how it works



squawk wraps calls to [libpg_query-sys](https://github.com/tdbgamer/libpg_query-sys) in a safe

interface and parses the JSON into easier to work with structures.

libpg_query-sys in turn uses [bindgen](https://rust-lang.github.io/rust-bindgen/) to bind to

[libpg_query](https://github.com/lfittl/libpg_query), which itself wraps Postgres' SQL

parser in a bit of C code that outputs the parsed AST into a JSON string.



Squawk then runs the rule functions over the parsed AST, gathers and pretty

prints the rule violations.