https://github.com/yu-iskw/sqlfluff-complexity
A custom SQLFluff plugin to calculate SQL complexity
https://github.com/yu-iskw/sqlfluff-complexity
agent-harness dbt harness-engineering sql sqlfluff
Last synced: 23 days ago
JSON representation
A custom SQLFluff plugin to calculate SQL complexity
- Host: GitHub
- URL: https://github.com/yu-iskw/sqlfluff-complexity
- Owner: yu-iskw
- License: apache-2.0
- Created: 2026-05-01T09:50:56.000Z (about 2 months ago)
- Default Branch: main
- Last Pushed: 2026-06-02T10:09:07.000Z (27 days ago)
- Last Synced: 2026-06-02T12:10:23.135Z (27 days ago)
- Topics: agent-harness, dbt, harness-engineering, sql, sqlfluff
- Language: Python
- Homepage: https://pypi.org/project/sqlfluff-complexity/
- Size: 4.2 MB
- Stars: 4
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
- Agents: AGENTS.md
Awesome Lists containing this project
README
# sqlfluff-complexity
SQLFluff rules and reports for finding SQL and dbt models that are too complex to review safely.
`sqlfluff-complexity` adds CPX rules to SQLFluff for CTE count, join count, nested
subquery depth, `CASE` expressions, nested `CASE` depth, boolean predicates, window functions,
CTE dependency depth, set operations (`UNION` / `INTERSECT` / `EXCEPT`), inline derived tables, and
an aggregate weighted complexity score. The same metric engine also powers a companion
`sqlfluff-complexity report` command for non-blocking console, JSON, SARIF, and standalone
HTML reports.
## Who It Is For
- Analytics engineers who want dbt models to stay small enough to review.
- Data platform teams that already run SQLFluff in local development or CI.
- Teams that want gradual complexity reporting before turning on strict lint failures.
## Quick Start
**Requires SQLFluff 4.x** in the same environment (`sqlfluff>=4,<5`). SQLFluff 3 is not supported as of `sqlfluff-complexity` 0.4.0; pin `sqlfluff-complexity<0.4` if you cannot upgrade SQLFluff yet.
Install the custom SQLFluff plugin in the same Python environment where you run SQLFluff:
```bash
pip install sqlfluff-complexity
```
If your project uses `uv`:
```bash
uv add --dev sqlfluff-complexity
```
Then enable CPX rules in `.sqlfluff` and run SQLFluff:
```ini
[sqlfluff]
dialect = postgres
rules = CPX_C101,CPX_C102,CPX_C103,CPX_C104,CPX_C105,CPX_C106,CPX_C107,CPX_C108,CPX_C109,CPX_C110,CPX_C201
```
```bash
sqlfluff lint models/
```
For a complete walkthrough, see [docs/quickstart.md](docs/quickstart.md).
**Migrating `complexity_weights`:** Use a JSON object in `[sqlfluff:rules:CPX_C201]` (not comma-separated `key:value` lists). See [Configuration: aggregate score](docs/configuration.md#aggregate-score) and [CHANGELOG](CHANGELOG.md).
## Documentation
- [Quick start](docs/quickstart.md): install, configure, and run the first lint.
- [Configuration](docs/configuration.md): thresholds, aggregate weights, [SQLFluff nested config by path](docs/configuration.md#sqlfluff-nested-configuration-by-path), and `path_overrides`.
- [Rules reference](docs/rules.md): CPX rule codes and what each metric counts.
- [Reporting](docs/reporting.md): console, JSON, SARIF, and HTML report mode.
- [Python API](docs/python-api.md): stable programmatic imports (today: `report` pipeline and related config helpers; more subcommands documented here as they ship).
- [Adoption](docs/adoption.md): calibration, CI/SARIF examples, before/after SQL vignette.
- [dbt usage](docs/dbt.md): SQLFluff dbt templater compatibility and v1 boundaries.
- [Dialects](docs/dialects.md): tested dialects and dbt adapter mapping caveats.
- [Docs index](docs/index.md): all user, contributor, and design documents.
- [Internal import migration](docs/migration-internal.md): if you import `sqlfluff_complexity.core` submodules directly, update paths after subpackage refactors.
## Large dbt projects
This plugin does **not** read dbt artifacts (`manifest.json`, `run_results.json`, `catalog.json`) or graph metadata. Complexity is measured from the SQLFluff parse tree only—which aligns well with [SQLFluff’s dbt templater](docs/dbt.md) so `ref()`, `source()`, and macros compile before linting.
Practical adoption patterns:
- Use the dbt templater when you need compiled SQL fidelity; keep `sqlfluff-complexity` focused on structural signals from parsed SQL.
- Tune thresholds per rule and use [`path_overrides`](docs/configuration.md) on `CPX_C201` where staging vs marts need different budgets.
- Prefer `sqlfluff lint` on changed models in CI and [`sqlfluff-complexity report`](docs/reporting.md) for broader visibility without failing builds.
- For **CTE dependency depth**, see [docs/rules.md](docs/rules.md) (CPX_C107) and [docs/reporting.md](docs/reporting.md) (report vs per-`WITH` lint).
See also [dbt usage](docs/dbt.md) and [configuration](docs/configuration.md).
## Project Status
- Native SQLFluff plugin rules and the companion report CLI are available in the package.
- v1 measures SQLFluff-parsed SQL; direct dbt `manifest.json` and DAG-level metrics are deferred.
- Architecture decisions are recorded in [docs/adr/](docs/adr/).
## Development
Contributor setup, Nox sessions, fixture authoring, ADR workflow, and verifier guidance live in
[CONTRIBUTING.md](CONTRIBUTING.md). Agent-specific project instructions live in
[AGENTS.md](AGENTS.md).