{"id":13484835,"url":"https://github.com/prisma/prisma-engines","last_synced_at":"2025-05-08T23:36:38.967Z","repository":{"id":36951376,"uuid":"204033634","full_name":"prisma/prisma-engines","owner":"prisma","description":"🚂 Engine components of Prisma ORM","archived":false,"fork":false,"pushed_at":"2025-05-08T17:11:53.000Z","size":2648909,"stargazers_count":1252,"open_issues_count":51,"forks_count":262,"subscribers_count":18,"default_branch":"main","last_synced_at":"2025-05-08T22:35:22.911Z","etag":null,"topics":["prisma","rust"],"latest_commit_sha":null,"homepage":"https://www.prisma.io/docs/concepts/components/prisma-engines","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/prisma.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-08-23T16:25:29.000Z","updated_at":"2025-05-08T17:17:19.000Z","dependencies_parsed_at":"2023-09-25T02:11:56.010Z","dependency_job_id":"bb07befa-fe3c-494b-ab93-de04ff736e69","html_url":"https://github.com/prisma/prisma-engines","commit_stats":{"total_commits":7546,"total_committers":80,"mean_commits":94.325,"dds":0.7283328915981977,"last_synced_commit":"e9d4b3877607013ee1d01de5158e44e6003052b8"},"previous_names":[],"tags_count":8397,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prisma%2Fprisma-engines","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prisma%2Fprisma-engines/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prisma%2Fprisma-engines/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prisma%2Fprisma-engines/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/prisma","download_url":"https://codeload.github.com/prisma/prisma-engines/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253166392,"owners_count":21864467,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["prisma","rust"],"created_at":"2024-07-31T17:01:35.619Z","updated_at":"2025-05-08T23:36:38.924Z","avatar_url":"https://github.com/prisma.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"# Prisma Engines\n\n[![Query Engine](https://github.com/prisma/prisma-engines/actions/workflows/test-query-engine.yml/badge.svg)](https://github.com/prisma/prisma-engines/actions/workflows/test-query-engine.yml)\n[![Schema Engine + sql_schema_describer](https://github.com/prisma/prisma-engines/actions/workflows/test-schema-engine.yml/badge.svg)](https://github.com/prisma/prisma-engines/actions/workflows/test-schema-engine.yml)\n[![Cargo docs](https://github.com/prisma/prisma-engines/actions/workflows/on-push-to-main.yml/badge.svg)](https://github.com/prisma/prisma-engines/actions/workflows/on-push-to-main.yml)\n\nThis repository contains a collection of engines that power the core stack for\n[Prisma](https://github.com/prisma/prisma), most prominently [Prisma\nClient](https://www.prisma.io/client) and [Prisma\nMigrate](https://www.prisma.io/migrate).\n\nIf you're looking for how to install Prisma or any of the engines, the [Getting\nStarted](https://www.prisma.io/docs/getting-started) guide might be useful.\n\nThis document describes some of the internals of the engines, and how to build\nand test them.\n\n## What's in this repository\n\nThis repository contains four engines:\n\n- *Query engine*, used by the client to run database queries from Prisma Client\n- *Query compiler*, an actively developed replacement for the query engine, used\n  to compile Prisma Client queries into query plans containing SQL statements\n  and other operations, which the client can use to execute Prisma queries\n  entirely in JS land\n- *Schema engine*, used to create and run migrations and introspection\n- *Prisma Format*, historically used to format prisma files (hence the name),\n  now powers other LSP functionality as well\n\nAdditionally, the *psl* (Prisma Schema Language) is the library that defines how\nthe language looks like, how it's parsed, etc.\n\nYou'll also find:\n- *libs*, for various (small) libraries such as macros, user facing errors,\n    various connector/database-specific libraries, etc.\n- a `docker-compose.yml` file that's helpful for running tests and bringing up\n    containers for various databases\n- a `shell.nix` file for bringing up all dependencies and making it easy to\n    build the code in this repository (the use of this file and `nix` is\n    entirely optional, but can be a good and easy way to get started)\n- an `.envrc` file to make it easier to set everything up, including the `nix\n    shell`\n\n## Documentation\n\nThe [API docs (cargo doc)](https://prisma.github.io/prisma-engines/) are\npublished on our fabulous repo page.\n\n## Building Prisma Engines\n\n**Prerequisites:**\n\n- Installed the latest stable version of the Rust toolchain. You can get the\n  toolchain at [rustup](https://rustup.rs/) or the package manager of your\n  choice.\n- Linux only: OpenSSL is required to be installed.\n- Installed [direnv](https://github.com/direnv/direnv), then `direnv allow` on\n  the repository root.\n  - Make sure direnv is [hooked](https://direnv.net/docs/hook.html) into your shell\n  - Alternatively: Load the defined environment in `./.envrc` manually in your\n    shell.\n\nNote for nix users: it should be enough to `direnv allow`.\n**How to build:**\n\nTo build all engines, simply execute `cargo build` on the repository root. This\nbuilds non-production debug binaries. If you want to build the optimized\nbinaries in release mode, the command is `cargo build --release`.\n\nDepending on how you invoked `cargo` in the previous step, you can find the\ncompiled binaries inside the repository root in the `target/debug` (without\n`--release`) or `target/release` directories (with `--release`):\n\n| Prisma Component | Path to Binary                            |\n| ---------------- | ----------------------------------------- |\n| Query Engine     | `./target/[debug\\|release]/query-engine`  |\n| Schema Engine    | `./target/[debug\\|release]/schema-engine` |\n| Prisma Format    | `./target/[debug\\|release]/prisma-fmt`    |\n\n## Prisma Schema Language\n\nThe *Prisma Schema Language* is a library which defines the data structures and\nparsing rules for prisma files, including the available database connectors. For\nmore technical details, please check the [library README](./psl/README.md).\n\nThe PSL is used throughout the schema engine, as well as\nprisma format. The DataModeL (DML), which is an annotated version of the PSL is\nalso used as input for the query engine.\n\n## Query Engine\n\nThe *Query Engine* is how Prisma Client queries are executed. Here's a brief\ndescription of what it does:\n- takes as inputs an annotated version of the Prisma Schema file called the\n    DataModeL (DML),\n- using the DML (specifically, the datasources and providers), it builds up a\n    [GraphQL](https://graphql.org) model for queries and responses,\n- runs as a server listening for GraphQL queries,\n- it translates the queries to the respective native datasource(s) and\n    returns GraphQL responses, and\n- handles all connections and communication with the native databases.\n\nWhen used through Prisma Client, there are two ways for the Query Engine to\nbe executed:\n- as a binary, downloaded during installation, launched at runtime;\n    communication happens via HTTP (`./query-engine/query-engine`)\n- as a native, platform-specific Node.js addon; also downloaded during\n    installation (`./query-engine/query-engine-node-api`)\n\n### Usage\n\nYou can also run the Query Engine as a stand-alone GraphQL server.\n\n**Warning**: There is no guaranteed API stability. If using it on production\nplease be aware the api and the query language can change any time.\n\nNotable environment flags:\n\n- `RUST_LOG_FORMAT=(devel|json)` sets the log format. By default outputs `json`.\n- `QE_LOG_LEVEL=(info|debug|trace)` sets the log level for the Query Engine. If\n    you need Query Graph debugging logs, set it to \"trace\"\n- `FMT_SQL=1` enables logging _formatted_ SQL queries\n- `PRISMA_DML_PATH=[path_to_datamodel_file]` should point to the datamodel file\n  location. This or `PRISMA_DML` is required for the Query Engine to run.\n- `PRISMA_DML=[base64_encoded_datamodel]` an alternative way to provide a\n  datamodel for the server.\n- `RUST_BACKTRACE=(0|1)` if set to 1, the error backtraces will be printed to\n  the STDERR.\n- `LOG_QUERIES=[anything]` if set, the SQL queries will be written to the `INFO`\n  log. Needs the right log level enabled to be seen from the terminal.\n- `RUST_LOG=[filter]` sets the filter for the logger. Can be either `trace`,\n  `debug`, `info`, `warning` or `error`, that will output ALL logs from every\n  crate from that level. The `.envrc` in this repo shows how to log different\n  parts of the system in a more granular way.\n\nStarting the Query Engine:\n\nThe engine can be started either with using the `cargo` build tool, or\npre-building a binary and running it directly. If using `cargo`, replace\nwhatever command that starts with `./query-engine` with `cargo run --bin query-engine --`.\n\nYou can also pass `--help` to find out more options to run the engine.\n\n### Metrics\n\nRunning `make show-metrics` will start Prometheus and Grafana with a default metrics dashboard.\nPrometheus will scrape the `/metrics` endpoint to collect the engine's metrics\n\nNavigate to `http://localhost:3000` to view the Grafana dashboard.\n\n## Schema Engine\n\nThe *Schema Engine* does a couple of things:\n- creates new migrations by comparing the prisma file with the current state of\n    the database, in order to bring the database in sync with the prisma file\n- run these migrations and keeps track of which migrations have been executed\n- (re-)generate a prisma schema file starting from a live database\n\nThe engine uses:\n- the prisma files, as the source of truth\n- the database it connects to, for diffing and running migrations, as well as\n  keeping track of migrations in the `_prisma_migrations` table\n- the `prisma/migrations` directory which acts as a database of existing\n  migrations\n\n## Prisma format\n\nPrisma format can format prisma schema files. It also comes as a WASM module via\na node package. You can read more [here](./prisma-schema-wasm/README.md).\n\n## Debugging\n\nWhen trying to debug code, here's a few things that might be useful:\n- use the language server; being able to go to definition and reason about code\n    can make things a lot easier,\n- add `dbg!()` statements to validate code paths, inspect variables, etc.,\n- you can control the amount of logs you see, and where they come from using the\n    `RUST_LOG` environment variable; see [the documentation](https://docs.rs/env_logger/0.9.1/env_logger/#enabling-logging)\n\n## Testing\n\nThere are two test suites for the engines: Unit tests and\nintegration tests.\n\n- **Unit tests**: They test internal\n  functionality of individual crates and components.\n\n  You can find them across the whole codebase, usually in `./tests` folders at\n  the root of modules. These tests can be executed via `cargo test`. Note that\n  some of them will require the `TEST_DATABASE_URL` enviornment variable set up.\n\n- **Integration tests**: They run GraphQL queries against isolated\n  instances of the Query Engine and asserts that the responses are correct.\n\n  You can find them at `./query-engine/connector-test-kit-rs`.\n\n### Set up \u0026 run tests:\n\n**Prerequisites:**\n\n- Installed Rust toolchain.\n- Installed Docker.\n- Installed `direnv`, then `direnv allow` on the repository root.\n  - Alternatively: Load the defined environment in `./.envrc` manually in your shell.\n\n**Setup:**\n\nThere are helper `make` commands to set up a test environment for a specific\ndatabase connector you want to test. The commands set up a container (if needed)\nand write the `.test_config` file, which is picked up by the integration\ntests:\n\n- `make dev-mysql`: MySQL 5.7\n- `make dev-mysql8`: MySQL 8\n- `make dev-postgres`: PostgreSQL 10\n- `make dev-sqlite`: SQLite\n- `make dev-mongodb_5`: MongoDB 5\n\n\\*_On windows:_\nIf not using WSL, `make` is not available and you should just see what your\ncommand does and do it manually. Basically this means editing the\n`.test_config` file and starting the needed Docker containers.\n\nTo actually get the tests working, read the contents of `.envrc`. Then `Edit environment variables for your account` from Windows settings, and add at least\nthe correct values for the following variables:\n\n- `WORKSPACE_ROOT` should point to the root directory of `prisma-engines` project.\n- `PRISMA_BINARY_PATH` is usually\n  `%WORKSPACE_ROOT%\\target\\release\\query-engine.exe`.\n- `SCHEMA_ENGINE_BINARY_PATH` should be\n  `%WORKSPACE_ROOT%\\target\\release\\schema-engine.exe`.\n\nOther variables may or may not be useful.\n\n**Run:**\n\nRun `cargo test` in the repository root.\n\n### Testing driver adapters\n\nPlease refer to the [Testing driver adapters](./query-engine/connector-test-kit-rs/README.md) section in the connector-test-kit-rs README.\n\n**ℹ️ Important note on developing features that require changes to the both the query engine, and driver adapters code**\n\nAs explained in [Testing driver adapters](./query-engine/connector-test-kit-rs/README.md), running `DRIVER_ADAPTER=$adapter make qe-test`\nwill ensure you have prisma checked out in your filesystem in the same directory as prisma-engines. This is needed because the driver adapters code is symlinked in prisma-engines.\n\nWhen working on a feature or bugfix spanning adapters code and query-engine code, you will need to open sibling PRs in `prisma/prisma` and `prisma/prisma-engines` respectively.\nLocally, each time you run `DRIVER_ADAPTER=$adapter make test-qe` tests will run using the driver adapters built from the source code in the working copy of prisma/prisma. All good.\n\nIn CI, tho', we need to denote which branch of prisma/prisma we want to use for tests. In CI, there's no working copy of prisma/prisma before tests run.\nThe CI jobs clones prisma/prisma `main` branch by default, which doesn't include your local changes. To test in integration, we can tell CI to use the branch of prisma/prisma containing\nthe changes in adapters. To do it, add the following tag to your PR's description on a separate line:\n\n```\n/prisma-branch your/branch\n```\n\nReplace `your/branch` with the name of your branch in the `prisma` repository.\n\nGitHub actions will then pick up the branch name and use it to clone that branch's code of prisma/prisma, and build the driver adapters code from there.\n\nWhen it's time to merge the sibling PRs, you'll need to merge the prisma/prisma PR first, so when merging the engines PR you have the code of the adapters ready in prisma/prisma `main` branch.\n\n### Testing engines in `prisma/prisma`\n\nYou can trigger releases from this repository to npm that can be used for testing the engines in `prisma/prisma` either automatically or manually:\n\n#### Automated integration releases from this repository to npm\n\nAny branch name starting with `integration/` will, first, run the full test suite in GH Actions and, second, run the release workflow (build and upload engines to S3 \u0026 R2).\nTo trigger the release on any other branch, you have two options:\n- Either run [build-engines](https://github.com/prisma/prisma-engines/actions/workflows/build-engines.yml) workflow on a specified branch manually.\n- Or add `[integration]` string anywhere in your commit messages/\n\nThe journey through the pipeline is the same as a commit on the `main` branch.\n- It will trigger [`prisma/engines-wrapper`](https://github.com/prisma/engines-wrapper) and publish a new [`@prisma/engines-version`](https://www.npmjs.com/package/@prisma/engines-version) npm package but on the `integration` tag.\n- Which triggers [`prisma/prisma`](https://github.com/prisma/prisma) to create a `chore(Automated Integration PR): [...]` PR with a branch name also starting with `integration/`\n- Since in `prisma/prisma` we also trigger the publish pipeline when a branch name starts with `integration/`, this will publish all `prisma/prisma` monorepo packages to npm on the `integration` tag.\n- Our [ecosystem-tests](https://github.com/prisma/ecosystem-tests/) tests will automatically pick up this new version and run tests, results will show in [GitHub Actions](https://github.com/prisma/ecosystem-tests/actions?query=branch%3Aintegration)\n\nThis end to end will take minimum ~1h20 to complete, but is completely automated :robot:\n\nNotes:\n- tests and publishing workflows are run in parallel in both `prisma/prisma-engines` and `prisma/prisma` repositories. So, it is possible that the engines would be published and only then test suite will\ndiscover a defect. It is advised that to keep an eye on both test and publishing workflows.\n\n#### Manual integration releases from this repository to npm\n\nAdditionally to the automated integration release for `integration/` branches, you can also trigger a publish **manually** in the Buildkite `[Test] Prisma Engines` job if that succeeds for _any_ branch name. Click \"🚀 Publish binaries\" at the bottom of the test list to unlock the publishing step. When all the jobs in `[Release] Prisma Engines` succeed, you also have to unlock the next step by clicking \"🚀 Publish client\". This will then trigger the same journey as described above.\n\n## Parallel rust-analyzer builds\n\nWhen rust-analzyer runs `cargo check` it will lock the build directory and stop any cargo commands from running until it has completed. This makes the build process feel a lot longer. It is possible to avoid this by setting a different build path for\nrust-analyzer. To avoid this. Open VSCode settings and search for `Check on Save: Extra Args`. Look for the `Rust-analyzer › Check On Save: Extra Args` settings and add a new directory for rust-analyzer. Something like:\n\n```\n--target-dir:/tmp/rust-analyzer-check\n```\n\n\n## Community PRs: create a local branch for a branch coming from a fork\n\nTo trigger an [Automated integration releases from this repository to npm](#automated-integration-releases-from-this-repository-to-npm) or [Manual integration releases from this repository to npm](#manual-integration-releases-from-this-repository-to-npm) branches of forks need to be pulled into this repository so the Buildkite job is triggered. You can use these GitHub and git CLI commands to achieve that easily:\n\n```\ngh pr checkout 4375\ngit checkout -b integration/sql-nested-transactions\ngit push --set-upstream origin integration/sql-nested-transactions\n```\n\nIf there is a need to re-create this branch because it has been updated, deleting it and re-creating will make sure the content is identical and avoid any conflicts.\n\n```\ngit branch --delete integration/sql-nested-transactions\ngh pr checkout 4375\ngit checkout -b integration/sql-nested-transactions\ngit push --set-upstream origin integration/sql-nested-transactions --force\n```\n\n## Security\n\nIf you have a security issue to report, please contact us at [security@prisma.io](mailto:security@prisma.io?subject=[GitHub]%20Prisma%202%20Security%20Report%20Engines)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprisma%2Fprisma-engines","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprisma%2Fprisma-engines","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprisma%2Fprisma-engines/lists"}