{"id":19571631,"url":"https://github.com/serlo/database-layer","last_synced_at":"2025-04-27T03:32:31.949Z","repository":{"id":37032358,"uuid":"327547805","full_name":"serlo/database-layer","owner":"serlo","description":"GraphQL-inspired API in front of the mysql database of https://serlo.org/","archived":true,"fork":false,"pushed_at":"2024-07-15T20:50:48.000Z","size":953466,"stargazers_count":11,"open_issues_count":6,"forks_count":3,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-04-05T01:33:49.270Z","etag":null,"topics":["database","mysql","mysql-database","rust","serlo","service"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/serlo.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-01-07T08:21:50.000Z","updated_at":"2025-03-21T08:49:41.000Z","dependencies_parsed_at":"2023-02-17T22:15:46.063Z","dependency_job_id":"8297198a-5a45-47c6-bef7-2810f185e7a7","html_url":"https://github.com/serlo/database-layer","commit_stats":null,"previous_names":[],"tags_count":82,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serlo%2Fdatabase-layer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serlo%2Fdatabase-layer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serlo%2Fdatabase-layer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serlo%2Fdatabase-layer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/serlo","download_url":"https://codeload.github.com/serlo/database-layer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251085145,"owners_count":21533821,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","mysql","mysql-database","rust","serlo","service"],"created_at":"2024-11-11T06:19:36.786Z","updated_at":"2025-04-27T03:32:26.937Z","avatar_url":"https://github.com/serlo.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"https://raw.githubusercontent.com/serlo/frontend/staging/apps/web/public/_assets/img/serlo-logo-gh.svg\" alt=\"Serlo Logo\" title=\"Serlo\" align=\"right\" height=\"75\" /\u003e\n\n# serlo.org – Database Layer\n\nThe database layer provides a RESTful API in front of the database of [Serlo](https://serlo.org/).\n\n## Setup\n\nYou need the following installed on your system:\n\n- [Docker](https://docs.docker.com/engine/installation/)\n- [Rust](https://www.rust-lang.org) version from [.tool-versions](.tool-versions). [^1]\n- [Node.js](https://nodejs.org/) and [yarn cli](https://yarnpkg.com/) versions from [.tool-versions](.tool-versions), in order to run the pact tests (= contract test suite we use). [^1]\n\n[^1]: You can use [asdf](https://asdf-vm.com/) for the installation.\n\n### Install sqlx-cli\n\nYou need to install [`sqlx-cli`](https://github.com/launchbadge/sqlx/tree/master/sqlx-cli) via\n\n```sh\ncargo install sqlx-cli\n```\n\n### Install dependencies for yarn and contract tests\n\nRun `yarn` to install all necessary node dependencies (needed for development and running the contract tests).\n\n### Install `jq` for `yarn fetch`\n\nFor the command `yarn fetch` the tool [`jq`](https://stedolan.github.io/jq/) needs to be installed.\n\n### Install gcc\n\nOn Ubuntu you also need [`gcc`](https://gcc.gnu.org/) to run cargo.\n\n### Database\n\nRun `yarn start` to start the database. It will be available under `mysql://root:secret@localhost:3306/serlo` and in it you will see an anonymized dump of our database from 2015. You can use the following commands together with the database:\n\n- `yarn mysql` – Start a shell for the database\n- `yarn mysql:import-anonymous-data` – Import a current and anonymized dump of the Serlo database (normally one day old) – This needs [`gsutil` installed](https://cloud.google.com/storage/docs/gsutil_install) and you need to auth as well\n- `yarn mysql:rollback` – Rollback to the 2015 dump of the database\n\n## Development\n\nRun `cargo run` to install dependencies and start the webserver.\n\nNow you can mock a request from the terminal:\n\n```sh\ncurl -H \"Content-Type: application/json\" -X POST -d '{\"type\":\"UuidQuery\",\"payload\":{\"id\":1565}}' http://localhost:8080/\n```\n\nHappy coding!\n\n### sqlx and `yarn sqlx:prepare`\n\nWe use [sqlx](https://github.com/launchbadge/sqlx) for creating and executing SQL queries. Here it is necessary that you run locally a local database (see section above) in order to be able to compile the source code. Also in the end of each PR the command `yarn sqlx:prepare` needs to be executed and the changes in [`sqlx-data.json`](./sqlx-data.json) need to be committed.\n\n### Run all checks\n\nThe command `yarn check:all` will run all checks (like `cargo test` or `cargo clippy`) against the codebase. When you run `yarn check:all --no-uncommitted-changes` there is also a check whether you have uncommitted changes in your workspace. With this command you can test whether you are ready to open a pull request.\n\nIn case you want to run all tests automatically before pushing you can use a [git hook](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks). Here you can execute in a shell in the directory of this repository:\n\n```sh\necho \"yarn check:all --no-uncommitted-changes\" \u003e .git/hooks/pre-push\nchmod +x .git/hooks/pre-push\n```\n\n### Test endpoints with `yarn fetch`\n\nWith `yarn fetch` you can test queries against the database layer. The script will compile and run a local version of the database layer if necessary. You can call it via\n\n```sh\nyarn fetch [Message] [Payload]\n```\n\nExample:\n\n```sh\nyarn fetch UuidQuery '{\"id\": 1}'\n```\n\nYou can also omit the second argument if the endpoint does not need a payload:\n\n```sh\nyarn fetch SubjectsQuery\n```\n\n### Run contract tests\n\nIn order to run contract tests you need to start the server of the database layer via `cargo run` in a shell. Afterwards you can execute the contract tests with `yarn pacts`. There is also the script [`./scripts/pacts.sh`](./scripts/pacts.sh`) which automatically compiles and runs the server if necessary before running the contract tests.\n\nYou can also provide the path to a local pact file in case you want to test new API changes against the database layer. Example:\n\n```sh\nPACT_FILE=../api.serlo.org/pacts/api.serlo.org-serlo.org-database-layer.json ./scripts/pacts.sh\n```\n\nYou can generate the pacts .json file (`api.serlo.org-serlo.org-database-layer.json`) locally by checking out the [api.serlo.org/](https://github.com/serlo/api.serlo.org/)\nrepository and running `yarn pacts` at the root of the cloned repository. The generated .json file will be in `path_to_api.serlo.org_repository/pacts`.\n\n### Other helpful commands\n\n- `cargo test` – Run all tests (see https://doc.rust-lang.org/book/ch11-01-writing-tests.html )\n- `cargo bench` - Run performance benchmarks to test for example how changing a query affects its performance\n- `cargo clippy` – Lint the whole codebase (see https://github.com/rust-lang/rust-clippy )\n- `yarn mysql` – Start a MySQL shell for the local mysql server.\n- `yarn format` – Format all local source files.\n- `yarn update-version` – Start process for adding new server version (only required for deployment)\n\n- See also [`package.json`](./package.json) for the list of all yarn scripts.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fserlo%2Fdatabase-layer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fserlo%2Fdatabase-layer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fserlo%2Fdatabase-layer/lists"}