Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/serlo/database-layer
GraphQL-inspired API in front of the mysql database of https://serlo.org/
https://github.com/serlo/database-layer
database mysql mysql-database rust serlo service
Last synced: 4 days ago
JSON representation
GraphQL-inspired API in front of the mysql database of https://serlo.org/
- Host: GitHub
- URL: https://github.com/serlo/database-layer
- Owner: serlo
- License: apache-2.0
- Created: 2021-01-07T08:21:50.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2024-06-24T20:58:12.000Z (5 months ago)
- Last Synced: 2024-06-24T22:34:27.727Z (5 months ago)
- Topics: database, mysql, mysql-database, rust, serlo, service
- Language: Rust
- Homepage:
- Size: 899 MB
- Stars: 13
- Watchers: 7
- Forks: 3
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# serlo.org – Database Layer
The database layer provides a RESTful API in front of the database of [Serlo](https://serlo.org/).
## Setup
You need the following installed on your system:
- [Docker](https://docs.docker.com/engine/installation/)
- [Rust](https://www.rust-lang.org) version from [.tool-versions](.tool-versions). [^1]
- [Node.js](https://nodejs.org/) and [yarn cli](https://yarnpkg.com/) versions from [.tool-versions](.tool-versions), in order to run the pact tests (= contract test suite we use). [^1]
[^1]: You can use [asdf](https://asdf-vm.com/) for the installation.
### Install sqlx-cli
You need to install [`sqlx-cli`](https://github.com/launchbadge/sqlx/tree/master/sqlx-cli) via
```sh
cargo install sqlx-cli
```
### Install dependencies for yarn and contract tests
Run `yarn` to install all necessary node dependencies (needed for development and running the contract tests).
### Install `jq` for `yarn fetch`
For the command `yarn fetch` the tool [`jq`](https://stedolan.github.io/jq/) needs to be installed.
### Install gcc
On Ubuntu you also need [`gcc`](https://gcc.gnu.org/) to run cargo.
### Database
Run `yarn start` to start the database. It will be available under `mysql://root:secret@localhost:3306/serlo` and in it you will see an anonymized dump of our database from 2015. You can use the following commands together with the database:
- `yarn mysql` – Start a shell for the database
- `yarn mysql:import-anonymous-data` – Import a current and anonymized dump of the Serlo database (normally one day old) – This needs [`gsutil` installed](https://cloud.google.com/storage/docs/gsutil_install) and you need to auth as well
- `yarn mysql:rollback` – Rollback to the 2015 dump of the database
## Development
Run `cargo run` to install dependencies and start the webserver.
Now you can mock a request from the terminal:
```sh
curl -H "Content-Type: application/json" -X POST -d '{"type":"UuidQuery","payload":{"id":1565}}' http://localhost:8080/
```
Happy coding!
### sqlx and `yarn sqlx:prepare`
We use [sqlx](https://github.com/launchbadge/sqlx) for creating and executing SQL queries. Here it is necessary that you run locally a local database (see section above) in order to be able to compile the source code. Also in the end of each PR the command `yarn sqlx:prepare` needs to be executed and the changes in [`sqlx-data.json`](./sqlx-data.json) need to be committed.
### Run all checks
The command `yarn check:all` will run all checks (like `cargo test` or `cargo clippy`) against the codebase. When you run `yarn check:all --no-uncommitted-changes` there is also a check whether you have uncommitted changes in your workspace. With this command you can test whether you are ready to open a pull request.
In case you want to run all tests automatically before pushing you can use a [git hook](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks). Here you can execute in a shell in the directory of this repository:
```sh
echo "yarn check:all --no-uncommitted-changes" > .git/hooks/pre-push
chmod +x .git/hooks/pre-push
```
### Test endpoints with `yarn fetch`
With `yarn fetch` you can test queries against the database layer. The script will compile and run a local version of the database layer if necessary. You can call it via
```sh
yarn fetch [Message] [Payload]
```
Example:
```sh
yarn fetch UuidQuery '{"id": 1}'
```
You can also omit the second argument if the endpoint does not need a payload:
```sh
yarn fetch SubjectsQuery
```
### Run contract tests
In order to run contract tests you need to start the server of the database layer via `cargo run` in a shell. Afterwards you can execute the contract tests with `yarn pacts`. There is also the script [`./scripts/pacts.sh`](./scripts/pacts.sh`) which automatically compiles and runs the server if necessary before running the contract tests.
You can also provide the path to a local pact file in case you want to test new API changes against the database layer. Example:
```sh
PACT_FILE=../api.serlo.org/pacts/api.serlo.org-serlo.org-database-layer.json ./scripts/pacts.sh
```
You can generate the pacts .json file (`api.serlo.org-serlo.org-database-layer.json`) locally by checking out the [api.serlo.org/](https://github.com/serlo/api.serlo.org/)
repository and running `yarn pacts` at the root of the cloned repository. The generated .json file will be in `path_to_api.serlo.org_repository/pacts`.
### Other helpful commands
- `cargo test` – Run all tests (see https://doc.rust-lang.org/book/ch11-01-writing-tests.html )
- `cargo bench` - Run performance benchmarks to test for example how changing a query affects its performance
- `cargo clippy` – Lint the whole codebase (see https://github.com/rust-lang/rust-clippy )
- `yarn mysql` – Start a MySQL shell for the local mysql server.
- `yarn format` – Format all local source files.
- `yarn update-version` – Start process for adding new server version (only required for deployment)
- See also [`package.json`](./package.json) for the list of all yarn scripts.