{"id":20153002,"url":"https://github.com/csye7125-fall2023-group05/webapp-db","last_synced_at":"2026-02-03T07:33:22.850Z","repository":{"id":218844925,"uuid":"701883798","full_name":"csye7125-fall2023-group05/webapp-db","owner":"csye7125-fall2023-group05","description":"Flyway migration scripts to setup database tables for replication of highly available Postgresql clusters","archived":false,"fork":false,"pushed_at":"2023-11-20T03:51:46.000Z","size":193,"stargazers_count":0,"open_issues_count":1,"forks_count":2,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-07-07T02:05:08.362Z","etag":null,"topics":["db-migration","db-migration-service","dockerfile","flyway","flyway-migrations","flyway-postgresql"],"latest_commit_sha":null,"homepage":"","language":"Dockerfile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/csye7125-fall2023-group05.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-10-07T21:05:42.000Z","updated_at":"2024-01-24T03:57:37.000Z","dependencies_parsed_at":"2024-01-24T05:48:36.669Z","dependency_job_id":null,"html_url":"https://github.com/csye7125-fall2023-group05/webapp-db","commit_stats":null,"previous_names":["csye7125-fall2023-group05/webapp-db"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/csye7125-fall2023-group05/webapp-db","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csye7125-fall2023-group05%2Fwebapp-db","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csye7125-fall2023-group05%2Fwebapp-db/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csye7125-fall2023-group05%2Fwebapp-db/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csye7125-fall2023-group05%2Fwebapp-db/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/csye7125-fall2023-group05","download_url":"https://codeload.github.com/csye7125-fall2023-group05/webapp-db/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csye7125-fall2023-group05%2Fwebapp-db/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29037528,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-03T06:39:36.383Z","status":"ssl_error","status_checked_at":"2026-02-03T06:39:32.787Z","response_time":96,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["db-migration","db-migration-service","dockerfile","flyway","flyway-migrations","flyway-postgresql"],"created_at":"2024-11-13T23:16:24.088Z","updated_at":"2026-02-03T07:33:22.834Z","avatar_url":"https://github.com/csye7125-fall2023-group05.png","language":"Dockerfile","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🗄️ Database Migrations\n\n[Flyway](https://documentation.red-gate.com/fd/postgresql-184127604.html) is a database migration tool. We will use the flyway migration scripts that will migrate our database from our local workstation to a container running an `init container`, which will run the migration scripts before starting our REST API.\n\nWe will work with `PostgreSQL ^14` as our primary database, so we would need to setup our local environment accordingly.\n\n## ⏬ Local _PostgreSQL_ setup\n\n\u003e NOTE: The steps mentioned below are only for MacOS. For other distros, please refer to the [official Postgresql documentation](https://www.postgresql.org/download/).\n\n- Install `postgres` locally:\n\n```bash\nbrew install postgresql@15\n# validate installation\npsql --version\n```\n\n- After `postgres` is installed, we need the postgres service up and running:\n\n```bash\nbrew service start postgres@15\n```\n\n- Let's start by connecting to the default `postgres` database:\n\n```bash\npsql postgres\n```\n\n- Create a role in postgres and assign permissions:\n\n```bash\nCREATE ROLE \u003cuser\u003e WITH LOGIN PASSWORD \u003cuser_password\u003e;\nALTER ROLE \u003cuser\u003e CREATEDB;\n# if required, add other permissions as well: REPLICATION, CREATEROLE, etc\n\\du # to list all users and permissions\n\\q # quit postgres cli\n```\n\n- Connect to `postgres` db using new role created, create and connect to a new database:\n\n```bash\n# connect to db using new user\npsql -d postgres -U \u003cuser\u003e\n# create a database\nCREATE DATABASE \u003cdb_name\u003e;\n\\c \u003cdb_name\u003e # connect to new db\n```\n\n## 🚚 Working with Flyway\n\n\u003e NOTE: The steps mentioned below are only for MacOS. For other distros, please refer to the [official Flyway documentation](https://www.red-gate.com/products/flyway/editions).\n\n- Install `flyway` locally:\n\n  ```bash\n  brew install flyway\n  # verify installation\n  flyway version\n  ```\n\n- Creating migration scripts:\n  Migration scripts ending with `.sql` are the scripts used to migrate/setup our database using `flyway` for SQL based migrations. These should be placed in a folder called `sql`, with proper naming conventions as mentioned in the [documentation](https://documentation.red-gate.com/fd/migrations-184127470.html).\n\n- Create a `flyway.conf` migration configuration file:\n  This file contains the database connection URL, which connects to our database instance/service. It also contains the `username` and `password` for the database we connect to and run the migrations on. For more details on the configuration file, refer [this documentation](https://documentation.red-gate.com/fd/configuration-files-184127472.html).\n  Alternatively, you can also follow the [`example.flyway.conf`](./example.flyway.conf) file for configuration options.\n\n- Using environment variables:\n  Since we will use the kubernetes `Secret` resource to configure our flyway environment variables, we just need to follow the naming convention for the variables in the `flyway.conf` configuration file:\n\n  ```ini\n  flyway.url=${DB_FLYWAY_CONNECTION_URL}\n  flyway.user=${DB_FLYWAY_USERNAME}\n  flyway.password=${DB_FLYWAY_PASSWORD}\n  ```\n\n- To run migrations locally on a postgres service, we can configure the migration options in the `flyway.conf` file and run the `migrate command`.\n\n  ```bash\n  flyway migrate\n  ```\n\n\u003e NOTE: If the migrations are already performed, or there are no migrations to be performed, flyway returns a no-op.\n\n## 🐳 Containerize migrations\n\nHere, we containerize the migration scripts and configuration files into a docker image. Since this image will run within an `initContainer` in a k8s deployment, this docker image does not need to run any _CMD_ to run the migrations (since this will be done within the `initContainer`). A `.dockerignore` file is also needed to prevent copying files and folders that are not required in the final build image.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsye7125-fall2023-group05%2Fwebapp-db","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcsye7125-fall2023-group05%2Fwebapp-db","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsye7125-fall2023-group05%2Fwebapp-db/lists"}