Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/halvardssm/deno-nessie
A modular Deno library for PostgreSQL, MySQL, MariaDB and SQLite migrations
https://github.com/halvardssm/deno-nessie
deno deno-mysql deno-postgres deno-sqlite denoland hacktoberfest mysql postgres postgresql sqlite3 typescript
Last synced: 8 days ago
JSON representation
A modular Deno library for PostgreSQL, MySQL, MariaDB and SQLite migrations
- Host: GitHub
- URL: https://github.com/halvardssm/deno-nessie
- Owner: halvardssm
- License: mit
- Created: 2020-04-15T13:55:54.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2024-04-10T07:30:49.000Z (8 months ago)
- Last Synced: 2024-11-21T10:44:20.995Z (21 days ago)
- Topics: deno, deno-mysql, deno-postgres, deno-sqlite, denoland, hacktoberfest, mysql, postgres, postgresql, sqlite3, typescript
- Language: TypeScript
- Homepage:
- Size: 786 KB
- Stars: 529
- Watchers: 11
- Forks: 31
- Open Issues: 8
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: .github/CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
- Code of conduct: .github/CODE_OF_CONDUCT.md
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
- awesome-deno - nessie - 为PostgreSQL，MySQL 和 SQLite创建，迁移和回滚迁移。 (Uncategorized / Uncategorized)
- awesome-list - deno-nessie
- awesome-deno-cn - @halvardssm/deno-nessie
- awesome-deno - nessie - Create, migrate and rollback migrations for PostgreSQL, MySQL and SQLite. (Tools / Online Playgrounds)
- awesome-deno - nessie - Create, migrate and rollback migrations for PostgreSQL, MySQL and SQLite. (Modules / Database)
README
Nessie
A modular database migration tool for Deno inspired by Laravel and Phinx.
Supports PostgreSQL, MySQL, MariaDB and SQLite.
**Call for donations**: If you are using Nessie commercially, please consider
supporting the future development. See
[this issue](https://github.com/halvardssm/deno-nessie/issues/130) for more
information.
> ⚠️ With the native Prisma support for Deno, I no longer use Nessie for my
> projects. This means that Nessie will be unmaintained in the near future. See
> the related [issue](https://github.com/halvardssm/deno-nessie/issues/165) for
> more information.
🎉 **Version 2 is released**: To migrate from version 1 follow the steps in the
[migration section](#migrate-from-version-1) bellow.
> See documentation for the
> [clients](https://doc.deno.land/https/deno.land/x/nessie/mod.ts).
> Even though all examples in this readme applies unversioned usage, you should
> always use a version when using Nessie.
---
## Contents
- [Contents](#contents)
- [Available Via](#available-via)
- [CLI Usage](#cli-usage)
- [Flags](#flags)
- [Deno flags and Permissions](#deno-flags-and-permissions)
- [Config file](#config-file)
- [Remote Migration or Seed files](#remote-migration-or-seed-files)
- [Custom Migration or Seed templates](#custom-migration-or-seed-templates)
- [Docker usage](#docker-usage)
- [Uses](#uses)
- [Examples](#examples)
- [Clients](#clients)
- [How to make a client](#how-to-make-a-client)
- [Migrate from version 1](#migrate-from-version-1)
- [Contributing](#contributing)
## Available Via
- https://deno.land/x/nessie
- https://raw.githubusercontent.com/halvardssm/deno-nessie
- https://nest.land/package/Nessie
- https://hub.docker.com/repository/docker/halvardm/nessie
## CLI Usage
> It is suggested you restrict the permissions Nessie has as much as possible,
> to only the permissions its needs. An example of this is:
>
> ```shell
> deno install --unstable --allow-net=: --allow-read=. --allow-write=nessie.config.ts,db -f https://deno.land/x/nessie/cli.ts
> ```
- `init`: Generates a `nessie.config.ts` file and also the `db` folder where
migration and seed files will be placed. Two options are available: `--mode`
and `--dialect`.
- `--mode` can be one of `config` or `folders`. If mode is not set, it will
create a `nessie.config.ts` file and the `db` folder structure, otherwise it
will create the selected one.
- `--dialect` is used for the config file and can be one of `pg`, `mysql` or
`sqlite`. If not set, it will create a general config file including all
three dialects, otherwise it will include only the selected one.
```shell
deno run -A --unstable https://deno.land/x/nessie/cli.ts init
deno run -A --unstable https://deno.land/x/nessie/cli.ts init --mode folders
deno run -A --unstable https://deno.land/x/nessie/cli.ts init --mode config --dialect pg
deno run -A --unstable https://deno.land/x/nessie/cli.ts init --mode config --dialect sqlite
deno run -A --unstable https://deno.land/x/nessie/cli.ts init --mode config --dialect mysql
```
- `make:migration [name]` & `make [name]`: Create migration, `name` has to be
snake- and lowercase, it can also include numbers. You can also provide the
flag `--migrationTemplate ` or use the
`migrationTemplate` property in the config file to tell Nessie which template
to use when generating a new migration.
```shell
deno run -A --unstable https://deno.land/x/nessie/cli.ts make:migration create_users
deno run -A --unstable https://deno.land/x/nessie/cli.ts make create_users
deno run -A --unstable https://deno.land/x/nessie/cli.ts make --migrationTemplate some_custom_template create_users
```
- `make:seed [name]`: Create seed, `name` has to be snake- and lowercase, it can
also include numbers. You can also provide the flag
`--seedTemplate ` or use the `seedTemplate` property in
the config file to tell Nessie which template to use when generating a new
migration.
```shell
deno run -A --unstable https://deno.land/x/nessie/cli.ts make:seed add_users
deno run -A --unstable https://deno.land/x/nessie/cli.ts make:seed --seedTemplate some_custom_template add_users
```
- `migrate [amount?]`: Run migration - will migrate your migrations in your
migration folder (sorted by timestamp) newer than the latest migration in your
db. Amount defines how many migrations, defaults to all available if not set.
```shell
deno run -A --unstable https://deno.land/x/nessie/cli.ts migrate
deno run -A --unstable https://deno.land/x/nessie/cli.ts migrate 1
deno run -A --unstable https://deno.land/x/nessie/cli.ts migrate -c ./nessie.config.ts
```
- `rollback [amount?]`: Rollback - will rollback your migrations. Amount defines
how many migrations, defaults to 1 if not set.
```shell
deno run -A --unstable https://deno.land/x/nessie/cli.ts rollback
deno run -A --unstable https://deno.land/x/nessie/cli.ts rollback 2
deno run -A --unstable https://deno.land/x/nessie/cli.ts rollback all
```
- `seed [matcher?]`: Seed - will seed your database. Optional matcher will match
all files in your seed folder by string literal or RegExp.
```shell
deno run -A --unstable https://deno.land/x/nessie/cli.ts seed
deno run -A --unstable https://deno.land/x/nessie/cli.ts seed seed_file.js
deno run -A --unstable https://deno.land/x/nessie/cli.ts seed ".+.ts"
```
- `status`: Outputs the status of Nessie. Will output detailed information about
current state of the migrations. Includes two flags: `--output` and
`--file-names`.
`--output` sets the output format and can be one of `log` or `json`. If not
set, `log` is chosen by default. If result is used programatically, chose
`json` as the format of `log` can change with every patch version, while the
format of `json` will be considered more stable.
`--file-names` adds the file names to each of the outputs.
```shell
deno run -A --unstable https://deno.land/x/nessie/cli.ts status
totalAvailableMigrationFiles: 1
completedMigrations: 0
newAvailableMigrations: 1
deno run -A --unstable https://deno.land/x/nessie/cli.ts status --output json
{"totalAvailableMigrationFiles":1,"completedMigrations":0,"newAvailableMigrations":1}
deno run -A --unstable https://deno.land/x/nessie/cli.ts status --file-names
totalAvailableMigrationFiles: 1
20210508115213_test1.ts
completedMigrations: 0
newAvailableMigrations: 1
20210508115213_test1.ts
deno run -A --unstable https://deno.land/x/nessie/cli.ts status --file-names --output json
{"totalAvailableMigrationFiles":1,"completedMigrations":0,"newAvailableMigrations":1,"totalAvailableMigrationFileNames":["20210508115213_test1.ts"],"completedMigrationNames":[],"newAvailableMigrationNames":["20210508115213_test1.ts"]}
```
- `update_timestamps`: Update timestamps - will update timestamps to the new
format. Will only update timestamps where the value is less than 1672531200000
(2023-01-01) so that the timestamps won't be updated multiple times.
```shell
deno run -A --unstable https://deno.land/x/nessie/cli.ts update_timestamps
```
### Flags
- `-c, --config`: Path to config file, will default to `./nessie.config.ts`
- `-d, --debug`: Enables verbose output.
- `--migrationTemplate`: Path or URL to a custom migration template. Only used
together with the `make` commands.
- `--seedTemplate`: Path or URL to a custom seed template. Only used together
with the `make` commands.
### Deno flags and Permissions
While the examples simply show `-A` as the permission flag, you can also limit
the permissions according to your needs. Bellow you will see what Nessie
actually needs.
- `--unstable`: Needed by [std/fs/copy](https://deno.land/[email protected]/fs/copy.ts)
as it uses `Deno.utimeSync` and `Deno.utime` which are still unstable.
- `--allow-read`: Nessie needs read access to be able to read the migration and
seed folders, it also checks for the presence of the config file.
- `--allow-write`: When initiating Nessie or creating a new migration or seed
file, write access is required.
- `--allow-net`: This is needed to import the nessie code in your config file.
It is also needed if you use a remote config or migration files, this flag is
needed to get access.
### Config file
The config interface is exported from `mod.ts` as `NessieConfig`.
```ts
export interface NessieConfig {
/** Can be any class which extends `AbstractClient`. */
client: AbstractClient;
/**
* The folders where migration files are located.
* Can be a relative path or an absolute path.
* Defaults to ['./db/migrations/'] if additionalMigrationFiles is not populated
*/
migrationFolders?: string[];
/**
* The folders where seed files are located.
* Can be a relative path or an absolute path.
* Defaults to ['./db/seeds/'] if additionalSeedFiles is not populated
*/
seedFolders?: string[];
/**
* Additional migration files which will be added to the
* list to parse when running the migrate or rollback command.
* Can be any format supported by `import()` e.g. url or path
*/
additionalMigrationFiles?: string[];
/**
* Additional seed files which will be added to the list to
* match against when running the seed command.
* Can be any format supported by `import()` e.g. remote url or path
*/
additionalSeedFiles?: string[];
/** Custom migration template, can be path or url. When also using the CLI flag `--migrationTemplate`, it will have precidence. */
migrationTemplate?: string;
/** Custom seed template, can be path or url. When also using the CLI flag `--seedTemplate`, it will have precidence. */
seedTemplate?: string;
/** Enables verbose output for debugging */
debug?: boolean;
}
```
### Remote Migration or Seed files
With the introduction of `additionalMigrationFiles` and `additionalSeedFiles`,
you can now include remote migration and seed files which you can fetch for
example via ftp or using api's like gihub or gitlab. Any input which can be
given to the dynamic `import()` can be provided.
```ts
// nessie.config.ts
...
additionalMigrationFiles: ['https://example.com/some_migration_file.ts'],
additionalSeedFiles: ['https://example.com/some_seed_file.ts'],
...
```
See the [example folder](./examples) for more examples.
### Custom Migration or Seed templates
As your project grows, or you are starting to have multiple project but want the
same logic across the migrations, you might find it tedious to change the seed
and migration files after creating them. To get around this, you can provide the
options `migrationTemplate` and `seedTemplate` in the config file, or use the
corresponding flags from the command line. There are no restrictions to what the
file has to contain, so you can even provide an empty file if that is your
preferred starting point.
A general usecase for providing custom templates is in the case that you use a
custom `AbstractMigration` or `AbstractSeed` class, and want to use this for all
your future migrations and seeds.
See the [example folder](./examples) for more examples.
## Docker usage
See the specific [Nessie image docs](./image/README.md) for using Nessie with a
docker image.
## Uses
- [Cliffy](https://deno.land/x/cliffy/)
- [Deno Postgres](https://deno.land/x/postgres/)
- [Deno MySQL](https://deno.land/x/mysql/)
- [Deno SQLite](https://deno.land/x/sqlite/)
## Examples
See [example repo](https://github.com/halvardssm/deno-rest-api) for a REST API
which uses Oak and Nessie.
See the [example folder](./examples) for more examples.
Nessie uses the `AbstractMigration` class which you can extend to access the
client and its properties. This enables better flexibility in migrations and
seeds and allows a more complex workflow.
`nessie.config.ts` with all default values
```ts
import {
ClientPostgreSQL,
NessieConfig,
} from "https://deno.land/x/nessie/mod.ts";
const clientPg = new ClientPostgreSQL({
database: "nessie",
hostname: "localhost",
port: 5432,
user: "root",
password: "pwd",
});
const config: NessieConfig = {
client: clientPg,
migrationFolders: ["./db/migrations"],
seedFolders: ["./db/seeds"],
additionalMigrationFiles: [],
additionalSeedFiles: [],
migrationTemplate: undefined,
seedTemplate: undefined,
debug: false,
};
export default config;
```
Minimal example of a migration file
```ts
import {
AbstractMigration,
ClientPostgreSQL,
Info,
} from "https://deno.land/x/nessie/mod.ts";
export default class extends AbstractMigration {
async up({ dialect }: Info): Promise {
await this.client.queryArray("CREATE TABLE table1 (id int)");
}
async down({ dialect }: Info): Promise {
await this.client.queryArray("DROP TABLE table1");
}
}
```
Seed file
```ts
import {
AbstractSeed,
ClientPostgreSQL,
Info,
} from "https://deno.land/x/nessie/mod.ts";
export default class extends AbstractSeed {
async run({ dialect }: Info): Promise {
await this.client.queryArray("INSERT INTO table1 VALUES (1234)");
}
}
```
See the [example folder](./examples) for more
## Clients
Provided clients are
- [PostgreSQL](./clients/ClientPostgreSQL.ts) (also works for CockroachDB)
- [MySQL](./clients/ClientMySQL.ts) & [MySQL 5.5](./clients/ClientMySQL55.ts)
- Currently it works with password for 5.*, but for >=8 you have to send a
blank password, see
[issue 37](https://github.com/manyuanrong/deno_mysql/issues/37)
- [SQLite](./clients/ClientSQLite.ts)
> If you would like to see your DB flavor supported, take a look at how to make
> a client plugin with examples in the [clients folder](./clients) or in the
> next section .
### How to make a client
A client needs to extend [AbstractClient](./clients/AbstractClient.ts).
`query`: Takes a query string or array of query strings and sends them of to the
database for execution. Should return whatever the database responds.
`prepare`: Will be run when the migration or rollback commands are executed.
This should create the connection, set up the `nessie_migrations` table and
prepare the database for incoming migrations.
`migrate`: Takes a number as an optional input, will default to all files if not
set. Will run `Math.min(amount, numberOfFiles)` migration files. Only handles
the `up` method.
`rollback`: Takes a number as an optional input, will default to 1 if not set.
Will run `Math.min(amount, numberOfFiles)` migration files. Only handles the
`down` method.
`seed`: Takes an optional matcher as input. Matcher can be regex or string. Will
seed the database. Handles the `run` method in seed files.
`close`: Will be the last method run before the program is finished. This should
close the database connection.
See the existing examples in the [clients](./clients) folder for reference.
## Migrate from version 1
If you are migrating from version 1, and starting from scratch is not an option,
there are a couple of steps to perform. If you have any questions regarding the
steps, please open a new discussion (not an issue).
> ⚠️ The contributors of Nessie takes no responsibility for any dataloss which
> might occur, so make sure to back up all existing data in your databases
> before migrating and test it locally before executing any changes to your
> production environment.
1. Back up existing data
2. Update the `nessie.config.ts` file to the new syntax (see the
[examples](./examples) folder)
3. (Skip this step if you already updated the timestamps) Run the
`update_timestamps` command from the CLI to update the migration file names
and database entries (only the `nessie_migrations` table). Use the latest
version of Nessie e.g.
```shell
deno run -A --unstable https://deno.land/x/[email protected]/cli.ts update_timestamps
```
4. Update all imports of Nessie to the latest version in your migration and seed
files
5. Update the migration and seed files to use the new format (class based
migration files)
If you come across any issues, you can open a new discussion on GitHub.
## Contributing
All contributions are welcome, make sure to read the
[contribution guideline](./.github/CONTRIBUTING.md).