{"id":19532124,"url":"https://github.com/thetutlage/knex-dynamic-connection","last_synced_at":"2025-04-05T10:08:35.871Z","repository":{"id":35058946,"uuid":"201744544","full_name":"thetutlage/knex-dynamic-connection","owner":"thetutlage","description":"Adds support for dynamically returning connection config for knex queries. Helpful when you want to deal with write/read replicas","archived":false,"fork":false,"pushed_at":"2025-01-31T09:45:41.000Z","size":577,"stargazers_count":39,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"develop","last_synced_at":"2025-03-29T09:09:13.082Z","etag":null,"topics":["database","knexjs"],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thetutlage.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-11T09:32:34.000Z","updated_at":"2025-01-31T09:45:26.000Z","dependencies_parsed_at":"2024-06-14T10:25:57.020Z","dependency_job_id":null,"html_url":"https://github.com/thetutlage/knex-dynamic-connection","commit_stats":{"total_commits":54,"total_committers":4,"mean_commits":13.5,"dds":0.07407407407407407,"last_synced_commit":"55128a6a5c70b9f7975864bc0b0fbaff66c0dbe3"},"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thetutlage%2Fknex-dynamic-connection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thetutlage%2Fknex-dynamic-connection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thetutlage%2Fknex-dynamic-connection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thetutlage%2Fknex-dynamic-connection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thetutlage","download_url":"https://codeload.github.com/thetutlage/knex-dynamic-connection/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247318744,"owners_count":20919484,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","knexjs"],"created_at":"2024-11-11T01:48:58.114Z","updated_at":"2025-04-05T10:08:35.850Z","avatar_url":"https://github.com/thetutlage.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Knex Dynamic Connection\n\nThis module is meant to patch knex and add support for defining dynamic connection configuration.\n\n[![gh-workflow-image]][gh-workflow-url] [![npm-image]][npm-url] ![](https://img.shields.io/badge/Typescript-294E80.svg?style=for-the-badge\u0026logo=typescript)\n\n\u003c!-- START doctoc generated TOC please keep comment here to allow auto update --\u003e\n\u003c!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --\u003e\n## Table of contents\n\n- [Why you need it?](#why-you-need-it)\n- [Is it reliable?](#is-it-reliable)\n- [How does it actually work?](#how-does-it-actually-work)\n- [Will this work in the future?](#will-this-work-in-the-future)\n\n\u003c!-- END doctoc generated TOC please keep comment here to allow auto update --\u003e\n\n## Why you need it?\nKnex.js doesn't have inbuilt support for read/write connections. One can create two seperate instances of knex for read and write, but that still doesn't completely solve the problem. For example:\n\n```js\nconst Knex = require('knex')\n\nconst writeConfig = {\n  client: 'pg',\n  connection: {\n  }\n}\nconst writer = Knex(writeConfig)\n\nconst readConfig = {\n  client: 'pg',\n  connection: {\n  }\n}\nconst reader = Knex(readConfig)\n```\n\nNow, if you want to use multiple servers for read operations, you cannot do that, since knex.js allows only one connection server and will pool connections within that server.\n\n**Following is not possible**\n\n```js\nconst readConfig = {\n  client: 'pg',\n  connection: [\n    {\n      host: ''\n    },\n    {\n      host: ''\n    }\n  ]\n}\n```\n\nWith the help of this module, you can make knex create a connection using the dynamic config for every query.\n\n```sh\nnpm i knex-dynamic-connection\n```\n\n```js\nconst Knex = require('knex')\nconst { patchKnex } = require('knex-dynamic-connection')\n\nconst readConfig = {\n  client: 'pg',\n  connection: {},\n  replicas: [\n    {\n      host: '',\n    },\n    {\n      host: '',\n    }\n  ],\n}\n\nconst knex = Knex(readConfig)\nlet roundRobinCounter = 0\n\npatchKnex(knex, (originalConfig) =\u003e {\n  const node = roundRobinCounter++ % originalConfig.replicas.length\n  return originalConfig.replicas[node]\n})\n```\n\nThe `patchKnex` method overwrites the `acquireRawConnection` on all the dialects and make them fetch the config from your callback vs reading it from a static source.\n\n## Is it reliable?\nYes!\n\n1. I have copied the code of `acquireRawConnection` from the knex codebase and have just made one line of change to read the config from a different source.\n2. I have written tests for `select`, `insert`, `transactions` and `schema` methods.\n3. The code is tested against `mssql`, `mysql`, `mysql2` and `pg`.\n4. The connection is still managed inside the pool, so don't worry about any extra maintaince overhead.\n\n## How does it actually work?\nKnex.js makes use of [tarn.js](https://github.com/vincit/tarn.js/) for managing pool resources and everytime pool needs a connection, knexjs calls [acquireRawConnection](https://github.com/tgriesser/knex/blob/master/lib/client.js#L258) on the dialect in use.\n\nThe dialect creates a new connection to the database server, **but uses static configuration**. I have just added a patch, which will rely on your closure to return the config vs using the same static config all the time.\n\n## Will this work in the future?\nI am using Knex.js to write the ORM of https://adonisjs.com/ and will keep a close eye on the releases of Knex to make sure that I incorporate any changes made of the underlying code and keep this module upto date. If knex.js team plans to re-write the entire codebase (which is less likely to happen), then I will pitch this change to be a first class citizen.\n\n## Can I help?\nYes, there are currently no tests for oracle. It will be great, if you can help set it up using docker\n\n[gh-workflow-image]: https://img.shields.io/github/actions/workflow/status/thetutlage/knex-dynamic-connection/test.yml?style=for-the-badge\n[gh-workflow-url]: https://github.com/thetutlage/knex-dynamic-connection/actions/workflows/test.yml \"Github action\"\n\n[npm-image]: https://img.shields.io/npm/v/knex-dynamic-connection.svg?style=for-the-badge\u0026logo=npm\n[npm-url]: https://npmjs.org/package/knex-dynamic-connection \"npm\"\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthetutlage%2Fknex-dynamic-connection","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthetutlage%2Fknex-dynamic-connection","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthetutlage%2Fknex-dynamic-connection/lists"}