Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/go-gorm/dbresolver

Multiple databases, read-write splitting FOR GORM
https://github.com/go-gorm/dbresolver

Last synced: 5 days ago
JSON representation

Multiple databases, read-write splitting FOR GORM

Awesome Lists containing this project

README 

        
# DBResolver



DBResolver adds multiple databases support to GORM, the following features are supported:



* Multiple sources, replicas

* Read/Write Splitting

* Automatic connection switching based on the working table/struct

* Manual connection switching

* Sources/Replicas load balancing

* Works for RAW SQL

* Transaction



## Quick Start



```go

import (

  "gorm.io/gorm"

  "gorm.io/plugin/dbresolver"

  "gorm.io/driver/mysql"

)



DB, err := gorm.Open(mysql.Open("db1_dsn"), &gorm.Config{})



DB.Use(dbresolver.Register(dbresolver.Config{

  // use `db2` as sources, `db3`, `db4` as replicas

  Sources:  []gorm.Dialector{mysql.Open("db2_dsn")},

  Replicas: []gorm.Dialector{mysql.Open("db3_dsn"), mysql.Open("db4_dsn")},

  // sources/replicas load balancing policy

  Policy: dbresolver.RandomPolicy{},

  // print sources/replicas mode in logger

  ResolverModeReplica: true,

}).Register(dbresolver.Config{

  // use `db1` as sources (DB's default connection), `db5` as replicas for `User`, `Address`

  Replicas: []gorm.Dialector{mysql.Open("db5_dsn")},

}, &User{}, &Address{}).Register(dbresolver.Config{

  // use `db6`, `db7` as sources, `db8` as replicas for `orders`, `Product`

  Sources:  []gorm.Dialector{mysql.Open("db6_dsn"), mysql.Open("db7_dsn")},

  Replicas: []gorm.Dialector{mysql.Open("db8_dsn")},

}, "orders", &Product{}, "secondary"))

```



### Automatic connection switching



DBResolver will automatically switch connections based on the working table/struct



For RAW SQL, DBResolver will extract the table name from the SQL to match the resolver, and will use `sources` unless the SQL begins with `SELECT`, for example:



```go

// `User` Resolver Examples

DB.Table("users").Rows() // replicas `db5`

DB.Model(&User{}).Find(&AdvancedUser{}) // replicas `db5`

DB.Exec("update users set name = ?", "jinzhu") // sources `db1`

DB.Raw("select name from users").Row().Scan(&name) // replicas `db5`

DB.Create(&user) // sources `db1`

DB.Delete(&User{}, "name = ?", "jinzhu") // sources `db1`

DB.Table("users").Update("name", "jinzhu") // sources `db1`



// Global Resolver Examples

DB.Find(&Pet{}) // replicas `db3`/`db4`

DB.Save(&Pet{}) // sources `db2`



// Orders Resolver Examples

DB.Find(&Order{}) // replicas `db8`

DB.Table("orders").Find(&Report{}) // replicas `db8`

```



### Read/Write Splitting



Read/Write splitting with DBResolver based on the current using [GORM callback](https://gorm.io/docs/write_plugins.html).



For `Query`, `Row` callback, will use `replicas` unless `Write` mode specified

For `Raw` callback, statements are considered read-only and will use `replicas` if the SQL starts with `SELECT`



### Manual connection switching



```go

// Use Write Mode: read user from sources `db1`

DB.Clauses(dbresolver.Write).First(&user)



// Specify Resolver: read user from `secondary`'s replicas: db8

DB.Clauses(dbresolver.Use("secondary")).First(&user)



// Specify Resolver and Write Mode: read user from `secondary`'s sources: db6 or db7

DB.Clauses(dbresolver.Use("secondary"), dbresolver.Write).First(&user)

```



### Transaction



When using transaction, DBResolver will keep using the transaction and won't switch to sources/replicas based on configuration



But you can specifies which DB to use before starting a transaction, for example:



```go

// Start transaction based on default replicas db

tx := DB.Clauses(dbresolver.Read).Begin()



// Start transaction based on default sources db

tx := DB.Clauses(dbresolver.Write).Begin()



// Start transaction based on `secondary`'s sources

tx := DB.Clauses(dbresolver.Use("secondary"), dbresolver.Write).Begin()

```



### Load Balancing



GORM supports load balancing sources/replicas based on policy, the policy is an interface implements following interface:



```go

type Policy interface {

	Resolve([]gorm.ConnPool) gorm.ConnPool

}

```



Currently only the `RandomPolicy` implemented and it is the default option if no policy specified.



### Connection Pool



```go

DB.Use(

  dbresolver.Register(dbresolver.Config{ /* xxx */ }).

  SetConnMaxIdleTime(time.Hour).

  SetConnMaxLifetime(24 * time.Hour).

  SetMaxIdleConns(100).

  SetMaxOpenConns(200)

)

```