https://github.com/csmadhu/gob

Bulk Inserts to Postgres, MySQL, Cassandra
https://github.com/csmadhu/gob

cassandra golang mysql postgresql

Last synced: 2 months ago
JSON representation

Bulk Inserts to Postgres, MySQL, Cassandra

• Host: GitHub
• URL: https://github.com/csmadhu/gob
• Owner: csmadhu
• License: gpl-3.0
• Created: 2020-08-20T06:03:53.000Z (almost 6 years ago)
• Default Branch: master
• Last Pushed: 2020-09-15T19:08:59.000Z (almost 6 years ago)
• Last Synced: 2025-03-16T16:44:11.185Z (over 1 year ago)
• Topics: cassandra, golang, mysql, postgresql
• Language: Go
• Homepage:
• Size: 92.8 KB
• Stars: 3
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# gob

Bulk upserts to PostgreSQL, MySQL, Cassandra

[![Go Report Card](https://goreportcard.com/badge/github.com/csmadhu/gob)](https://goreportcard.com/report/github.com/csmadhu/gob)

[![GoDoc](https://godoc.org/github.com/csmadhu/gob?status.svg)](https://pkg.go.dev/github.com/csmadhu/gob?tab=doc)

[![Conventional Commit](https://img.shields.io/badge/Conventional%20Commits-1.0.0-yellow.svg)](https://conventionalcommits.org)

[![Gitmoji](https://img.shields.io/badge/gitmoji-%20😜%20😍-FFDD67.svg?style=flat-square)](https://gitmoji.carloscuesta.me)

[![Go](https://github.com/csmadhu/gob/workflows/Go/badge.svg)](https://github.com/csmadhu/gob/actions)

![Supported Go Versions](https://img.shields.io/badge/Go-1.15+-lightgrey.svg)

[![GitHub Release](https://img.shields.io/github/release/csmadhu/gob.svg)](https://github.com/csmadhu/gob/releases)

[![License](https://img.shields.io/badge/license-GPL%20(%3E%3D%202)-blue)](https://github.com/csmadhu/gob/blob/master/LICENSE)

---------------------------------------

  * [Installation](#installation)

  * [Usage](#usage)

  * [Options](#options)

  * [Examples](#examples)

---------------------------------------

## Installation

```bash

go get -u github.com/csmadhu/gob

```

# Usage

```go

package main

import (

	"context"

	"fmt"

	"log"

	"github.com/csmadhu/gob"

)

func main() {

	g, err := gob.New(gob.WithDBProvider(gob.DBProviderPg),

		gob.WithDBConnStr("postgres://postgres:postgres@localhost:5432/gob?pool_max_conns=1"))

	if err != nil {

		log.Fatalf("init gob; err: %v", err)

	}

	defer g.Close()

	// upsert records to table student

	var rows []gob.Row

	for i := 0; i < 10; i++ {

		row := gob.NewRow()

		row.Add("name", fmt.Sprintf("foo-%d", i))

		row.Add("age", 20)

		rows = append(rows, row)

	}

	if err := g.Upsert(context.Background(), gob.UpsertArgs{

		Model:          "students",

		Keys:           []string{"name"},

		ConflictAction: gob.ConflictActionUpdate,

		Rows:           rows}); err != nil {

		log.Fatalf("upsert students; err: %v", err)

	}

}

```

## Options

All options are optional. Options not applicable to Database provider is ignored.

	

		option

		description

		type

		default

		DB providers

	

	

		WithBatchSize

		Transaction Batch Size

		int

		10000

		


PostgreSQL

MySQL



	

	

		WithDBProvider

		Database provider

		gob.DBProvider

		DBProviderPg

		

			


				
PostgreSQL

				
MySQL

				
Cassandra

			

		

	

	

		WithDBConnStr

		DB URL/DSN

			


				
PostgreSQL postgres://username:password@host:port/batabase


					References

						


							
https://pkg.go.dev/github.com/jackc/pgconn?tab=doc#ParseConfig

							
https://pkg.go.dev/github.com/jackc/pgx/v4?tab=doc#ParseConfig

						

				

				
MySQL username:password@(host:port)/database



				
Cassandra cassandra://username:password@host1--host2--host3:port/keyspace?consistency=quorum&compressor=snappy&tokenAware=true


					References

						


						
https://godoc.org/github.com/gocql/gocql#Consistency

						
https://godoc.org/github.com/gocql/gocql#Compressor

						
https://godoc.org/github.com/gocql/gocql#PoolConfig

						
https://godoc.org/github.com/gocql/gocql#HostSelectionPolicy

						
https://godoc.org/github.com/gocql/gocql#TokenAwareHostPolicy

						

				

			

		

		string

		postgres://postgres:postgres@localhost:5432/gob?pool_max_conns=1

		

			


				
PostgreSQL

				
MySQL

				
Cassandra

			

		

	

	

		WithConnIdleTime

		Maximum amount of time conn may be idle

		time.Duration

		3 second

		

			


				
PostgreSQL

				
MySQL

			

		

	

	

		WithConnLifeTime

		Maximum amount of time conn may be reused

		time.Duration

		3 second

		

			


				
PostgreSQL

				
MySQL

				
Cassandra

			

		

	

	

		WithIdleConns

		Maximum number of connections idle in pool

		int

		2

		

			


				
PostgreSQL

				
MySQL

			

		

	

	

		WithOpenConns

		Maximum number of connections open to database

		int

		10

		

			


				
PostgreSQL

				
MySQL

				
Cassandra

			

		

	


## Examples

Examples for supported Database provider can be found [here](https://github.com/csmadhu/gob/tree/master/examples)