{"id":20947528,"url":"https://github.com/kristijorgji/goseeder","last_synced_at":"2025-05-14T02:30:26.184Z","repository":{"id":37959725,"uuid":"306086662","full_name":"kristijorgji/goseeder","owner":"kristijorgji","description":"Go database seeder inspired from Laravel/Lumen seeder and more","archived":false,"fork":false,"pushed_at":"2024-03-19T17:23:22.000Z","size":27,"stargazers_count":35,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-06-19T18:09:59.051Z","etag":null,"topics":["data","database","go","seeder","seeders","table","test-seeds","testing"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kristijorgji.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-10-21T16:38:56.000Z","updated_at":"2024-06-16T17:51:17.000Z","dependencies_parsed_at":"2024-03-19T18:47:10.416Z","dependency_job_id":null,"html_url":"https://github.com/kristijorgji/goseeder","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kristijorgji%2Fgoseeder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kristijorgji%2Fgoseeder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kristijorgji%2Fgoseeder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kristijorgji%2Fgoseeder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kristijorgji","download_url":"https://codeload.github.com/kristijorgji/goseeder/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225270633,"owners_count":17447635,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data","database","go","seeder","seeders","table","test-seeds","testing"],"created_at":"2024-11-19T00:11:54.210Z","updated_at":"2024-11-19T00:11:54.793Z","avatar_url":"https://github.com/kristijorgji.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# goseeder\n\n![GitHub Workflow Status](https://github.com/kristijorgji/goseeder/workflows/CI/badge.svg)\n[![GoDev](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go\u0026logoColor=white\u0026style=flat-square)](https://pkg.go.dev/github.com/kristijorgji/goseeder?tab=doc)\n[![codecov](https://img.shields.io/codecov/c/github/kristijorgji/goseeder/badge.svg)](https://codecov.io/gh/kristijorgji/goseeder)\n[![Go Report Card](https://goreportcard.com/badge/kristijorgji/goseeder)](https://goreportcard.com/report/kristijorgji/goseeder)\n[![Sourcegraph](https://sourcegraph.com/github.com/kristijorgji/goseeder/-/badge.svg)](https://sourcegraph.com/github.com/kristijorgji/goseeder?badge)\n\n#### Motivation\nGolang is a great language and getting better as community and frameworks, but there are still a lot of pieces missing for developing fast, accurate way and avoiding repetitions.\n \nI was searching for a go seeder similar to the one that Laravel/Lumen provides and could not find one.\n\nKnowing that this is such an important key element of any big project for testing and seeding projects with dummy data I decided to create one myself and share.\n\n#### Features\nFor now the library supports only MySql as a database driver for its utility functions like `FromJson`\nand `FromJsonIntoTable` provided by `Seeder` struct, but it is db agnostic for your custom seeders you can use any\ndatabase that is supported by `sql.DB`\n\n`goseeder`\n1. It is designed for different kind of usages, for both programmatically or building into your exe and run via cli args\n2. Allows specifying seeds for different environments such as predefined test and custom specified envs by the user\n3. Allows specifying list (or single) seed name for execution\n4. Allows having common seeds that execute for every env, unless specified not to do so with the respective cli or option\n5. Provides out of the box functions like `(s Seeder) FromJson` to seed the table from json data and more data formats and drivers coming soon\n\n# Table of Contents\n\n- [Installation](#installation)\n- [Usage method 1: Turn your executable into seedable via cli args](#usage-method-1-turn-your-executable-into-seedable-via-cli-args)\n    - [1. Change Your Main Function](#1-change-your-main-function)\n    - [2. Registering Your Seeds](#2-registering-your-seeds)\n    - [3. Run Seeds Only For Specific Env](#3-run-seeds-only-for-specific-env)\n    - [4. Run Seeds By Name](#4-run-seeds-by-name)\n- [Usage method 2: Programmatically](#usage-method-2-programmatically)\n- [Summary Of Cli Args](#summary-of-cli-args)\n- [License](#license)\n\n## Installation\n\n```sh\ngo get github.com/kristijorgji/goseeder\n```\n\n## Usage method 1: Turn your executable into seedable via cli args\n\n[Please check examples/simpleshop](examples/simpleshop) for a full working separate go project that uses the seeder\n\nBelow I will explain once more all the steps needed to have goseeder up and running for your project.\n\n### 1. Change Your Main Function\n\nIn order to give your executable seeding abilities and support for its command line arguments, the first thing we have to do is to wrap our main function\nwith the provided `goseeder.WithSeeder`\n\n```go\nfunc WithSeeder(conProvider func() *sql.DB, clientMain func())\n```\n\nThe function requires as argument \n1. one function that returns a db connection necessary to seed\n2. your original main function, which will get executed if no seed is requested\n\nOne such main file can look like below:\n\n```go\n// main.go\npackage main\n\nimport (\n\t\"database/sql\"\n\t\"fmt\"\n\t_ \"github.com/go-sql-driver/mysql\"\n\t\"github.com/joho/godotenv\"\n\t\"github.com/kristijorgji/goseeder\"\n\t\"log\"\n\t\"net/url\"\n\t\"os\"\n\t_ \"simpleshop/db/seeds\"\n)\n\nfunc main() {\n\terr := godotenv.Load()\n\tif err != nil {\n\t\tlog.Panic(\"Error loading .env file\")\n\t}\n\n\tgoseeder.WithSeeder(connectToDbOrDie, func() {\n\t\tmyMain()\n\t})\n}\n\nfunc myMain() {\n\tfmt.Println(\"Here you will execute whatever you were doing before using github.com/kristijorgji/goseeder like start your webserver etc\")\n}\n\nfunc connectToDbOrDie() *sql.DB {\n\tdbDriver := os.Getenv(\"DB_DRIVER\")\n\tdbHost := os.Getenv(\"DB_HOST\")\n\tdbPort := os.Getenv(\"DB_PORT\")\n\tdbName := os.Getenv(\"DB_DATABASE\")\n\tdbUser := os.Getenv(\"DB_USERNAME\")\n\tdbPassword := os.Getenv(\"DB_PASSWORD\")\n\n\tdbSource := fmt.Sprintf(\n\t\t\"%s:%s@tcp(%s:%s)/%s?parseTime=true\",\n\t\tdbUser,\n\t\turl.QueryEscape(dbPassword),\n\t\tdbHost,\n\t\tdbPort,\n\t\tdbName,\n\t)\n\tcon, err := sql.Open(dbDriver, dbSource)\n\tif err != nil {\n\t\tlog.Fatalf(\"Error opening DB: %v\", err)\n\t}\n\n\treturn con\n}\n\n```\n\n### 2. Registering Your Seeds\n\nGreat! After step 1 our executable is able to run in seed mode or default mode.\n\nNow we want to know how to register our custom seeds.\n\nIf you look at the imports in main file from step one, we might notice that we import\n`_ \"simpleshop/db/seeds\"` even though we do not use them directly.\n\nThis is mandatory because our seeds will get registered during package initialisation as we will see later.\n\nThe recommended project folder structure to work properly with `goseeder` is to have the following path for the\nseeders `db/seeds` and the package name to be `seeds`\n\nInside the folder you can add your seeders, for example lets seed some data into the `categories` table from one json\nfile located at `db/seeds/data/categories.json`.\n\nTo do that we create our `categories.go` file at `db/seeds` folder:\n\n```go\n// db/seeds/categories.go\npackage seeds\n\nimport (\n\t\"github.com/kristijorgji/goseeder\"\n)\n\nfunc categoriesSeeder(s goseeder.Seeder) {\n\ts.FromJson(\"categories\")\n}\n\n```\n\nTo use this seed, the last step is to register it.\n\nSeeds can be registered as:\n1. `common` seeds that run for all environments\n2. for a specific environment like `test`, `yourcustomenv` (more in step 3)\n\nWe are going to create below a seed that runs for all environments, so we will not specify any env while registering it.\n\nTo do that we create in the `db/seeds` folder the file `common.go` that will register seeds that get always executed\nregardless of the environment:\n\n```go\n// db/seeds/common.go\npackage seeds\n\nimport \"github.com/kristijorgji/goseeder\"\n\nfunc init() {\n\tgoseeder.Register(categoriesSeeder)\n}\n```\n\nWe used `goseeder.Register` to register our seed function to run for all environments.\n\nThat is all for the basic usage of goseeder!!!\n\nOur function in categories.go file is now registered and ready to be used.\n\nNow you can run\n\n```\ngo run main.go --gseed\n```\n\nand it will run all your seeds against the provided db connection.\n\nThe framework will look for `categories.json` file in the path `db/seeds/data`, and insert all the entries there in a\ntable named `categories` (inferred from the file name)\n\nIf you have a seed registered for another environment, for example a test seed, the framework instead will look for the\njson file at `db/seeds/data/test`\n\nSo the rule is it will always lookup in this pattern `db/seeds/data/[environment]/[specifiedFileName].[type]`\n\nYou can also give a seed a custom name, if you do not want the function name to be used by default.\nYou can register a seed in a fully flexible way like:\n\n```go\n// db/seeds/common.go\npackage seeds\n\nimport \"github.com/kristijorgji/goseeder\"\n\nfunc init() {\n    Registration{\n\t\tName: \"another_name_for_cat_seeder\",\n\t\tEnv:  \"stage\",\n\t}.Complete(categoriesSeeder)\n}\n```\n\n### 3. Run Seeds Only For Specific Env\n\nMany times we want to have seeds only for `test` environment, test purpose and want to avoid having thousand of randomly\ngenerated rows inserted into production database by mistake!\n\nOr we just want to have granular control, to have separate data to populate our app/web in different way\nfor `staging` `prod` `yourcustomenv` and so on.\n\ngoseeder is designed to take care of this by using one of the following methods:\n\n- `goseeder.RegisterForTest(seeder func(s Seeder)` - registers the specified seed for the env named `test`\n- `goseeder.RegisterForEnv(env string, seeder func(s Seeder))` - will register your seeder to be executed only for the\n  custom specified env\n- `goseeder.RegisterForEnvNamed(env string, seeder func(s Seeder), name string)` - will register your seeder to be\n  executed only for the custom specified env with the given name. That name is used during progress output and in case\n  of errors\n\nLet's add to our previous categories.go seeder one seed function specific only for test env!\nThe file now will look like:\n\n```go\npackage seeds\n\nimport (\n\t\"fmt\"\n\t\"github.com/kristijorgji/goseeder\"\n\t\"simpleshop/util\"\n)\n\nfunc categoriesSeeder(s goseeder.Seeder) {\n\ts.FromJson(\"categories\")\n}\n\nfunc testCategoriesSeeder(s goseeder.Seeder) {\n\tfor i := 0; i \u003c 100; i++ {\n\t\tstmt, _ := s.DB.Prepare(`INSERT INTO categories(id, name) VALUES (?,?)`)\n\t\t_, err := stmt.Exec(util.RandomInt(1, int64(^uint16(0))), []byte(fmt.Sprintf(`{\"en\": \"%s\"}`, util.RandomString(7))))\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t}\n}\n\n```\n\nFinally, lets create our registrator file for all test seeds same way as we did with `common.go`, we will\ncreate `test.go` now with content as below:\n\n```go\n// db/seeds/test.go\npackage seeds\n\nimport \"github.com/kristijorgji/goseeder\"\n\nfunc init() {\n\tgoseeder.RegisterForTest(testCategoriesSeeder)\n}\n\n```\n\nThat is all!\n\nNow if you run your app without specifying test env, only the common env seeders will run and you cannot mess by mistake\nproduction or other environments!\n\nTo run the test seeder above you have to run:\n\n```bash\ngo run main.go --gseed --gsenv=test\n```\n\nThis will run only the tests registered for the env `test` and the `common` seeds. A seed is known as common if it is\nregistered without environment via `Register` method and has empty string env.\n\nIf you do not want common seeds to get executed, just specify the flag `--gs-skip-common`\n\nThe above call to run only seeds for test `env`, and ignore the common ones then would be:\n\n```bash\ngo run main.go --gseed --gsenv=test --gs-skip-common\n```\n\n### 4. Run Seeds By Name\n\nWhen we register a seed like shown in step 2, the seed name is the same as the function name, so our seed is\ncalled `categoriesSeeder` because that is the name of the function we register below\n\n```go\nfunc init() {\n\tgoseeder.Register(categoriesSeeder)\n}\n```\n\nThis is important because we are totally flexible and can do cool things like execute only the specific seed functions\nthat we want!\n\nLet's assume that we have 100 seed functions, and want to execute only one of them which is named categoriesSeeder (that\nwe registered above) and ignore all the other seeds.\n\nEasy as this, just run:\n\n```bash\ngo run main.go --gseeder --gsnames=categoriesSeeder\n```\n\nIf you want to execute multiple seeds by specifying their names, just use comma separated value\nlike `--gsnames=categoriesSeeder,someOtherSeeder`\n\n## Usage method 2: Programmatically\n\n`goseeder` is designed to fit all needs for being the best seeding tool.\n\nThat means that you might want to seed data before your unit tests programmatically without using cli args.\n\nThat is straightforward to do with goseeder.\nLet us assume we want to test our api that connects to some database in the package `api`,\n\nThe file `api/main_test.go` might look like below:\n\n```go\n//api/main_test.go\npackage api\n\nimport (\n\t\"database/sql\"\n\t_ \"db/seeds\" // please import your seeds package so they register or register programatically here too if you want before the seeder Execute is called\n\t\"log\"\n\t\"os\"\n\t\"testing\"\n    \"github.com/kristijorgji/goseeder\"\n)\n\n\nfunc TestMain(m *testing.M) {\n\tcon := db.ConnectToTestDb()\n\tSeedTestData(con)\n\tr := m.Run()\n\tos.Exit(r)\n}\n\nfunc SeedTestData(con *sql.DB) {\n\tlog.Println(\"Seeding test database\")\n\tgoseeder.SetDataPath(\"../db/seeds/data\")\n\terr := goseeder.Execute(con, goseeder.ForEnv(\"test\"), goseeder.ShouldSkipCommon(true))\n\tif err != nil {\n\t\tlog.Fatal(\"Seeding test data failed\\n\")\n\t\tos.Exit(-2)\n\t}\n}\n```\n\nHow nice is that ?!\n\nAfter the execution you have a database with data sourcing only from your seeds registered for test env (test seeds) !!!\nThe above is production code used by one company, but you might need to adjust to your needs.\n\nAnother common use case is to want to execute programmatically the seeder because you don't want to turn your executable\ninto seedable (you don't want to use method 1)).\n\nThen again you can just create another file `myseeder.go` and inside it do your custom logic or handling of args then\njust execute\n`goseeder.Execute`\n\nYour `myseeder.go` might look like\n\n```go\npackage main\n\nimport (\n\t_ \"db/seeds\"\n\t\"github.com/kristijorgji/goseeder\"\n)\n\nfunc main() {\n\terr := goseeder.Execute(connectToDbOrDie())\n    if err != nil {\n        log.Fatal(\"Seeding test data failed\\n\")\n        os.Exit(-2)\n    }\n}\n\n// your func here to connect to db \n// connectToDbOrDie\n\n```\n\nThen you have your server or app executable separate for example in `main.go` file, and the seeder functionality\nseparated in `myseeder.go`\n\nYou can easily run your seeder `go run myseeder.go`, or build and run etc based on your requirements.\n\nYou can pass all the necessary options to the  `goSeeder.Execute` method.\nIf you want to execute seeders for a particular env only (skip common seeds) for example you do it like:\n\n```go\ngoseeder.Execute(con, goseeder.ForEnv(\"test\"), goseeder.ShouldSkipCommon(true))\n```\n\nThese are the possible options you can give after the mandatory db connection:\n- `ForEnv(env string)` - you can specify here the env for which you want to execute\n- `ForSpecificSeeds(seedNames []string)` - just specify array of seed names you want to execute\n- `ShouldSkipCommon(value bool)` - this option has effect only if also gsenv if set, then will not run the common\n  seeds (seeds that do not have any env specified\n\n## Summary Of Cli Args\n\nYou can always run\n\n```bash\nrun go run main.go --help\n````\n\nto see all the available arguments and their descriptions.\n\nFor the current version the result is:\n\n```bash\nINR00009:simpleshop kristi.jorgji$ go run main.go --help\nUsage of /var/folders/rd/2bkszcpx6xgcddpn7f3bhczn1m9fb7/T/go-build358407825/b001/exe/main:\n  -gs-skip-common\n        goseeder - this arg has effect only if also gsenv if set, then will not run the common seeds (seeds that do not have any env specified)\n  -gseed\n        goseeder - if set will seed\n  -gsenv string\n        goseeder - env for which seeds to execute\n  -gsnames string\n        goseeder - comma separated seeder names to run specific ones\n```\n\n## A note on `common` seeds\n\n`common` is presented as one environment, but it is not such.\nIt is a special way of executing particular seeds always! in all environments, together with respective env seeds.\n\nIf you want to skip common executions, the means to do so are provided already in this documentation.\nFrom cli with the flag\n\n```\n-gs-skip-common\n```\n\nand programmatically call `ShouldSkipCommon(true)`, like simple example:\n\n```\nerr := goseeder.Execute(con, goseeder.ForEnv(\"test\"), goseeder.ShouldSkipCommon(true))\n```\n\n## License\n\ngoseeder is released under the MIT Licence @kristijorgji. See the bundled LICENSE file for details.\n\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkristijorgji%2Fgoseeder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkristijorgji%2Fgoseeder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkristijorgji%2Fgoseeder/lists"}