{"id":29710664,"url":"https://github.com/fastbill/go-service-toolkit","last_synced_at":"2025-10-03T14:34:36.666Z","repository":{"id":35012773,"uuid":"171499208","full_name":"fastbill/go-service-toolkit","owner":"fastbill","description":"A collection of opiniated packages to quickly set up (micro-)services in Go","archived":false,"fork":false,"pushed_at":"2024-06-18T11:41:21.000Z","size":434,"stargazers_count":0,"open_issues_count":3,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-06-21T01:42:23.191Z","etag":null,"topics":["database-setup","go","golang","labstack-echo","logging","microservice","observability"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fastbill.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-19T15:32:17.000Z","updated_at":"2024-06-18T11:41:25.000Z","dependencies_parsed_at":"2024-06-21T00:24:40.573Z","dependency_job_id":"414968b0-e3ca-46c4-aa14-02f2d7d42109","html_url":"https://github.com/fastbill/go-service-toolkit","commit_stats":null,"previous_names":[],"tags_count":29,"template":false,"template_full_name":null,"purl":"pkg:github/fastbill/go-service-toolkit","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fastbill%2Fgo-service-toolkit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fastbill%2Fgo-service-toolkit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fastbill%2Fgo-service-toolkit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fastbill%2Fgo-service-toolkit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fastbill","download_url":"https://codeload.github.com/fastbill/go-service-toolkit/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fastbill%2Fgo-service-toolkit/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266753988,"owners_count":23979145,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-23T02:00:09.312Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database-setup","go","golang","labstack-echo","logging","microservice","observability"],"created_at":"2025-07-23T21:39:18.457Z","updated_at":"2025-10-03T14:34:36.574Z","avatar_url":"https://github.com/fastbill.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Service Toolkit [![Build Status](https://travis-ci.com/fastbill/go-service-toolkit.svg?branch=master)](https://travis-ci.com/fastbill/go-service-toolkit) [![Go Report Card](https://goreportcard.com/badge/github.com/fastbill/go-service-toolkit)](https://goreportcard.com/report/github.com/fastbill/go-service-toolkit) [![GoDoc](https://godoc.org/github.com/fastbill/go-service-toolkit?status.svg)](https://godoc.org/github.com/fastbill/go-service-toolkit)\n\nThe service toolkit bundles configuration manangement, setting up logging, ORM, REDIS cache and configuring the web framework. It uses opinionated (default) settings to reduce the amount of boilerplate code needed for these tasks. With the toolkit a new Go mircoservice can be set up very quickly.\n\nSee [main.go in the example folder](https://github.com/fastbill/go-service-toolkit/blob/master/example/main.go) for a full, working example.\n\n# Configuration and Environment Variables\nFollowing the [12-Factor App Guideline](https://12factor.net/config) our service retrieves its configuration from the environment variables. To avoid having to pass a lot of variables that change rarely or never, we keep most values in `.env` files that are then loaded\ninto environment variables by the envloader package. Values from these files serve as default and are overwritten by values from the environment.\n\nYou need to tell the envloader in which folder to look for the `.env` files. By default it will only load the `prod.env` file from that folder. If the environment variable `ENV` is set to e.g. `dev` the the loader will load `dev.env` first and only load additional values not set in there from `prod.env`.\n\n## Usage\n```go\nimport (\n\ttoolkit \"github.com/fastbill/go-service-toolkit/v4\"\n)\n\n\nfunc main() {\n    // ATTENTION: This needs to be called before any other function from the toolkit is used to ensure the environment variables are correct.\n    tookit.MustLoadEnvs(\"config\")\n}\n```\n\n# Observability - Logging and Metrics\nWe bundle logging and capturing custom metrics in one `Obs` struct (short for observance). In the future tools for tracing might also be added. Due to the bundling only one struct needs to be passed around in the application and not 2 or 3. Additionally the observance struct provides a method to create request specific observance instances that automatically add full url, method and request id to every log message created with that instance. It also adds the request headers specified via `LoggedHeaders` to the logger with the given field name when the method `CopyWithRequest` is used.\n\nWe use [Logrus](https://github.com/sirupsen/logrus) as logger under the hood but it is wrapped with a custom interface so we do not depend directly on the interface provided by Logrus. Logs will be written to StdOut in JSON format. If you pass a Sentry URL and version all log entries with level error or higher will be pushed to Sentry. This is done via hooks in Logrus.\n\nThe `Obs` struct has a `PanicRecover` method that can be used as deferred function in your setup. It will log the stack trace in case a panic happens in the main Goroutine.\n\n## Usage\n```go\nimport (\n\t\"time\"\n\n\t\"github.com/fastbill/go-service-toolkit/v4\"\n)\n\nfunc main() {\n\tobsConfig := toolkit.ObsConfig{\n\t\tAppName:              \"my-test-app\",\n\t\tLogLevel:             \"debug\", // required\n\t\tSentryURL:            \"https://xyz:xyz@sentry.com/123\",\n\t\tVersion:              \"1.0.0\",\n\t\tMetricsURL:           \"http://example.com\",\n\t\tMetricsFlushInterval: 1 * time.Second,\n\t\tLoggedHeaders: map[string]string{\n\t\t\t\"FastBill-RequestId\": \"requestId\",\n\t\t},\n\t}\n\n\tobs := toolkit.MustNewObs(obsConfig)\n\tdefer obs.PanicRecover()\n}\n```\n\nThe request specific observance is best created in a middleware function that creates a custom context like this:\n```go\nfunc SetupCustomContext(obs *observance.Observance) echo.MiddlewareFunc {\n\treturn func(next echo.HandlerFunc) echo.HandlerFunc {\n\t\treturn func(c echo.Context) error {\n\t\t\treturn next(\u0026context{\n\t\t\t\tc,\n\t\t\t\tobs.CopyWithRequest(c.Request()),\n\t\t\t})\n\t\t}\n\t}\n}\n```\n\nFor testing there is a test logger provided. See the example [here](https://godoc.org/github.com/fastbill/go-service-toolkit/observance#example-NewTestLogger) to find out how to use it.\n\nTODO: Add metrics usage example\n\n# Database\nThe toolkit allows to set up the database (MySQL or PostgreSQL). The `MustSetupDB` includes the following things:\n* Create a database connection\n* Check it works via sending a ping\n* Create the database with the given name in case it did not exist yet\n* Set up the ORM: [GORM](http://gorm.io/)\n\nAdditionally `MustEnsureDBMigrations` runs all migrations from the given folder that are missing so far. For that, the package [migrate](https://github.com/golang-migrate/migrate) is used.\n\n## Usage\n```go\nimport (\n    \"github.com/fastbill/go-service-toolkit/v4\"\n)\n\nfunc main() {\n\tdbConfig := toolkit.DBConfig{\n\t\tDialect:  \"mysql\",\n\t\tHost:     \"localhost\",\n\t\tPort:     \"3306\",\n\t\tUser:     \"root\",\n\t\tPassword: \"***\",\n\t\tName:     \"test-db\",\n\t}\n\tdb := toolkit.MustSetupDB(dbConfig, obs.Logger)\n\tdefer func() {\n\t\tif err := db.Close(); err != nil {\n\t\t\t// log the error\n\t\t}\n\t}()\n\n\ttoolkit.MustEnsureDBMigrations(\"migrations\", dbConfig)\n}\n```\n\n# Redis Cache\nThe function `MustNewCache` sets up a new REDIS client. A prefix can be provided that will be added to all keys. The client includes methods to work with JSON data.\n\n## Usage\n```go\nimport (\n    \"github.com/fastbill/go-service-toolkit/v4\"\n)\n\nfunc main() {\n\tcache := toolkit.MustNewCache(\"localhost\", \"6400\", \"testPrefix\")\n\tdefer func() {\n\t\tif err := cache.Close(); err != nil {\n\t\t\t// log the error\n\t\t}\n\t}()\n}\n```\n\n# Server\nThe server package sets up an [Echo](https://echo.labstack.com/) server that includes graceful shutdown, timeouts, CORS, an error handler that can handle [HTTPErrors](https://github.com/fastbill/httperrors) etc. The individual features are described below.\n\n## Usage\n```go\nimport (\n    \"github.com/fastbill/go-service-toolkit/v4/server\"\n)\n\nfunc main() {\n    echoServer, connectionsClosed := server.New(obs, \"https://example.com\", \"1m\")\n\t\n\t// Set up routes etc.\n\n\terr := echoServer.Start(\":8080\")\n\tif err != nil {\n\t\tobs.Logger.Warn(err)\n\t}\n    \u003c-connectionsClosed\n}\n```\n\n## CORS\nWhen setting up the server via `New` the second argument defines the CORS `AllowOrigins` value. Multiple URLs can be passed as comma separated string. If an empty string is passed, no CORS middleware is applied and same-origin restrictions apply.\n\n## Timeout\nWhen setting up the server via `New` the third argument is optional and can contain a timeout duration in the format described [here](https://golang.org/pkg/time/#ParseDuration). If it is ommited a default timeout of 30 seconds is applied for all connections. The timeout applies to reading headers, reading the request and writing the response.\n\n## Graceful Shutdown\nWhen the application receives `SIGINT` or `SIGTERM` a shutdown procedure is initated. The server does not accept new connections and waits for a maximum of 9 seconds for the ongoining requests to be finished. As soon as all HTTP connections are closed the server is shut down. For this graceful shutdown to work correctly, you need to wait for the provided channel to be closed at the end of your main Goroutine as shown below, otherwise the program will completely terminate before the graceful shutdown was completed.\n\n## Parsing and Validating JSON\nThe default configuration includes a custom `Bind` method for the context object that performs the [default Echo `Bind`](https://echo.labstack.com/guide/request) that parses the JSON request but also validates the input struct via [github.com/go-playground/validator](https://github.com/go-playground/validator) in case the struct definition includes the respective validation tags.\n\n## Error Handling and Logging\nWhen an error is returned from an Echo HTTP handler it will encounter a custom error handler that was added to the server. If the error is an [HTTPError](https://github.com/fastbill/httperrors) or one of Echos own HTTP errors it will not be logged. The response will contain the status code and body specified by those errors. The behavoir is different for all other error types. They will lead to a `500` response with the message of the error in the body. Additionally these errors will be logged automatically. The log entry will include the URL, method, request id and account id.\n\n**Examples**\n```go\nimport \"github.com/fastbill/go-httperrors\"\n\n//...\nechoServer.GET(\"/\", func (c echo.Context) error {\n\treturn httperrors.New(http.StatusForbidden, \"not allowed\")\n})\n// The HTTP response will be 403 with body {\"message\": \"not allowed\"} and the error will not be logged.\n// The developer needs to take care of logging the error if necessary.\n\nechoServer.GET(\"/\", func (c echo.Context) error {\n\tsomeError := errors.New(\"test error\")\n\treturn someError\n})\n// The HTTP response will be 500 with body {\"message\": \"test error\"} and the error will be logged.\n// No additional logging by the developer is needed.\n```\n\nIf you want to supress the automatic logging for 500 cases and log the error yourself instead, then return an HTTPError instead of the naked error:\n```go\nimport \"github.com/fastbill/go-httperrors\"\nechoServer.GET(\"/\", func (c echo.Context) error {\n\tsomeError := errors.New(\"test error\")\n\t// log the error yourself\n\treturn httperrors.New(http.StatusInternalServerError, someError)\n})\n```\n\n## Other Features\n* HTTP2 is disabled by default \n* Trailing slashes will be removed from the URL via [echo.labstack.com/middleware/trailing-slash](https://echo.labstack.com/middleware/trailing-slash)\n* If a panic happens somewhere in the HTTP handler it will be recovered and logged via [echo.labstack.com/middleware/recover](https://echo.labstack.com/middleware/recover), the server will not crash\n\n\n# Handlertest\nThis package helps with testing the echo handlers by providing a `CallHandler` method. It allows to specifiy default headers and middleware that should be applied for all handler tests.\nAddionally you can define the following parameters that should be applied when the handler function is called.\n* Route\n* Method\n* Body (can be `string`, `[]byte` or `io.Reader`)\n* Query parameters (they will be added to the query parameters in the route and overwrite the value of a particular parameter if it already exists)\n* Headers (they will overwrite the values that were set in the default headers)\n* Path parameters\n* Middleware (will be applied before the default middleware)\n* [Testify mocks](https://github.com/stretchr/testify#mock-package) for which should be checked whether their expectations were met after the handler was called\n* Sleeping time before the assertions for the mocks are performed (e.g. when they are called in another Go routine)\n\nAll parameters and default parameters are optional.\n\nAs response, the `CallHandler` method returns the error that the echo handler returned and the [response recorder](https://golang.org/pkg/net/http/httptest/#ResponseRecorder).\n\n## Usage\n### Minimal Case\n```go\nimport (\n\t\"testing\"\n    \"github.com/fastbill/go-service-toolkit/v4/handlertest\"\n)\n\nfunc TestMyHandler(t *testing.T) {\n\ts := handlertest.Suite{}\n\trec, err := s.CallHandler(tNew, myHandler, nil, nil)\n\t// Do the assertions on the response and the error.\n}\n```\nThis will call `myHandler` with the route `\\` and the method `GET` without additional headers etc.\n\n### Full Example\n```go\nimport (\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/fastbill/go-service-toolkit/v4/handlertest\"\n\t\"github.com/labstack/echo/v4\"\n)\n\nvar s = handlertest.Suite{\n\tDefaultHeaders: map[string]string{\n\t\t\"Content-Type\": \"application/json\",\n\t},\n\tDefaultMiddleware: []echo.MiddlewareFunc{mwDefault},\n}\n\nfunc mwDefault(next echo.HandlerFunc) echo.HandlerFunc {\n\treturn func(c echo.Context) error {\n\t\t// Do something\n\t\treturn next(c)\n\t}\n}\n\nfunc TestMyHandler(t *testing.T) {\n\tmwCustom := func(next echo.HandlerFunc) echo.HandlerFunc {\n\t\treturn func(c echo.Context) error {\n\t\t\t// Do something\n\t\t\treturn next(c)\n\t\t}\n\t}\n\n\tparams := \u0026handlertest.Params{\n\t\tRoute:  \"/some/path?query1=value1\",\n\t\tMethod: \"PUT\",\n\t\tBody:   `{\"id\":123}`,\n\t\tHeaders: map[string]string{\n\t\t\t\"testHeader\": \"someValue\",\n\t\t},\n\t\tQuery: map[string]string{\n\t\t\t\"query2\": \"value2\",\n\t\t},\n\t\tPathParams: []handlertest.PathParam{\n\t\t\t{Name: \"param1\", Value: \"value1\"},\n\t\t},\n\t\tMiddleware:        []echo.MiddlewareFunc{mwCustom},\n\t\tSleepBeforeAssert: 100 * time.Millisecond,\n\t}\n\n\trec, err := s.CallHandler(t, myHandler, params, []handlertest.MockAsserter{myMock})\n\t// Do the assertions on the response and the error.\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffastbill%2Fgo-service-toolkit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffastbill%2Fgo-service-toolkit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffastbill%2Fgo-service-toolkit/lists"}