Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/highlyavailable/go-template

A structured Go app template with a Makefile for build automation, envconfig for environment management, Zap and lumberjack for logging with rotation/retention, OpenTelemetry and Prometheus for metrics/tracing, and Docker support. Includes a Gin server with health checks, Swagger docs, and various other integrations.
https://github.com/highlyavailable/go-template

docker go grafana kafka lumberjack makefile otel prometheus template zap-logger

Last synced: about 14 hours ago
JSON representation

Host: GitHub
URL: https://github.com/highlyavailable/go-template
Owner: highlyavailable
Created: 2024-07-26T13:30:48.000Z (2 months ago)
Default Branch: main
Last Pushed: 2024-08-02T19:37:40.000Z (about 2 months ago)
Last Synced: 2024-09-19T10:22:40.117Z (9 days ago)
Topics: docker, go, grafana, kafka, lumberjack, makefile, otel, prometheus, template, zap-logger
Language: Go
Homepage:
Size: 7.62 MB
Stars: 0
Watchers: 1
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

README

# Go Template Project

## Overview

Fork for quick creation of new Go applications. It includes a structured directory layout with a Makefile for build automation, `envconfig` for environment configuration/management, `Zap` and `lumberjack` for structured logging with rotation and retention, `OpenTelemetry` for observability and tracing, `Prometheus` for metrics collection and monitoring. Includes a Dockerfile and docker-compose.yaml for containerization. Contains a basic HTTP Gin server with health check and Swagger documentation, with additional integrations for Kafka, Postgres, MongoDB, and Redis.

## Getting Started

### Prerequisites

- Go 1.21 (specified in go.mod)
- make installed
- Docker (optional)

### Setup

1. **Clone the Repository**

```bash
git clone
cd
```

2. **Install Dependencies**

Install the project dependencies using `go mod`:

```bash
cd goapp
go mod tidy
```

3. **Rename the Application**

Update the application name using the `rename_app.sh` script.

```bash
./rename_app.sh
# Example
./rename_app.sh goapp myapp
```

4. **Set Up Environment Variables**

Ensure that required environment variables are set. Refer to the `.env.example` file for reference.

### Makefile Targets

The Makefile in the root of the project provides a set of commands to help with building, running, and managing the application. Running `make first` will set up your project by creating necessary files and installing dependencies.

#### Setup

- **`first`**: Performs an initial setup by:
- Creating the `.env` file if it doesn’t exist.
- Installing Swaggo for Swagger documentation.
- Generating Swagger documentation.
- Building the application.
- Running the application.

#### Environment Management

- **`env`**: Checks for the `.env` file and creates it if missing.

#### Swagger Documentation

- **`swaggo`**: Installs the Swaggo tool for generating Swagger documentation.
- **`swagger`**: Generates Swagger documentation from the codebase.

#### Build

- **`build`**: Builds the application binary defaults to MacOS.
- **`build-verbose`**: Builds the application with verbose output.
- **`build-linux`**: Builds the application for Linux.
- **`build-windows`**: Builds the application for Windows.
- **`build-race`**: Builds the application with race detector enabled.

#### Run

- **`run`**: Runs the built application binary, stored in the `build` directory.

#### Docker Management

- **`docker-recycle`**: Recycles Docker containers using the `docker-recycle.sh` script, located in the root of the project.
- **`docker-exec`**: Executes a shell in the Docker container.

#### Miscellaneous

- **`tidy`**: Tidy will clean up the Go module cache in the Go module directory.
- **`monitors`**: Starts the monitoring stack with Prometheus and Grafana.

Running `make first` is recommended to get your project set up and ready for development.

### Docker

To run the application in Docker:

```bash
docker-compose up --build
```

Attach a shell to the container:

```bash
docker run -it --entrypoint /bin/sh go-app
```

## File Structure

```bash
> tree
.
├── Dockerfile
├── Makefile
├── README.md
├── assets
│   └── certs
├── docker-compose.yaml
├── docker-recycle.sh
├── goapp
│   ├── api
│   │   ├── handlers
│   │   │   └── health.go
│   │   └── routes
│   │   └── routes.go
│   ├── build
│   │   └── goapp
│   ├── cmd
│   │   └── goapp
│   │   └── main.go
│   ├── docs
│   │   ├── docs.go
│   │   ├── swagger.json
│   │   └── swagger.yaml
│   ├── go.mod
│   ├── go.sum
│   ├── internal
│   │   └── config
│   │   └── config.go
│   ├── pkg
│   │   ├── clients
│   │   │   └── clients.go
│   │   ├── db
│   │   │   ├── kafka
│   │   │   │   ├── example.go
│   │   │   │   ├── kafka.go
│   │   │   │   └── model.go
│   │   │   └── postgres
│   │   │   ├── example.go
│   │   │   ├── model.go
│   │   │   └── postgres.go
│   │   ├── logging
│   │   │   ├── logging.go
│   │   │   └── model.go
│   │   └── otel
│   │   └── otel.go
│   ├── scripts
│   ├── tests
│   │   ├── helpers
│   │   └── unit
│   └── web
│   ├── static
│   └── templates
├── logs
│   └── app.log
├── monitoring
│   ├── datasources
│   │   └── datasources.yaml
│   ├── docker-compose.yaml
│   └── prometheus.yaml
└── rename_app.sh

30 directories, 30 files
```

## Directory Overview

- **api**: Contains the HTTP handlers and routes.
- **build**: Build artifacts.
- **cmd**: Main applications for the project.
- **docs**: Documentation including Swagger.
- **internal**: Internal application code.
- **pkg**: Library code to be used by external applications.
- **db/kafka**: Kafka integration.
- **db/postgres**: PostgreSQL integration.
- **logging**: Logging setup.
- **otel**: OpenTelemetry setup.
- **scripts**: Scripts for various tasks.
- **tests**: Unit and helper tests.
- **web**: Web assets like static files and templates.
- **logs**: Application logs.
- **monitoring**: Monitoring setup for Grafana and Prometheus.

## Environment Variables

The application uses `envconfig` to manage environment variables. Ensure the following variables are set:

- **Kafka**:
- `KAFKA_BROKERS`
- `KAFKA_PRODUCER_TOPIC`
- `KAFKA_CONSUMER_TOPIC`
- `KAFKA_CONSUMER_GROUP`
- `KAFKA_CONSUMER_OFFSET`

- **Postgres**:
- `DB_HOST`
- `DB_PORT`
- `DB_USER`
- `DB_PASSWORD`
- `DB_NAME`
- `DB_SSL_MODE`

- **Prometheus**:
- `PROMETHEUS_PORT`

## Logging

The application uses Zap for structured logging with log rotation and retention. Logs are written to `app.log` and `error.log` in the `logs` directory.

## Monitoring

Prometheus is used for metrics collection and Grafana for visualization. The monitoring setup is available in the `monitoring` directory, just run `docker compose up` to start the monitoring stack which includes a single Prometheus instance and a Grafana instance. The Prometheus configuration is available in `monitoring/prometheus.yaml`, and the Grafana datasource configuration to pull metrics from Prometheus is available in `monitoring/datasources/datasources.yaml`.