Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/devopscorner/golang-adot
AWS Summit 2023 ASEAN --- COM301 Implementing Observability using AWS Distro for OpenTelemetry (ADOT). Usecase Implementation for Simple Golang RESTful API Application (Bookstore)
https://github.com/devopscorner/golang-adot
adot aws awscli awscliv2 devops devopscorner eks eksctl go helm-template helmchart opensearch opentelemetry otel otel-agent otel-collector terraform
Last synced: about 1 month ago
JSON representation
AWS Summit 2023 ASEAN --- COM301 Implementing Observability using AWS Distro for OpenTelemetry (ADOT). Usecase Implementation for Simple Golang RESTful API Application (Bookstore)
- Host: GitHub
- URL: https://github.com/devopscorner/golang-adot
- Owner: devopscorner
- License: apache-2.0
- Created: 2023-04-19T16:22:18.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2024-10-29T18:18:40.000Z (about 2 months ago)
- Last Synced: 2024-10-29T20:25:28.166Z (about 2 months ago)
- Topics: adot, aws, awscli, awscliv2, devops, devopscorner, eks, eksctl, go, helm-template, helmchart, opensearch, opentelemetry, otel, otel-agent, otel-collector, terraform
- Language: HCL
- Homepage: https://devopscorner.id
- Size: 2.21 MB
- Stars: 6
- Watchers: 4
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# Golang Bookstore ADOT
ADOT (AWS Distro for OpenTelemetry) Implementation for Simple Golang RESTful API Application (Bookstore)
[](https://goreportcard.com/badge/github.com/devopscorner/golang-adot/src)
[](https://github.com/devopscorner/golang-adot/graphs/contributors)
[](https://github.com/devopscorner/golang-adot/releases)
[](https://hub.docker.com/r/devopscorner/bookstore-adot/)
[](https://github.com/devopscorner/golang-adot/issues)
[](https://github.com/devopscorner/golang-adot/pulls)
[](https://github.com/devopscorner/golang-adot/fork)
[](https://github.com/devopscorner/golang-adot/stargazers)
[](https://img.shields.io/github/license/devopscorner/golang-adot)
---
## Available Tags
### Alpine GOLANG
| Image name | Size |
|------------|------|
| `devopscorner/bookstore-adot:latest` | [](https://hub.docker.com/repository/docker/devopscorner/bookstore-adot/tags?page=1&ordering=last_updated&name=latest)   |
| `devopscorner/bookstore-adot:alpine` | [](https://hub.docker.com/repository/docker/devopscorner/bookstore-adot/tags?page=1&ordering=last_updated&name=alpine)  |
| `devopscorner/bookstore-adot:alpine-latest` | [](https://hub.docker.com/repository/docker/devopscorner/bookstore-adot/tags?page=1&ordering=last_updated&name=alpine-latest) |
| `devopscorner/bookstore-adot:alpine-3.17` | [](https://hub.docker.com/repository/docker/devopscorner/bookstore-adot/tags?page=1&ordering=last_updated&name=alpine-3.17) |
### CICD Alpine-3.17
| Image name | Size |
|------------|------|
| `devopscorner/bookstore-adot:cicd-alpine` | [](https://hub.docker.com/repository/docker/devopscorner/bookstore-adot/tags?page=1&ordering=last_updated&name=cicd-alpine) |
| `devopscorner/bookstore-adot:cicd-alpine-latest` | [](https://hub.docker.com/repository/docker/devopscorner/bookstore-adot/tags?page=1&ordering=last_updated&name=cicd-alpine-latest) |
| `devopscorner/bookstore-adot:cicd-alpine-3.17` | [](https://hub.docker.com/repository/docker/devopscorner/bookstore-adot/tags?page=1&ordering=last_updated&name=cicd-alpine-3.17) |
### CICD CodeBuild-4.0
| Image name | Size |
|------------|------|
| `devopscorner/bookstore-adot:cicd-codebuild` | [](https://hub.docker.com/repository/docker/devopscorner/bookstore-adot/tags?page=1&ordering=last_updated&name=cicd-codebuild)   |
| `devopscorner/bookstore-adot:cicd-codebuild-latest` | [](https://hub.docker.com/repository/docker/devopscorner/bookstore-adot/tags?page=1&ordering=last_updated&name=cicd-codebuild-latest) |
| `devopscorner/bookstore-adot:cicd-codebuild-4.0` | [](https://hub.docker.com/repository/docker/devopscorner/bookstore-adot/tags?page=1&ordering=last_updated&name=cicd-codebuild-4.0) |
| `devopscorner/bookstore-adot:cicd-latest` | [](https://hub.docker.com/repository/docker/devopscorner/bookstore-adot/tags?page=1&ordering=last_updated&name=cicd-latest)  |
---
## Documentation
- Index Documentation, go to [this](docs/README.md) link
- Build, Tag & Push container image to **DockerHub**, go to [this](docs/container-bookstore-adot-dockerhub.md) link
- Build, Tag & Push container image to **Amazon ECR (Elastic Container Registry)**, go to [this](docs/container-bookstore-adot-ecr.md) link
- Workflow CI/CD Pipeline, go to [this](docs/workflow-cicd-bookstore-adot-pipeline.md) link
- Deployments:
- **AWS Developer Tools** (AWS CodeCommit, AWS CodeBuild & AWS CodePipeline), detail [here](docs/deployment-aws-developer-tools.md) link
- **Jenkins CI/CD**, detail [here](docs/deployment-jenkins.md) link
- **Terraform AWS CodeBuild, AWS CodePipeline & Amazon SNS**, detail [here](docs/deployment-terraform.md) link
## GO Repository Pattern
- Folder Structure
```
.
├── .env
├── .env.example
├── config
│ ├── config.go
│ ├── config_test.go
│ ├── const.go
│ ├── logger.go
│ └── value.go
├── controller
│ ├── book_controller.go
│ ├── book_controller_test.go
│ ├── login_controller.go
│ └── login_controller_test.go
├── driver
│ ├── db.go
│ ├── dynamodb.go
│ ├── mysql.go
│ ├── psql.go
│ └── sqlite.go
├── go.mod
├── go.sum
├── main.go
├── main_test.go
├── middleware
│ ├── auth_middleware.go
│ └── auth_middleware_test.go
├── migrate_book.go.example
├── migrate_book_dynamo.go.example
├── model
│ └── book.go
├── observability
│ ├── metrics.go
│ ├── provider.go
│ ├── tracing-otel.go
│ └── tracing-xray.go
├── repository
│ └── book_repository.go
├── routes
│ ├── book_routes.go
│ ├── main_routes.go
│ ├── telemetry_routes.go
│ └── tracing_routes.go
└── view
├── book_view.go
├── error_view.go
└── login_view.go
10 directories, 37 files
```
## Coverages:
### AWS Services
- Amazon Elastic Container Registry (ECR)
- Amazon Managed Service for Prometheus (AMP)
- Amazon Managed Service for Grafana (AMG)
- AWS Distro for OpenTelemetry (ADOT)
- AWS CloudWatch
- AWS X-Ray
- AWS Systems Manager Parameter Store
- AWS Developer Tools (AWS CodeCommit, AWS CodePipeline, AWS CodeBuild)
- Amazon EKS
- Amazon OpenSearch -- *(Optional)*
- Amazon DynamoDB
### Others:
- Docker
- Docker-Compose
- Golang
- HelmChart
- Postman
- Terraform
## Environment Variables
- Default Value
```
GIN_MODE=release
APP_URL=http://localhost
APP_PORT=8080
DB_CONNECTION=dynamo
DB_HOST=localhost
DB_PORT=
DB_DATABASE=dynamodb-golang-adot
DB_USERNAME=root
DB_PASSWORD=
JWT_AUTH_USERNAME=devopscorner
JWT_AUTH_PASSWORD=DevOpsCorner2023
JWT_SECRET=s3cr3t
```
- Multi Driver Connection
```
# default
DB_CONNECTION=dynamo
---
Available for:
- sqlite
- mysql
- postgres
- dynamo
```
- DynamoDB Connection
```
DB_CONNECTION=dynamo
---
DB_DATABASE --> Dynamo Table
AWS_REGION --> AWS Region (Dynamo Region)
AWS_ACCESS_KEY --> AWS Access Key
AWS_SECRET_KEY_ID --> AWS Secret Key Id
```
- AWS X-Ray (AWS Distribution Tracing System)
- `XRAY_VERSION`: Daemon Version X-Ray
- `latest`: Use the latest version of the AWS X-Ray daemon available.
- `3.x`: Use version 3.x of the AWS X-Ray daemon.
- `2.x`: Use version 2.x of the AWS X-Ray daemon.
- `1.x`: Use version 1.x of the AWS X-Ray daemon.
- `XRAY_DAEMON_ENDPOINT: Daemon Endpoint of X-Ray
```
XRAY_DAEMON_ENDPOINT=http://localhost:2000
```
- `XRAY_DAEMON_PORT: Daemon Endpoint Port of X-Ray
```
XRAY_DAEMON_PORT=2000
```
- Prometheus
```
PROMETHEUS_ENDPOINT=http://localhost:9090
PROMETHEUS_PORT=9090
```
- Grafana
```
GRAFANA_ENDPOINT=http://localhost:3000
GRAFANA_API_KEY=YOUR_GRAFANA_API_KEY
```
- OpenTelemetry
```
OTEL_INSTRUMENTATION_METRIC_ENABLED=true
OTEL_INSTRUMENTATION_TRACE_ENABLED=true
OTEL_INSTRUMENTATION_LOG_ENABLED=true
# Trace Type: xray / jaeger
OTEL_INSTRUMENTATION_TRACE_NAME=xray
OTEL_SERVICE_NAME=bookstore-adot
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
OTEL_EXPORTER_OTLP_PORT=4317
OTEL_EXPORTER_OTLP_INSECURE=true
OTEL_EXPORTER_OTLP_HEADERS=
OTEL_RESOURCE_ATTRIBUTES=
```
- Jaeger Environment
- `JAEGER_SERVICE_NAME`: The name of the service being instrumented (`JAEGER_SERVICE_NAME` = `OTEL_SERVICE_NAME`).
- `JAEGER_AGENT_HOST`: The host name or IP address of the Jaeger agent to use for sending trace data.
- `JAEGER_AGENT_PORT`: The port number of the Jaeger agent to use for sending trace data.
- `JAEGER_SAMPLER_TYPE`: The type of sampling to use for tracing (e.g. const, probabilistic, rateLimiting, remote).
- `JAEGER_SAMPLER_PARAM`: The parameter value to use for the selected sampler type.
- `JAEGER_SAMPLER_MANAGER_HOST_PORT`: The host name and port of the remote sampling manager to use (if using remote sampling).
- `JAEGER_REPORTER_LOG_SPANS`: Whether to log spans instead of sending them to a Jaeger agent (e.g. true or false).
- `JAEGER_REPORTER_MAX_QUEUE_SIZE`: The maximum number of spans that can be queued before they are sent to a Jaeger agent.
- `JAEGER_REPORTER_FLUSH_INTERVAL`: The interval at which to flush the span queue and send spans to a Jaeger agent (e.g. 1 * time.Second).
- `JAEGER_REPORTER_LOCAL_AGENT_HOST_PORT`: The host and port of the local Jaeger agent, if using UDP instead of HTTP.
- `JAEGER_REPORTER_COLLECTOR_ENDPOINT`: The endpoint URL of the Jaeger collector (e.g. `http://localhost:14268/api/traces`).
- `JAEGER_REPORTER_COLLECTOR_USER`: The username for authenticating with the Jaeger collector, if required.
- `JAEGER_REPORTER_COLLECTOR_PASSWORD`: The password for authenticating with the Jaeger collector, if required.
- `JAEGER_TAGS`: Additional tags to attach to the trace (e.g. key1=value1,key2=value2).
## Reproduce Testing
- RESTful API Testing, go to [this](docs/test-restful-api.md) link
## Reproduce Provisioning
1. Install Terraform Modules, detail [here](docs/reproduce-01-teraform-modules.md)
2. Provisioning HelmChart Template, detail [here](docs/reproduce-02-helm-template.md)
3. Provisioning TFState DB & Bucket, detail [here](docs/reproduce-03-provisioning-tfstate-db-bucket.md)
4. Provisioning Infra Core, detail [here](docs/reproduce-04-provisioning-infra-core.md)
5. Provisioning Infra CI/CD, detail [here](docs/reproduce-05-provisioning-infra-cicd.md)
6. Provisioning Resources Amazon EKS, detail [here](docs/reproduce-06-provisioning-resources-eks.md)
7. Provisioning Resources Managed Services Prometheus (AMP), detail [here](docs/reproduce-07-provisioning-resources-amp.md)
8. Provisioning Resources Managed Services Grafana (AMG), detail [here](docs/reproduce-08-provisioning-resources-amg.md)
9. Provisioning Resources Managed Services OpenSearch, detail [here](docs/reproduce-09-provisioning-resources-opensearch.md)
10. Provisioning Resources EKS ADOT Collector, detail [here](docs/reproduce-10-provisioning-resources-eks-adot-collector.md)
## Snapshots
- Grafana Dashboard
- XRay
## Tested Environment
### Versioning
- Docker version
```
docker version
Client:
Cloud integration: v1.0.22
Version: 20.10.17-rd
API version: 1.41
Go version: go1.17.11
Git commit: c2e4e01
Built: Fri Jul 22 18:31:17 2022
OS/Arch: darwin/amd64
Context: default
Experimental: true
```
- Docker-Compose version
```
docker-compose -v
---
Docker Compose version v2.11.1
```
- AWS Cli
```
aws --version
---
aws-cli/2.8.7 Python/3.9.11 Darwin/21.6.0 exe/x86_64 prompt/off
```
- Terraform Cli
```
terraform version
---
Terraform v1.3.5
on darwin_amd64
- provider registry.terraform.io/hashicorp/aws v3.74.3
- provider registry.terraform.io/hashicorp/local v2.1.0
- provider registry.terraform.io/hashicorp/null v3.1.0
- provider registry.terraform.io/hashicorp/random v3.1.0
- provider registry.terraform.io/hashicorp/time v0.7.2
```
- Terraform Environment Cli
```
tfenv -v
---
tfenv 2.2.2
```
- Golang Version Manager (GVM)
```
gvm version
---
Go Version Manager v1.0.22 installed at /Users/devopscorner/.gvm
```
- Golang Version
```
go version
---
go version go1.19.6 darwin/arm64
```
## Security Check
Make sure that you didn't push sensitive information in this repository
- [ ] AWS Credentials (AWS_ACCESS_KEY, AWS_SECRET_KEY)
- [ ] AWS Account ID
- [ ] AWS Resources ARN
- [ ] Username & Password
- [ ] Private (id_rsa) & Public Key (id_rsa.pub)
- [ ] DNS Zone ID
- [ ] APP & API Key
## Copyright
- Author: **Dwi Fahni Denni (@zeroc0d3)**
- Vendor: **DevOps Corner Indonesia (devopscorner.id)**
- License: **Apache v2**