{"id":19435439,"url":"https://github.com/goadesign/clue","last_synced_at":"2025-04-07T10:27:38.198Z","repository":{"id":38331553,"uuid":"445958897","full_name":"goadesign/clue","owner":"goadesign","description":"🔍 Seamless Observability for Distributed Systems 🔍","archived":false,"fork":false,"pushed_at":"2025-03-31T08:54:18.000Z","size":2000,"stargazers_count":70,"open_issues_count":3,"forks_count":11,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-03-31T09:04:53.042Z","etag":null,"topics":["go","goa","observability","opentelemetry"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/goadesign.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-01-09T00:25:24.000Z","updated_at":"2025-03-26T19:09:19.000Z","dependencies_parsed_at":"2023-02-11T15:00:38.187Z","dependency_job_id":"72bc5f96-f9fe-472b-95f9-b084a0cc7da2","html_url":"https://github.com/goadesign/clue","commit_stats":{"total_commits":424,"total_committers":11,"mean_commits":38.54545454545455,"dds":0.3207547169811321,"last_synced_commit":"5fb02554f329c7c6b20d93ac8ecff6f66e5e73b7"},"previous_names":[],"tags_count":37,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goadesign%2Fclue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goadesign%2Fclue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goadesign%2Fclue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goadesign%2Fclue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/goadesign","download_url":"https://codeload.github.com/goadesign/clue/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247634627,"owners_count":20970572,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","goa","observability","opentelemetry"],"created_at":"2024-11-10T15:06:25.684Z","updated_at":"2025-04-07T10:27:38.160Z","avatar_url":"https://github.com/goadesign.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# clue: Microservice Instrumentation\n\n[![Go Reference](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go\u0026logoColor=white\u0026style=for-the-badge)](https://pkg.go.dev/goa.design/clue)\n[![License](https://img.shields.io/badge/License-MIT%202.0-blue?style=for-the-badge)](LICENSE)\n\n[![Build Status](https://img.shields.io/github/actions/workflow/status/goadesign/clue/ci.yaml?style=for-the-badge)](https://github.com/goadesign/clue/actions?query=branch%3Amain+event%3Apush)\n[![codecov](https://img.shields.io/codecov/c/github/goadesign/clue/main?style=for-the-badge\u0026token=HVP4WT1PS6)](https://codecov.io/gh/goadesign/clue)\n[![Go Report Card](https://img.shields.io/badge/go%20report-A+-brightgreen.svg?style=for-the-badge)](https://goreportcard.com/report/goa.design/clue)\n\n\u003c/div\u003e\n\n## Overview\n\nClue provides a set of Go packages for instrumenting microservices. The\nemphasis is on simplicity and ease of use. Although not a requirement, Clue\nworks best when used in microservices written using\n[Goa](https://github.com/goadesign/goa).\n\nClue covers the following topics:\n\n* Instrumentation: the [clue](clue/) package configures OpenTelemetry\n  for service instrumentation.\n* Logging: the [log](log/) package provides a context-based logging API that\n  intelligently selects what to log.\n* Health checks: the [health](health/) package provides a simple way for\n  services to expose a health check endpoint.\n* Dependency mocks: the [mock](mock/) package provides a way to mock\n  downstream dependencies for testing.\n* Debugging: the [debug](debug/) package makes it possible to troubleshoot\n  and profile services at runtime.\n* Interceptors: the [interceptors](interceptors/) package provides a set of\n  helpful Goa interceptors.\n\nClue's goal is to provide all the ancillary functionality required to efficiently\noperate a microservice style architecture including instrumentation, logging,\ndebugging and health checks. Clue is not a framework and does not provide\nfunctionality that is already available in the standard library or in other\npackages. For example, Clue does not provide a HTTP router or a HTTP server\nimplementation. Instead, Clue provides a way to instrument existing HTTP or gRPC\nservers and clients using the standard library and the OpenTelemetry API.\n\nLearn more about Clue's observability features in the [Observability](https://goa.design/docs/5-real-world/2-observability/)\nsection of the [goa.design](https://goa.design) documentation. The guide covers how to effectively monitor,\ndebug and operate microservices using Clue's instrumentation capabilities.\n\n## Packages\n\n* The `clue` package provides a simple API for configuring OpenTelemetry\n  instrumentation. The package also provides a way to configure the log\n  package to automatically annotate log messages with trace and span IDs.\n  The package also implements a dynamic trace sampler that can be used to\n  sample traces based on a target maximum number of traces per second.\n\n* The `log` package offers a streamlined, context-based structured logger that\n  efficiently buffers log messages. It smartly determines the optimal time to\n  flush these messages to the underlying logger. In its default configuration,\n  the log package flushes logs upon the logging of an error or when a request is\n  traced. This design choice minimizes logging overhead for untraced requests,\n  ensuring efficient logging operations.\n\n* The `health` package provides a simple way to expose a health check endpoint\n  that can be used by orchestration systems such as Kubernetes to determine\n  whether a service is ready to receive traffic. The package implements the\n  concept of checkers that can be used to implement custom health checks with\n  a default implementation that relies on the ability to ping downstream\n  dependencies.\n\n* The `mock` package provides a way to mock downstream dependencies for testing.\n  The package provides a tool that generates mock implementations of interfaces\n  and a way to configure the generated mocks to validate incoming payloads and\n  return canned responses.\n\n* The `debug` package provides a way to dynamically control the log level of a\n  running service. The package also provides a way to expose the pprof Go\n  profiling endpoints and a way to expose the log level control endpoint.\n\n## Installation\n\nClue requires Go 1.21 or later. Install the packages required for your\napplication using `go get`, for example:\n\n```bash\ngo get goa.design/clue/clue\ngo get goa.design/clue/log\ngo get goa.design/clue/health\ngo get goa.design/clue/mock\ngo get goa.design/clue/debug\n```\n\n## Usage\n\nThe following snippet illustrates how to use `clue` to instrument a HTTP server:\n\n```go\nctx := log.Context(context.Background(),    // Create a clue logger context.\n    log.WithFunc(log.Span))                 // Log trace and span IDs.\n\nmetricExporter := stdoutmetric.New()        // Create metric and span exporters.\nspanExporter := stdouttrace.New()\ncfg := clue.NewConfig(ctx, \"service\", \"1.0.0\", metricExporter, spanExporter)\nclue.ConfigureOpenTelemetry(ctx, cfg)       // Configure OpenTelemetry.\n\nhandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n    w.Write([]byte(\"Hello, World!\"))\n})                                          // Create HTTP handler.\nhandler = otelhttp.NewHandler(handler, \"service\" ) // Instrument handler.\nhandler = debug.HTTP()(handler)             // Setup debug log level control.\nhandler = log.HTTP(ctx)(handler)            // Add logger to request context and log requests.\n\nmux := http.NewServeMux()                   // Create HTTP mux.\nmux.HandleFunc(\"/\", handler)                // Mount handler.\ndebug.MountDebugLogEnabler(mux)             // Mount debug log level control handler.\ndebug.MountDebugPprof(mux)                  // Mount pprof handlers.\nmux.HandleFunc(\"/health\",\n    health.NewHandler(health.NewChecker())) // Mount health check handler.\n\nhttp.ListenAndServe(\":8080\", mux)           // Start HTTP server.\n```\n\nSimilarly, the following snippet illustrates how to instrument a gRPC server:\n\n```go\nctx := log.Context(context.Background(),    // Create a clue logger context.\n    log.WithFunc(log.Span))                 // Log trace and span IDs.\n\nmetricExporter := stdoutmetric.New()\nspanExporter := stdouttrace.New()\ncfg := clue.NewConfig(ctx, \"service\", \"1.0.0\", metricExporter, spanExporter)\nclue.ConfigureOpenTelemetry(ctx, cfg)       // Configure OpenTelemetry.\n\nsvr := grpc.NewServer(\n    grpc.ChainUnaryInterceptor(\n        log.UnaryServerInterceptor(ctx),    // Add logger to request context and log requests.\n        debug.UnaryServerInterceptor()),    // Enable debug log level control\n    grpc.StatsHandler(otelgrpc.NewServerHandler()), // Instrument server.\n)\n```\n\nNote that in the case of gRPC, a separate HTTP server is required to expose the\ndebug log level control, pprof and health check endpoints:\n\n```go\nmux := http.NewServeMux()                   // Create HTTP mux.\ndebug.MountDebugLogEnabler(mux)             // Mount debug log level control handler.\ndebug.MountDebugPprof(mux)                  // Mount pprof handlers.\nmux.HandleFunc(\"/health\",\n    health.NewHandler(health.NewChecker())) // Mount health check handler.\n\ngo http.ListenAndServe(\":8081\", mux)        // Start HTTP server.\n```\n\n## Exporters\n\nExporter are responsible for exporting telemetry data to a backend. The\n[OpenTelemetry Exporters documentation](https://opentelemetry.io/docs/instrumentation/go/exporters/)\nprovides a list of available exporters.\n\nFor example, configuring an [OTLP](https://opentelemetry.io/docs/specs/otlp/)\ncompliant span exporter can be done as follows:\n\n```go\nimport \"go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc\"\n// ...\nspanExporter, err := otlptracegrpc.New(\n    context.Background(),\n    otlptracegrpc.WithEndpoint(\"localhost:4317\"),\n    otlptracegrpc.WithTLSCredentials(insecure.NewCredentials()))\n```\n\nWhile configuring an OTLP compliant metric exporters can be done as follows:\n\n```go\nimport \"go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc\"\n// ...\nmetricExporter, err := otlpmetricgrpc.New(\n    context.Background(),\n    otlpmetricgrpc.WithEndpoint(\"localhost:4317\"),\n    otlpmetricgrpc.WithTLSCredentials(insecure.NewCredentials()))\n```\n\nThese exporters can then be used to configure Clue:\n\n```go\n// Configure OpenTelemetry.\ncfg := clue.NewConfig(ctx, \"service\", \"1.0.0\", metricExporter, spanExporter)\nclue.ConfigureOpenTelemetry(ctx, cfg)\n```\n\n## Clients\n\nHTTP clients can be instrumented using the Clue `log` and OpenTelemetry `otelhttptrace` packages. The `log.Client` function wraps a HTTP transport and logs the request and response. The `otelhttptrace.Client` function wraps a HTTP transport and adds OpenTelemetry tracing to the request.\n\n```go\nimport (\n    \"context\"\n    \"net/http\"\n    \"net/http/httptrace\"\n\n    \"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp\"\n    \"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttptrace\"\n    \"goa.design/clue/log\"\n)\n\n// ...\nhttpc := \u0026http.Client{\n    Transport: log.Client(\n        otelhttp.NewTransport(\n            http.DefaultTransport,\n            otelhttp.WithClientTrace(func(ctx context.Context) *httptrace.ClientTrace {\n                return otelhttptrace.NewClientTrace(ctx)\n            }),\n        ),\n    ),\n}\n```\n\nSimilarly, gRPC clients can be instrumented using the Clue `log` and OpenTelemetry `otelgrpc` packages.\n\n```go\nimport (\n    \"context\"\n\n    \"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc\"\n    \"goa.design/clue/log\"\n)\n\n// ...\ngrpcconn, err := grpc.DialContext(ctx,\n    \"localhost:8080\",\n    grpc.WithUnaryInterceptor(log.UnaryClientInterceptor()),\n    grpc.WithStatsHandler(otelgrpc.NewClientHandler()))\n)\n```\n\n## Custom Instrumentation\n\nClue relies on the OpenTelemetry API for creating custom instrumentation. The\nfollowing snippet illustrates how to create a custom counter and span:\n\n```go\n// ... configure OpenTelemetry like in example above\nclue.ConfigureOpenTelemetry(ctx, cfg)\n\n// Create a meter and tracer\nmeter := otel.Meter(\"mymeter\")\ntracer := otel.Tracer(\"mytracer\")\n\n// Create a counter\ncounter, err := meter.Int64Counter(\"my.counter\",\n    metric.WithDescription(\"The number of times the service has been called\"),\n    metric.WithUnit(\"{call}\"))\nif err != nil {\n    log.Fatalf(\"failed to create counter: %s\", err)\n}\n\nhandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n    // Create a span\n    ctx, span := tracer.Start(r.Context(), \"myhandler\")\n    defer span.End()\n\n    // ... do something\n\n    // Add custom attributes to span and counter\n    attr := attribute.Int(\"myattr\", 42)\n    span.SetAttributes(attr)\n    counter.Add(ctx, 1, metric.WithAttributes(attr))\n\n    // ... do something else\n    if _, err := w.Write([]byte(\"Hello, World!\")); err != nil {\n        log.Errorf(ctx, err, \"failed to write response\")\n    }\n})\n```\n\n## Goa\n\nThe `log` package provides a Goa endpoint middleware that adds the service and\nmethod names to the logger context. The `debug` package provides a Goa endpoint\nmiddleware that logs the request and response payloads when debug logging is\nenabled. The example below is a snippet extracted from the `main` function of\nthe\n[genforecaster](example/weather/services/forecaster/cmd/forecaster/main.go#L88)\nservice:\n\n```go\nsvc := forecaster.New(wc)\nendpoints := genforecaster.NewEndpoints(svc)\nendpoints.Use(debug.LogPayloads())\nendpoints.Use(log.Endpoint)\n```\n\n## Example\n\nThe [weather](example/weather) example illustrates how to use Clue to instrument\na system of Goa microservices. The example comes with a set of scripts that can\nbe used to install all necessary dependencies including the\n[SigNoz](https://signoz.io/) instrumentation backend.  See the\n[README](example/weather/README.md) for more information.\n\n## Migrating from v0.x to v1.x\n\nThe v1.0.0 release of `clue` is a major release that introduces breaking\nchanges. The following sections describe the changes and how to migrate.\n\n### Initialization\n\nThe `clue` package provides a cohesive API for both metrics and tracing,\neffectively replacing the previous `metrics` and `trace` packages. The\ntraditional `Context` function, utilized in the `metrics` and `trace` packages\nfor setting up telemetry, has been deprecated. In its place, the `clue` package\nintroduces the `NewConfig` function, which generates a `Config` object used in\nconjunction with the `ConfigureOpenTelemetry` function to facilitate telemetry\nsetup.\n\nv0.x:\n\n```go\nctx := log.Context(context.Background())\ntraceExporter := tracestdout.New()\nmetricsExporter := metricstdout.New()\nctx = trace.Context(ctx, \"service\", traceExporter)\nctx = metrics.Context(ctx, \"service\", metricsExporter)\n```\n\nv1.x:\n\n```go\nctx := log.Context(context.Background())\ntraceExporter := tracestdout.New()\nmetricsExporter := metricstdout.New()\ncfg := clue.NewConfig(ctx, \"service\", \"1.0.0\", metricsExporter, traceExporter)\nclue.ConfigureOpenTelemetry(ctx, cfg)\n```\n\n### Instrumentation\n\nInstrumenting a HTTP service is now done using the standard `otelhttp` package:\n\nv0.x:\n\n```go\nhandler = trace.HTTP(ctx)(handler)\nhandler = metrics.HTTP(ctx)(handler)\nhttp.ListenAndServe(\":8080\", handler)\n```\n\nv1.x:\n\n```go\nhttp.ListenAndServe(\":8080\", otelhttp.NewHandler(\"service\", handler))\n```\n\nSimilarly, instrumenting a gRPC service is now done using the standard\n`otelgrpc` package. v1.x also switches to using a gRPC stats handler instead of\ninterceptors:\n\nv0.x:\n\n```go\ngrpcsvr := grpc.NewServer(\n        grpcmiddleware.WithUnaryServerChain(\n                trace.UnaryServerInterceptor(ctx),\n                metrics.UnaryServerInterceptor(ctx),\n        ),\n        grpcmiddleware.WithStreamServerChain(\n                trace.StreamServerInterceptor(ctx),\n                metrics.StreamServerInterceptor(ctx),\n        ),\n)\n```\n\nv1.x:\n\n```go\ngrpcsvr := grpc.NewServer(grpc.StatsHandler(otelgrpc.NewServerHandler()))\n```\n\n### Goa\n\nThe `otel` Goa plugin leverages the OpenTelemetry API to annotate spans and\nmetrics with HTTP routes.\n\nv0.x:\n\n```go\nmux := goahttp.NewMuxer()\nctx = metrics.Context(ctx, genfront.ServiceName,\n        metrics.WithRouteResolver(func(r *http.Request) string {\n                return mux.ResolvePattern(r)\n        }),\n)\n```\n\nv1.x:\n\n```go\npackage design\n\nimport (\n        . \"goa.design/goa/v3/dsl\"\n        _ \"goa.design/plugins/v3/clue\"\n)\n```\n\n## Contributing\n\nSee [Contributing](CONTRIBUTING.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoadesign%2Fclue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoadesign%2Fclue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoadesign%2Fclue/lists"}