{"id":25564932,"url":"https://github.com/webmafia/fluentlog","last_synced_at":"2026-03-14T08:30:15.337Z","repository":{"id":278422465,"uuid":"913971638","full_name":"webmafia/fluentlog","owner":"webmafia","description":"High-performance, asynchronous logging library for Go that implements the Fluent Forward protocol","archived":false,"fork":false,"pushed_at":"2025-02-19T17:02:53.000Z","size":355,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-19T18:21:25.843Z","etag":null,"topics":["fluentbit","fluentd","go","log"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/webmafia.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-08T17:45:15.000Z","updated_at":"2025-02-19T17:02:45.000Z","dependencies_parsed_at":"2025-02-19T18:21:28.659Z","dependency_job_id":"8378cd6b-7dad-438e-acd2-79bd00ef7e51","html_url":"https://github.com/webmafia/fluentlog","commit_stats":null,"previous_names":["webmafia/fluentlog"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/webmafia%2Ffluentlog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/webmafia%2Ffluentlog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/webmafia%2Ffluentlog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/webmafia%2Ffluentlog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/webmafia","download_url":"https://codeload.github.com/webmafia/fluentlog/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239921817,"owners_count":19718842,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fluentbit","fluentd","go","log"],"created_at":"2025-02-20T22:00:01.244Z","updated_at":"2026-03-14T08:30:15.292Z","avatar_url":"https://github.com/webmafia.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Fluentlog\n\n**Fluentlog** is a high-performance, asynchronous logging library for Go that implements the Fluent Forward protocol (used by [FluentBit](https://fluentbit.io/) and [Fluentd](https://www.fluentd.org/), hence the name). It is designed for minimal overhead and maximum flexibility, with zero allocations during logging operations.\n\n\u003e **Note:** While this package integrates with the awesome work of the Fluentd team, this is **not** an official package of theirs.\n\n## Features\n\n- **Asynchronous Logging:**  \n  Log entries are queued and processed in background workers to keep logging operations fast and non-blocking.\n\n- **Multiple Write Modes:**\n  - **Block:** Log calls block when the internal queue is full, ensuring that no message is lost.\n  - **Loose:** Log messages are dropped immediately if the queue is full, ensuring that the application is never blocked.\n  - **Fallback:** When the queue is full, log messages are written to a disk-based fallback buffer. *(Requires that the client implements the `BatchWriter` interface.)*\n\n- **Structured Logging:**  \n  Each log entry is a structured message that includes a tag, timestamp, and key-value pairs. All logging operations perform zero allocations.  \n  See [Structured Logging](#structured-logging) for details on how to pass metadata as key-value pairs.\n\n- **Severity Levels:**  \n  Log messages can be emitted with different syslog severity levels (e.g., DEBUG, INFO, WARN, ERROR, CRIT).\n\n- **Formatted Logging:**  \n  Support for both plain string messages and formatted (printf-style) messages.\n\n- **Sub-loggers with Inherited Metadata:**  \n  Create child loggers that automatically include additional context. The metadata for both logging operations and sub-loggers is provided as key-value pairs.  \n  For details, see [Structured Logging](#structured-logging).\n\n- **Panic Recovery:**  \n  Helper functions allow you to recover from panics and log them as critical errors.\n\n- **Fluent Forward Protocol:**  \n  The Forward client implements the [Fluent Forward protocol](https://github.com/fluent/fluentd/wiki/Forward-Protocol-Specification-v1.5), making it compatible with popular log collectors like FluentBit and Fluentd.\n\n## Installation\n\nInstall Fluentlog and its dependencies using `go get`:\n\n```sh\ngo get github.com/webmafia/fluentlog\n```\n\nWhen choosing between FluentBit and Fluentd, always pick the lighter FluentBit unless you need Fluentd's additional features.\n\n## Getting Started\n\nBelow is an example demonstrating how to set up a Fluentlog instance, create a logger, use sub-loggers with metadata, and log messages with different severity levels.\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"log\"\n\t\"os\"\n\t\"os/signal\"\n\t\"sync\"\n\n\t\"github.com/webmafia/fluentlog\"\n\t\"github.com/webmafia/fluentlog/fallback\"\n\t\"github.com/webmafia/fluentlog/forward\"\n)\n\nfunc main() {\n\t// Listen for interrupt signals to allow graceful shutdown.\n\tctx, stop := signal.NotifyContext(context.Background(), os.Interrupt)\n\tdefer stop()\n\n\tif err := startClient(ctx); err != nil {\n\t\tlog.Println(err)\n\t}\n}\n\nfunc startClient(ctx context.Context) error {\n\t// Configure the log client.\n\t// For example, using the forward client to send logs to a remote endpoint\n\t// that implements the Fluent Forward protocol.\n\taddr := \"localhost:24284\"\n\tcli := forward.NewClient(addr, forward.ClientOptions{\n\t\tSharedKey: []byte(\"secret\"),\n\t})\n\n\t// Create a new Fluentlog instance with desired options.\n\tinst, err := fluentlog.NewInstance(cli, fluentlog.Options{\n\t\tWriteBehavior:       fluentlog.Fallback,               // Use fallback when the log queue is full.\n\t\tFallback:            fallback.NewDirBuffer(\"fluentlog\"),   // Disk-based fallback buffer using \"fluentlog\" directory.\n\t\tStackTraceThreshold: fluentlog.NOTICE,                 // Threshold for including a stack trace.\n\t})\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer inst.Close()\n\n\t// Acquire a new logger.\n\tl := fluentlog.NewLogger(inst)\n\n\t// Create a sub-logger with additional metadata.\n\t// Pass metadata as key-value pairs: the odd arguments are the keys and the even arguments are the values.\n\tsub := l.With(\n\t\t\"component\", \"database\",\n\t\t\"operation\", \"query\",\n\t)\n\tdefer sub.Release()\n\n\t// Log several informational messages.\n\tfor i := 0; i \u003c 10; i++ {\n\t\tsub.Infof(\"message %d\", i+1)\n\t}\n\n\treturn nil\n}\n```\n\n## Structured Logging\n\nFluentlog supports structured logging by allowing you to pass key-value pairs as arguments to log messages and sub-loggers. The logging methods accept an initial message (or format) string followed by a variadic list of arguments. These arguments must be provided in pairs:\n\n- **Odd-indexed arguments:** Keys (must be strings according to the Fluent Forward protocol)\n- **Even-indexed arguments:** Corresponding values\n\nFor example, the following call:\n\n```go\nl.Info(\"Server started\",\n\t\"port\", 8080,\n\t\"env\", \"production\",\n)\n```\n\nlogs a message with metadata where `\"port\"` is paired with `8080` and `\"env\"` is paired with `\"production\"`.\n\nSimilarly, when creating sub-loggers with the `With` method:\n\n```go\nsub := l.With(\n\t\"component\", \"database\",\n\t\"operation\", \"query\",\n)\n\nsub.Info(\"Query executed successfully\",\n\t\"rows\", 42,\n)\n```\n\nthe sub-logger automatically includes the metadata (`\"component\": \"database\"`, `\"operation\": \"query\"`) in every log entry, and additional key-value pairs (like `\"rows\": 42`) can be provided during each logging call.\n\n## API Overview\n\n### Creating a Logger Instance\n\nThe logging system is built around the `Instance` type. Create a new instance using `NewInstance`:\n\n```go\ninst, err := fluentlog.NewInstance(cli, fluentlog.Options{\n    WriteBehavior:       fluentlog.Fallback,  // Choose Block, Loose, or Fallback.\n    Fallback:            fallback.NewDirBuffer(\"fluentlog\"),\n    BufferSize:          16,                  // Default is 16 if not specified.\n    StackTraceThreshold: fluentlog.NOTICE,\n})\nif err != nil {\n    // Handle error.\n}\ndefer inst.Close()\n\nl := inst.Logger()\n// Start logging with `l`.\n```\n\n### Logging with the Logger\n\nThe `Logger` type provides several methods to log messages at different severity levels. Each method returns an `identifier.ID` that can be used for tracing. For details on providing metadata, see [Structured Logging](#structured-logging).\n\n#### Plain Messages\n\n- `l.Debug(msg string, args ...any) identifier.ID`\n- `l.Info(msg string, args ...any) identifier.ID`\n- `l.Warn(msg string, args ...any) identifier.ID`\n- `l.Error(msg string, args ...any) identifier.ID`\n\nUsage:\n\n```go\nl.Info(\"Server started\", \"port\", 8080)\nl.Error(\"Failed to connect\", \"reason\", err)\n```\n\n#### Formatted Messages\n\n- `l.Debugf(format string, args ...any) identifier.ID`\n- `l.Infof(format string, args ...any) identifier.ID`\n- `l.Warnf(format string, args ...any) identifier.ID`\n- `l.Errorf(format string, args ...any) identifier.ID`\n\nUsage:\n\n```go\nl.Infof(\"Listening on port %d\", 8080)\nl.Errorf(\"Error: %v\", err)\n```\n\n### Creating Sub-Loggers with Metadata\n\nSub-loggers allow you to attach metadata that will be included with every log entry. Pass metadata as key-value pairs (see [Structured Logging](#structured-logging)):\n\n```go\nsub := l.With(\"component\", \"database\", \"operation\", \"query\")\ndefer sub.Release()\n\nsub.Info(\"Query executed successfully\", \"rows\", 42)\n```\n\n### Panic Recovery\n\nTo ensure that panics are logged instead of crashing the application, use the `Recover` helper in a deferred call within your goroutine:\n\n```go\ngo func() {\n    defer l.Recover() // Logs the panic as a critical error.\n    // Code that might panic.\n}()\n```\n\n## Write Behavior Modes\n\nFluentlog supports three write behavior modes via the `Options.WriteBehavior` setting:\n\n- **Block:**  \n  If the log queue is full, the log call will block until space becomes available. This ensures that no messages are lost but may cause delays.\n\n- **Loose:**  \n  Log messages are dropped if the queue is full. This prevents blocking but may lead to data loss during peak logging periods.\n\n- **Fallback:**  \n  When the queue is full, log messages are written to a fallback buffer on disk. This mode prevents both blocking and data loss. *(Requires that the client implements the `BatchWriter` interface.)*\n\nSet the mode when creating the logger instance:\n\n```go\ninst, err := fluentlog.NewInstance(cli, fluentlog.Options{\n    WriteBehavior: fluentlog.Fallback,\n    Fallback:      fallback.NewDirBuffer(\"fluentlog\"),\n})\n```\n\n## Fallback Buffer\n\nWhen using the fallback write behavior, Fluentlog uses a disk-based fallback mechanism (e.g., `DirBuffer`) to temporarily store log messages. Key points include:\n\n- **Fallback Queue:**  \n  An unbuffered channel to queue messages when the main queue is full.\n\n- **Fallback Worker:**  \n  A dedicated goroutine that writes queued messages to disk and later attempts to flush them to the logging destination.\n\nThis design helps ensure that no log messages are lost even if the primary logging destination is temporarily unreachable.\n\n## Contributing\n\nContributions are welcome, but please open an issue first that describes your use case.\n\n## License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.\n\nBy following the guidelines and examples above, you can integrate Fluentlog into your application, take advantage of its zero-allocation logging operations, and ensure reliable logging using the Fluent Forward protocol. Happy coding!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwebmafia%2Ffluentlog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwebmafia%2Ffluentlog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwebmafia%2Ffluentlog/lists"}