{"id":26681286,"url":"https://github.com/culiops/fluentd-lambda-extension","last_synced_at":"2026-04-28T11:36:34.767Z","repository":{"id":182244957,"uuid":"602581974","full_name":"culiops/fluentd-lambda-extension","owner":"culiops","description":"AWS Lambda - Logs API Extension which send logs to Fluentd","archived":false,"fork":false,"pushed_at":"2023-03-04T05:47:41.000Z","size":30,"stargazers_count":2,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-26T07:15:08.816Z","etag":null,"topics":["aws","aws-lambda","fluentd","golang","logging"],"latest_commit_sha":null,"homepage":"https://www.youtube.com/@culiops","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/culiops.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-02-16T14:15:27.000Z","updated_at":"2025-02-20T03:57:19.000Z","dependencies_parsed_at":"2023-07-19T09:45:35.754Z","dependency_job_id":null,"html_url":"https://github.com/culiops/fluentd-lambda-extension","commit_stats":null,"previous_names":["culiops/fluentd-lambda-extension"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/culiops/fluentd-lambda-extension","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/culiops%2Ffluentd-lambda-extension","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/culiops%2Ffluentd-lambda-extension/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/culiops%2Ffluentd-lambda-extension/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/culiops%2Ffluentd-lambda-extension/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/culiops","download_url":"https://codeload.github.com/culiops/fluentd-lambda-extension/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/culiops%2Ffluentd-lambda-extension/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32379618,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-28T11:25:28.583Z","status":"ssl_error","status_checked_at":"2026-04-28T11:25:05.435Z","response_time":56,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-lambda","fluentd","golang","logging"],"created_at":"2025-03-26T07:15:11.270Z","updated_at":"2026-04-28T11:36:34.746Z","avatar_url":"https://github.com/culiops.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AWS Lambda - Logs API Extension for Fluentd\nThe provided code how to get a basic Logs API extension to send logs to Fluentd written in Golang up and running.\n\nIn this extension, we start by developing a simple extension and then add the ability to read logs from the Logs API. For more details on building an extension, please read the Extension API Developer Guide.\n\nWhen the Lambda service sets up the execution environment, it runs the extension (`fluentd-lambda-extension`). This extension first registers as an extension and then subscribes to the Logs API to receive the logs via HTTP protocol. It starts an HTTP listener which receives the logs and processes them.\n\n## System Compatible\n- Architectures: `AMD64` \u0026 `ARM64`\n- Runtimes: `All Linux`\n- Function code with container and non-container\n\n## Compile package and dependencies\n\nTo run this example, you will need to ensure that your build architecture matches that of the Lambda execution environment by compiling with `GOOS=linux` and (`GOARCH=amd64` or `GOARCH=arm64`).\n\nBuilding and saving package into a `bin` directory:\n```bash\n$ cd fluentd-lambda-extension/extensions\n$ GOOS=linux GOARCH=amd64 go build -o bin/fluentd-lambda-extension\n$ chmod +x bin/fluentd-lambda-extension\n```\n\n## Layer Setup - by Terraform\n```hcl\nresource \"aws_lambda_function\" \"example\" {\n  layers = [\n    module.lambda_layer_extension.lambda_layer_arn,\n  ]\n\n  environment {\n    variables = {\n      FLUENTD_HOST = module.instance.private_ip[0]\n    }\n  }\n}\n```\n\n## Layer Setup - by Manually\n### ZIP file manually setup\nThe extensions .zip file should contain a root directory called `extensions/`, where the extension executables are located.\n\nCreating zip package for the extension:\n```bash\n$ chmod +x extensions/bin/fluentd-lambda-extension\n$ cd extensions/bin \u0026\u0026 mkdir extensions\n$ mv fluentd-lambda-extension extensions\n$ zip -r extension.zip extensions\n```\n\nPublish a new layer using the `extension.zip`. The output of the following command should provides you a layer arn.\n```bash\naws lambda publish-layer-version \\\n --layer-name \"fluentd-lambda-extension\" \\\n --region \u003cuse your region\u003e \\\n --zip-file  \"fileb://extension.zip\"\n```\nNote the LayerVersionArn that is produced in the output.\ne.g. `\"LayerVersionArn\": \"arn:aws:lambda:\u003cregion\u003e:123456789012:layer:fluentd-lambda-extension:1\"`\n\nAdd the newly created layer version to a Lambda function.\n```bash\naws lambda update-function-configuration --region \u003cuse your region\u003e --function-name \u003cyour function name\u003e --layers \u003cLayerVersionArn from previous step\u003e\n```\n\n### Docker manually setup\n```\nFROM 123456789012.dkr.ecr.ap-southeast-1.amazonaws.com/fluentd-lambda-extension:v1 as extensions-layer\nCOPY --from=extensions-layer /opt/extensions/fluentd-lambda-extension-{amd64, arm64} /opt/extensions/\n```\n\n## Lambda Function Environment variables support\n- `FLUENTD_HOST`: fluentd host, which lambda will send log to, default: `localhost`\n- `FLUENTD_PORT`: fluent port, which lambda using to connect, default: `24224`\n- `FLUENTD_TAG_SUFFIX`: log suffix, default: `es.log`\n\nExample: if `function-test.es.log` is fluentd tag key, then `es.log` is `FLUENTD_TAG_SUFFIX`\n\nSample log line:\n```sh\n2022-09-30 03:25:03.000000000 +0000 fluentd-lambda-extension-examples.es.log: {\"function_name\":\"fluentd-lambda-extension-examples\",\"msg\":\"{\\\"record\\\":\\\"Finished test extensions. Well done!\\\\n\\\",\\\"time\\\":\\\"2022-09-30T03:25:02.224Z\\\",\\\"type\\\":\\\"function\\\"}\"}\n2022-09-30 03:25:03.000000000 +0000 fluentd-lambda-extension-examples.es.log: {\"function_name\":\"fluentd-lambda-extension-examples\",\"msg\":\"{\\\"record\\\":{\\\"requestId\\\":\\\"9df595ac-df7f-43b6-b97b-dcdfa6838fda\\\",\\\"status\\\":\\\"success\\\"},\\\"time\\\":\\\"2022-09-30T03:25:02.224Z\\\",\\\"type\\\":\\\"platform.runtimeDone\\\"}\"}\n2022-09-30 03:30:51.000000000 +0000 fluentd-lambda-extension-examples.es.log: {\"function_name\":\"fluentd-lambda-extension-examples\",\"msg\":\"{\\\"record\\\":{\\\"requestId\\\":\\\"9df595ac-df7f-43b6-b97b-dcdfa6838fda\\\"},\\\"time\\\":\\\"2022-09-30T03:25:03.224Z\\\",\\\"type\\\":\\\"platform.end\\\"}\"}\n2022-09-30 03:30:51.000000000 +0000 fluentd-lambda-extension-examples.es.log: {\"function_name\":\"fluentd-lambda-extension-examples\",\"msg\":\"{\\\"record\\\":{\\\"metrics\\\":{\\\"billedDurationMs\\\":1002,\\\"durationMs\\\":1001.22,\\\"maxMemoryUsedMB\\\":50,\\\"memorySizeMB\\\":2048},\\\"requestId\\\":\\\"9df595ac-df7f-43b6-b97b-dcdfa6838fda\\\"},\\\"time\\\":\\\"2022-09-30T03:25:03.224Z\\\",\\\"type\\\":\\\"platform.report\\\"}\"}\n```\n\n## Notes\nWhen deploying the Lambda function, be sure to include configure environment variables that the extension will leverage for communicating with the Fluentd endpoint(`FLUENTD_HOST`). This endpoint must be reachable from the Lambda function. This meant, `Lambda need to connect your VPC`.\n\n## References\n\n- https://github.com/aws-samples/aws-lambda-extensions\n- https://github.com/fluent/fluent-logger-golang/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fculiops%2Ffluentd-lambda-extension","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fculiops%2Ffluentd-lambda-extension","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fculiops%2Ffluentd-lambda-extension/lists"}