{"id":32149424,"url":"https://github.com/apodini/swift-log-elk","last_synced_at":"2025-10-21T09:57:19.044Z","repository":{"id":48603559,"uuid":"382035122","full_name":"Apodini/swift-log-elk","owner":"Apodini","description":"A logging backend for swift-log that sends logging messages to Logstash (eg. the ELK stack)","archived":false,"fork":false,"pushed_at":"2022-11-12T16:00:16.000Z","size":298,"stargazers_count":25,"open_issues_count":3,"forks_count":17,"subscribers_count":1,"default_branch":"develop","last_synced_at":"2025-10-21T09:57:07.343Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://Apodini.github.io/swift-log-elk/","language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Apodini.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-07-01T13:03:07.000Z","updated_at":"2024-08-09T05:42:31.000Z","dependencies_parsed_at":"2022-09-05T12:31:31.309Z","dependency_job_id":null,"html_url":"https://github.com/Apodini/swift-log-elk","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":"Apodini/Template-Repository","purl":"pkg:github/Apodini/swift-log-elk","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Apodini%2Fswift-log-elk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Apodini%2Fswift-log-elk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Apodini%2Fswift-log-elk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Apodini%2Fswift-log-elk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Apodini","download_url":"https://codeload.github.com/Apodini/swift-log-elk/tar.gz/refs/heads/develop","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Apodini%2Fswift-log-elk/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":280240315,"owners_count":26296527,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-21T02:00:06.614Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-21T09:57:17.347Z","updated_at":"2025-10-21T09:57:19.039Z","avatar_url":"https://github.com/Apodini.png","language":"Swift","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LoggingELK\n\n![Swift5.4+](https://img.shields.io/badge/Swift-5.4%2B-orange.svg?style=flat)\n[![release](https://img.shields.io/github/v/release/Apodini/swift-log-elk.svg?include_prereleases\u0026color=blue)](https://github.com/Apodini/swift-log-elk/releases)\n[![codecov](https://codecov.io/gh/Apodini/swift-log-elk/branch/develop/graph/badge.svg?token=M9a8FsTExH)](https://codecov.io/gh/Apodini/swift-log-elk)\n[![jazzy](https://raw.githubusercontent.com/Apodini/swift-log-elk/gh-pages/badge.svg)](https://apodini.github.io/swift-log-elk/)\n[![Build and Test](https://github.com/Apodini/swift-log-elk/actions/workflows/build-and-test.yml/badge.svg)](https://github.com/Apodini/swift-log-elk/actions/workflows/build-and-test.yml)\n[![license](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/Apodini/swift-log-elk/blob/master/LICENSE)\n\n### **LoggingELK is a logging backend library for Apple's swift-log**\n\nThe LoggingELK library provides a logging backend for Apple's [apple/swift-log](https://github.com/apple/swift-log/) package (which basically just defines a logging API). The log entries are properly formatted, cached, and then uploaded via HTTP/HTTPS to [elastic/logstash](https://github.com/elastic/logstash), which allows for further processing in its pipeline. The logs can then be stored in [elastic/elasticsearch](https://github.com/elastic/elasticsearch) and visualized in [elastic/kibana](https://github.com/elastic/kibana).\n\n## Features\n- Written completly in Swift\n- Supports both Darwin (macOS) and Linux platforms\n- Uploads the log data automatically to Logstash (eg. the ELK stack)\n- Caches the created log entries and sends them via HTTP either periodically or when exceeding a certain configurable memory threshold to Logstash\n- Converts the logging metadata to a JSON representation, which allows querying after those values (eg. filter after a specific parameter in Kibana)\n- Logs itself via a background activity logger (including protection against a possible infinite recursion)\n\n## Setup\n\nLoggingELK requires Xcode 12 or a Swift 5.4 toolchain with the Swift Package Manager. \n\n### Swift Package Manager\n\nAdd `swift-log` and the `swift-log-elk` package as a dependency to your `Package.swift` file.\n\n```swift\ndependencies: [\n    .package(url: \"https://github.com/apple/swift-log.git\", from: \"1.0.0\"),\n    .package(url: \"https://github.com/Apodini/swift-log-elk.git\", from: \"0.1.0\")\n]\n```\n\nAdd `Logging` (from `swift-log`) and `LoggingELK` (from `swift-log-elk`) to your target's dependencies.\n\n```swift\ntargets: [\n    .target(\n        name: \"ExampleWebService\",\n        dependencies: [\n            .product(name: \"Logging\", package: \"swift-log\"),\n            .product(name: \"LoggingELK\", package: \"swift-log-elk\")\n        ]\n    )\n]\n```\n\n### Setup Logging\n\nImport both `Logging` and `LoggingELK` modules:\n\n```swift\nimport Logging\nimport LoggingELK\n```\n\nSetup the `LogstashLogHandler` with the appropriate configuration and register the to be used logging backend once (!) during the lifetime of the application:\n\n```swift\n// Setup of LogstashLogHandler\nLogstashLogHandler.setup(hostname: \"0.0.0.0\", port: 31311)\n\n// Register LogstashLogHandler in the LoggingSystem\nLoggingSystem.bootstrap(LogstashLogHandler.init)\n```\n\n**Important:** Setup the `LogstashLogHandler` before registering it in the `LoggingSystem`!\n\nFurthermore, it's possible to register multiple logging backends. An option would be to send the logs to Logstash as well as print them to console:\n\n```swift\n// Setup of LogstashLogHandler\nLogstashLogHandler.setup(hostname: \"0.0.0.0\", port: 31311)\n\n// Register LogHandlers in the LoggingSystem\nLoggingSystem.bootstrap { label in\n    MultiplexLogHandler(\n        [\n            LogstashLogHandler(label: label),\n            StreamLogHandler.standardOutput(label: label)\n        ]\n    ) \n}\n```\n\nThe `LogstashLogHandler` can also be configured beyond the standard configuration values. Below you can see an example of the maximum possible configuration options. The developer can eg. specify if HTTPS (so TLS encryption) should be used, the to be used `EventLoopGroup` for handeling the HTTP requests, a `Logger` that logs background activity of the `LogstashLogHandler` or network connectivity, and a certain `uploadInterval`, so in what time intervals the log data should be uploaded to Logstash. Furthermore, the size of the buffer that caches the log data can be configured as well as the maximum total size of all the log buffers (since temporary buffers are created during uploading).\n\n**Important:** The `maximumTotalLogStorageSize` MUST be at least twice as large as the `logStorageSize` (this is also validated during instanciation of the `LogstashLogHandler`). The reason for this are the temporary buffers that are allocated during uploading of the log data, so that a simultaneous logging call doesn't block (except for the duration it takes to copy the logs to the temporary buffer which is very fast). \n\nWhy at least twice as large? The process of allocating temporary buffers could possibly be repeated, if the log storage runs full during uploading of \"old\" log data. A possible scenario is an environment, where the network conncection to Logstash is really slow and therefore the uploading takes long. This process could repeat itself over and over again until the `maximumTotalLogStorageSize` is reached. Then, a new logging call blocks until enought memory space is available again, achieved through a partial completed uploading of log data, resulting in freed temporary buffers. In practice, approaching the `maximumTotalLogStorageSize` should basically never happen, except in very resource restricted environments.\n\n```swift\n// Setup of LogstashLogHandler\nLogstashLogHandler.setup(\n    hostname: \"0.0.0.0\",\n    port: 31311,\n    useHTTPS: false,\n    eventLoopGroup: eventLoopGroup,\n    backgroundActivityLogger: logger,\n    uploadInterval: TimeAmount.seconds(5),\n    logStorageSize: 500_000,\n    maximumTotalLogStorageSize: 4_000_000\n)\n\n// Register LogstashLogHandler in the LoggingSystem\nLoggingSystem.bootstrap(LogstashLogHandler.init)\n```\n\nNow that the setup of the `LogstashLogHandler` is completed, you can use `SwiftLog` as usual (also with metadata etc.). \n\n```swift\nimport Logging\n\nlet logger = Logger(label: \"com.example.WebService\")\n\nlogger.info(\"This is a test!\")\n```\n\n### Setup Logstash (ELK stack)\n\nTo actually use the `LogstashLogHandler`, there's obviously one last step left: Set up a [elastic/logstash](https://github.com/elastic/logstash) instance where the logs are sent to. \nThe probably most easy setup of a local Logstash instance is to use [docker/elk](https://github.com/deviantony/docker-elk), which provides the entire Elastic stack (ELK) powered by [Docker](https://www.docker.com/) and [Docker-compose](https://docs.docker.com/compose/). The ELK stack allows us to collect, analyze and present the log data and much much more. Please follow the instructions in the [README.me](https://github.com/deviantony/docker-elk#readme) of the repository to setup and configure the ELK stack correctly.\n\nThen, we need to configure the Logstash pipeline to accept HTTP input on a certain host and port. This can be done in the [Logstash pipeline configuration file](https://github.com/deviantony/docker-elk/blob/main/logstash/pipeline/logstash.conf). \nJust adapt the `input` section of the file like this to allow logs to be uploaded locally on port 31311:\n\n```\ninput {\n    http {\n        host =\u003e \"0.0.0.0\"\n        port =\u003e 31311\n    }\n}\n```\n\nFurthermore, to use the timestamp created by the `LogstashLogHandler` (not the timestamp when the data is actually sent to Logstash), adapt the `filter` section of the [Logstash pipeline configuration file](https://github.com/deviantony/docker-elk/blob/main/logstash/pipeline/logstash.conf) like shown below. The second option eliminates the headers of the HTTP request from the `LogstashLogHandler` to Logstash, since those headers would also have been saved to the log entry (which are definitly not relevant to us).\n\n```\nfilter {\n    date {\n        match =\u003e [ \"timestamp\", \"ISO8601\" ]\n        locale =\u003e \"en_US\"       # POSIX\n        target =\u003e \"@timestamp\"\n    }\n\n    mutate {\n        remove_field =\u003e [\"headers\"]\n    }\n}\n```\n\nNow that the entire setup process is finished, create some log data that is then automatically sent to Logstash (eg. see [section above](#setup-logging)). \n\nSince we use the entire ELK stack, not just Logstash, we can use [elastic/kibana](https://github.com/elastic/kibana) to instantly visualize the uploaded log data. Access the Kibana web interface (on the respective port) and navigate to `Analytics/Discover`. Your created log messages (including metadata) should now be displayed here:\n\n![image](https://user-images.githubusercontent.com/25406915/127134981-45e0ce7f-9718-4550-a0b1-e1138e8035e4.png)\n\nCongrats, you sent your first logs via swift-log and swift-log-elk to [elastic/logstash](https://github.com/elastic/logstash), saved them in  [elastic/elasticsearch](https://github.com/elastic/elasticsearch) and visualized them with [elastic/kibana](https://github.com/elastic/kibana)! 🎉\n\n## Usage\n\nFor details on how to use the Logging features of [apple/swift-log](https://github.com/apple/swift-log/) exactly, please check out the [documentation of swift-log](https://github.com/apple/swift-log#readme).\n\n## Documentation\n\nTake a look at our [API reference](https://apodini.github.io/swift-log-elk/) for a full documentation of the package.\n\n## Contributing\nContributions to this project are welcome. Please make sure to read the [contribution guidelines](https://github.com/Apodini/.github/blob/release/CONTRIBUTING.md) first.\n\n## License\nThis project is licensed under the MIT License. See [License](https://github.com/Apodini/swift-log-elk/blob/release/LICENSE) for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapodini%2Fswift-log-elk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapodini%2Fswift-log-elk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapodini%2Fswift-log-elk/lists"}