{"id":28524001,"url":"https://github.com/cloudbase/coriolis-logger","last_synced_at":"2025-07-06T05:31:22.250Z","repository":{"id":147427652,"uuid":"216844508","full_name":"cloudbase/coriolis-logger","owner":"cloudbase","description":"Coriolis Logging","archived":false,"fork":false,"pushed_at":"2025-04-16T22:20:58.000Z","size":87,"stargazers_count":3,"open_issues_count":1,"forks_count":7,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-06-09T10:11:35.762Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudbase.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-22T15:14:35.000Z","updated_at":"2024-07-19T06:35:32.000Z","dependencies_parsed_at":"2024-07-22T14:16:49.309Z","dependency_job_id":null,"html_url":"https://github.com/cloudbase/coriolis-logger","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/cloudbase/coriolis-logger","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudbase%2Fcoriolis-logger","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudbase%2Fcoriolis-logger/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudbase%2Fcoriolis-logger/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudbase%2Fcoriolis-logger/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudbase","download_url":"https://codeload.github.com/cloudbase/coriolis-logger/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudbase%2Fcoriolis-logger/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263853265,"owners_count":23520114,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-09T10:10:14.728Z","updated_at":"2025-07-06T05:31:22.243Z","avatar_url":"https://github.com/cloudbase.png","language":"Go","readme":"# Coriolis logger\n\nCoriolis logger is a central logging daemon created to integrate in a Coriolis deployment. It is a small Syslog server, with support for multiple writers, both persistent (InfluxDB) and transient (web sockets, standard output). The goal of this project is to allow easy streaming of logs to any web sockets enabled destination (WebUI, CLI, etc), as well as allow users of Coriolis to easily download logs that are aggregated by this service.\n\nConceivably this app can be used with any application that logs to Syslog, and data-store packages can be written for it to allow sending logs to any backend. Currently we support the following writers and data stores:\n\n  * Data stores:\n    * InfluxDB\n  * Writers:\n    * Web Sockets\n    * Standard out (testing purposes mostly)\n\n## Building the binary\n\nClone the repo:\n\n```bash\ngit clone https://github.com/cloudbase/coriolis-logger\n```\n\nBuild the binary:\n\n```bash\ncd coriolis-logger\ngo install ./...\n```\n\n## Configuration\n\nCoriolis logger uses a simple ```toml``` file as a config:\n\n```toml\n[apiserver]\nbind = \"0.0.0.0\"\nport = 9998\nuse_tls = false\n\n# Authentication middleware to use. Available options are:\n#  * keystone\n#  * none\n# coriolis-logger will refuse to start if this option is\n# missing. To disable authentication, you must explicitly\n# set this option to \"none\"\nauth_middleware = \"keystone\"\n\n    [apiserver.keystone_auth]\n    # The keystone auth URI\n    auth_uri = \"http://127.0.0.1:5000/v3\"\n\n    # API server TLS config\n    [apiserver.tls]\n    crt = \"/tmp/certificate.pem\"\n    key = \"/tmp/key.pem\"\n    cacert = \"/tmp/ca-cert.pem\"\n\n[syslog]\n# Possible values: unixgram, tcp, udp\nlistener = \"unixgram\"\n\n# possible values:\n#   for unixgram: /path/to/socket\n#   for tcp/udp IP:port pair: 0.0.0.0:5144 \n# address = \"/tmp/coriolis-logger/syslog\"\naddress = \"/tmp/coriolis-logging.sock\"\n\n# Log format\n# possible values:\n#   rfc3164\n#   rfc5424\n#   rfc6587\n#   automatic\nformat = \"automatic\"\n\n# Whether to dump logs to stdout or not\n# this should only be enabled for testng purposes\nlog_to_stdout = false\n\n# storage backend for logs. Available options are:\n#   * influxdb\ndatastore = \"influxdb\"\n\n    [syslog.influxdb]\n    url = \"http://127.0.0.1:8086\"\n    # If influxDB auth is enabled, use this username\n    username = \"coriolis\"\n    # A super secret password. Obviously needs changing :-)\n    password = \"Passw0rd\"\n    # Logs database\n    database = \"coriolis\"\n    # duration in seconds after which this datastore\n    # flushes writes to the database\n    write_interval = 5\n    # Verify server enables mutual TLS authentication\n    verify_server = false\n    # Client TLS certificates\n    # cacert = \"/tmp/ca.pem\"\n    # client_crt = \"/tmp/client-crt.pem\"\n    # client_key = \"/tmp/client-key.pem\"\n\n    # The retention period for logs in days. Logs older than\n    # this, will be deleted. If missing, this option default\n    # to 3 days. This setting will be moved in the future\n    # under the [syslog] section, when we will support multiple\n    # datastores.\n    log_retention_period = 3\n```\n\n## Usage\n\nDepending on the authentication middleware used, additional headers may need to be set.\n\nGenerally available query parameters:\n\n|    Name    | Type    | Optional | Description                                                                  |\n| ---------- | ------- | -------- | ---------------------------------------------------------------------------- |\n| auth_type  | string  |   true   | Authentication token type. Supported authentication methods are: keystone. This option must match the authentication middleware enabled in the config for coriolis-logger |\n| auth_token | string  |   true   | Authentication token/credentials for the selected auth_type. |\n\n\n### List logs\n\n```\nGET /api/v1/logs/\n```\n\nExample:\n\n```bash\n$ curl -s -H \"X-Auth-Token: \u003ctoken_goes_here\u003e\" -X GET http://127.0.0.1:9998/api/v1/logs/ | jq\n{\n  \"logs\": [\n    {\n      \"log_name\": \"coriolis-api\"\n    },\n    {\n      \"log_name\": \"coriolis-conductor\"\n    },\n    {\n      \"log_name\": \"coriolis-dbsync\"\n    },\n    {\n      \"log_name\": \"coriolis-replica-cron\"\n    },\n    {\n      \"log_name\": \"coriolis-worker\"\n    }\n  ]\n}\n```\n\nAlternatively, authentication method and authentication info can be specified in the GET query args:\n\n```bash\ncurl -s -X GET \"http://127.0.0.1:9998/api/v1/logs/?auth_type=keystone\u0026auth_token=super_secret_token\" | jq\n{\n  \"logs\": [\n    {\n      \"log_name\": \"coriolis-api\"\n    },\n    {\n      \"log_name\": \"coriolis-conductor\"\n    },\n    {\n      \"log_name\": \"coriolis-dbsync\"\n    },\n    {\n      \"log_name\": \"coriolis-replica-cron\"\n    },\n    {\n      \"log_name\": \"coriolis-worker\"\n    }\n  ]\n}\n```\n\n### Download logs\n\n```\nGET /api/v1/logs/{log_name}/\n```\n\nQuery parameters:\n\n|      Name       | Type | Optional | Description                                                                  |\n| --------------- | ---- | -------- | ---------------------------------------------------------------------------- |\n|   start_date    | int  |   true   | Unix timestamp indicating the start date from which we want to download logs |\n|    end_date     | int  |   true   | Unix timestamp indicating the end date to which we want to download logs     |\n| disable_chunked | bool |   true   | If true, coriolis-logger will attempt to disable chunked transfer.           |\n\n### Stream logs using web sockets\n\n```\nGET /api/v1/ws/\n```\n\nQuery parameters:\n\n|    Name    |  Type   | Optional | Description                                                                               |\n| ---------- | ------- | -------- | ----------------------------------------------------------------------------------------- |\n| severity   |   int   |   true   | Severity level. Values range from 0 to 7. See https://tools.ietf.org/html/rfc5424#page-11 |\n| app_name   |  string |   true   | The name of the log we wish to stream. See the \"list\" section.                            |\n\n\nExample:\n\n```python\n#!/usr/bin/env python3\n\nimport websockets\nimport asyncio\nimport json\n\ntoken = \"super secret token\"\n\nasync def hello():\n    uri = \"ws://127.0.0.1:9998/api/v1/ws/?severity=7\u0026app_name=coriolis-worker\"\n    async with websockets.connect(uri, extra_headers={\"X-Auth-Token\": token}) as websocket:\n        while True:\n            msg = await websocket.recv()\n            asDict = json.loads(msg)\n            print(asDict[\"message\"])\n\ntry:\n    asyncio.get_event_loop().run_until_complete(hello())\nexcept KeyboardInterrupt:\n    print(\"stopped\")\n\n```\n\n## Using with docker\n\nIf coriolis-logger is configured to listen on ```/tmp/coriolis-logger.sock```, to use it with a docker container, you simply have to mount the socket file as ```/dev/log``` inside the container.\n\n```bash\n$ docker run --rm -it -v /tmp/coriolis-logging.sock:/dev/log ubuntu:latest bash\nroot@415b368958bc:/# ls -l /dev/log\nsrwxrwxr-x 1 1000 1000 0 Oct 21 23:11 /dev/log\nroot@415b368958bc:/# logger --rfc5424 \"this is a test\"\n```\n\nIf you are listening on web sockets using the above example, you should now see the log message on your screen. \n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudbase%2Fcoriolis-logger","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudbase%2Fcoriolis-logger","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudbase%2Fcoriolis-logger/lists"}