{"id":20066279,"url":"https://github.com/ossobv/swiftbunny","last_synced_at":"2026-05-13T06:03:17.739Z","repository":{"id":57472695,"uuid":"271507412","full_name":"ossobv/swiftbunny","owner":"ossobv","description":"CLI/daemon to help with log archiving to OpenStack Swift","archived":false,"fork":false,"pushed_at":"2023-12-18T13:00:32.000Z","size":100,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-02-15T19:19:12.285Z","etag":null,"topics":["archiving","openstack-swift","rabbitmq"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ossobv.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-06-11T09:37:39.000Z","updated_at":"2023-04-07T09:24:58.000Z","dependencies_parsed_at":"2022-08-31T02:00:24.832Z","dependency_job_id":null,"html_url":"https://github.com/ossobv/swiftbunny","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ossobv%2Fswiftbunny","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ossobv%2Fswiftbunny/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ossobv%2Fswiftbunny/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ossobv%2Fswiftbunny/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ossobv","download_url":"https://codeload.github.com/ossobv/swiftbunny/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241494184,"owners_count":19971871,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["archiving","openstack-swift","rabbitmq"],"created_at":"2024-11-13T13:55:46.277Z","updated_at":"2026-05-13T06:03:17.675Z","avatar_url":"https://github.com/ossobv.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"SwiftBunny :: CLI/daemon to help with log archiving to OpenStack Swift\n======================================================================\n\nReads RabbitMQ, reads Graylog+ElasticSearch JSON dumps.\n\nWrites daily/hourly logs files to OpenStack Swift.\n\nEncrypts data on the fly using *qpress* (``.qz1``) and *GnuPG*.\n\n*Why?* Organised long term storage of logs.\n\n\nOutput file format\n------------------\n\nTime is duplicated in the file, as the datetime is taken from the JSON object.\nThe compression will take care of excess size::\n\n    2020-01-25T00:00:12.217000Z: {\"some\": \"json\", \"dict\": \"...\"}\n    2020-01-25T00:00:12.496000Z: {\"another\": \"json\", \"object\": \"...\"}\n    ...\n\nTarget file names will look like::\n\n    example-swift-container-2020/example.rabbit.queue/2020/05/31/\n      example.rabbit-queue_2020-05-25T00:00:00Z+3600-0x72272_keyid.qz1.gpg\n\nWhich means:\n\n* the target container is ``example-swift-container-2020``;\n\n* the source queue is ``example.rabbit.queue``;\n\n* the first message is after ``2020-05-25``, at ``00:00:00`` (UTC);\n\n* the last message is before that time ``+3600`` (i.e. one hour later);\n\n* there are *no* late message (``-0``);\n\n* the file contains ``72272`` lines (records).\n\n* ``keyid`` may describe which node this was produced on, and a short\n  identifier which symmetric gpg key was used.\n\n\nSo, if you're looking for old records, you can:\n\n* find all files in the right period (also check nearby files with\n  ``-N``-times);\n\n* fetch, decrypt and decompress the files;\n\n* scan the lines by the consistently formatted timestamps.\n\n\nTODO\n----\n\n* Add commands, so we can do 'ls' and 'get' and 'post' on the archived files.\n  (Use argparse.)\n\n* Add sample ini-file.\n\n* Fix suffix (split node-X and keyname).\n\n* Add Immutable middleware to Swift so we can make it harder to\n  accidentally delete stored logs.\n\n* Don't forget about chmod 600 in swiftbunny.ini and chmod 700 on\n  /var/spool/swiftbunny.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fossobv%2Fswiftbunny","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fossobv%2Fswiftbunny","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fossobv%2Fswiftbunny/lists"}