{"id":27093785,"url":"https://github.com/schwamster/logging-lifecycle","last_synced_at":"2026-05-05T21:33:46.315Z","repository":{"id":128201786,"uuid":"81933146","full_name":"schwamster/logging-lifecycle","owner":"schwamster","description":null,"archived":false,"fork":false,"pushed_at":"2017-02-16T11:03:32.000Z","size":1881,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-01-26T11:12:39.584Z","etag":null,"topics":["aws","glacier","kinesis","life","logging","s3","sql","terraform"],"latest_commit_sha":null,"homepage":null,"language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/schwamster.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-02-14T10:19:56.000Z","updated_at":"2017-02-14T20:53:55.000Z","dependencies_parsed_at":null,"dependency_job_id":"dc41bd28-08f2-4f6e-90c1-16e26d5153c6","html_url":"https://github.com/schwamster/logging-lifecycle","commit_stats":{"total_commits":22,"total_committers":2,"mean_commits":11.0,"dds":0.4545454545454546,"last_synced_commit":"004d5987094042cab115f3ded2c3fb8a7b23fdb4"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/schwamster%2Flogging-lifecycle","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/schwamster%2Flogging-lifecycle/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/schwamster%2Flogging-lifecycle/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/schwamster%2Flogging-lifecycle/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/schwamster","download_url":"https://codeload.github.com/schwamster/logging-lifecycle/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247457743,"owners_count":20941906,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","glacier","kinesis","life","logging","s3","sql","terraform"],"created_at":"2025-04-06T08:50:02.582Z","updated_at":"2026-05-05T21:33:41.289Z","avatar_url":"https://github.com/schwamster.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# logging-lifecycle\n\nkeywords: aws, terraform, kinesis, lambda, log groups, s3, glacier, sqs\n\nthis repo is supposed to provide an example of how to create a log flow. push your log events from the log groups to s3, glacier, and sqs.\nall resources will be created with terraform. \n\n## prerequisites\n\neither install aws-cli and sign in or create or use an existing ssh key and you can then run the terraform scripts directly. Otherwise you add the keys in main.tf as explained here: [terraform docs](https://www.terraform.io/docs/providers/aws/index.html)\nhave some event source that pushes logs to a cloud watch log group. an easy example would be cloud trail -\u003e cloud watch see here: http://docs.aws.amazon.com/awscloudtrail/latest/userguide/send-cloudtrail-events-to-cloudwatch-logs.html\n\n## getting started\n\nmake sure you somehow provide authentification (see prerequisites).\n\nupdate variables.tf with your settings:\n\n* aws_region: region you want to use\n* log_group_name: log group that is used as the source for the events that will be pushed to kinesis\n\nthen run:\n\n    terraform plan\n\nto check out what resources would be created\n\nthen run: \n\n    terraform apply\n\nto create the resources (this can take a couple of minutes)\n\n\nif you want to destroy the resources run:\n\n    terraform destroy\n\n\n## overview\n\n![overview](logging-lifecyle.png)\n\nacknowledgement: diagram made with [draw.io](https://www.draw.io)\n\n\n## disclaimer\n\nrunning this terraform script is most likly going to cost you some money just as it would cost you money to create those resources via aws-cli, the console or other means.\n\n\n## The Lambda function that moves events to s3\n\nThe lambda function is created with dotnet core. I used amazons yeoman template to create =\u003e [yeoman template github](https://github.com/aws/aws-lambda-dotnet/tree/master/Blueprints)\nif you want to change the function you can build a new deployable package by navigating to MoveLogsToS3/src/MoveLogsToS3 and run \n\n    dotnet lambda package\n\nthen navigate to bin/Release/netcoreapp1.0 and copy MoveLogsToS3.zip to /lambdas\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fschwamster%2Flogging-lifecycle","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fschwamster%2Flogging-lifecycle","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fschwamster%2Flogging-lifecycle/lists"}