{"id":24494309,"url":"https://github.com/SWO-GS/athena-cloudtrail-partitioner","last_synced_at":"2025-10-02T15:31:15.688Z","repository":{"id":44081815,"uuid":"198960431","full_name":"SWO-GS/athena-cloudtrail-partitioner","owner":"SWO-GS","description":"Automate the daily partitioning of your CloudTrail bucket in Athena","archived":true,"fork":false,"pushed_at":"2023-10-19T18:12:57.000Z","size":687,"stargazers_count":28,"open_issues_count":18,"forks_count":7,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-05-16T07:40:33.166Z","etag":null,"topics":["athena","aws","aws-athena","aws-glue","cloudtrail","cloudtrail-logs","glue","gorillastack","partitioning"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SWO-GS.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-26T06:34:27.000Z","updated_at":"2025-01-15T08:28:17.000Z","dependencies_parsed_at":"2024-01-13T16:23:11.305Z","dependency_job_id":"e938ae20-c494-4c4b-a1f7-bcc6a07c2306","html_url":"https://github.com/SWO-GS/athena-cloudtrail-partitioner","commit_stats":null,"previous_names":["swo-gs/athena-cloudtrail-partitioner"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/SWO-GS/athena-cloudtrail-partitioner","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SWO-GS%2Fathena-cloudtrail-partitioner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SWO-GS%2Fathena-cloudtrail-partitioner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SWO-GS%2Fathena-cloudtrail-partitioner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SWO-GS%2Fathena-cloudtrail-partitioner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SWO-GS","download_url":"https://codeload.github.com/SWO-GS/athena-cloudtrail-partitioner/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SWO-GS%2Fathena-cloudtrail-partitioner/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278028937,"owners_count":25918069,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-02T02:00:08.890Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["athena","aws","aws-athena","aws-glue","cloudtrail","cloudtrail-logs","glue","gorillastack","partitioning"],"created_at":"2025-01-21T20:00:55.965Z","updated_at":"2025-10-02T15:31:15.680Z","avatar_url":"https://github.com/SWO-GS.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"# Athena CloudTrail Partitioner\n\nAWS Athena is a serverless query service that helps you query your unstructured S3 data without all the ETL.\n\nAthena allows you to query your CloudTrail log data from your S3 bucket on demand. However, it can be challenging to maintain sensible partitioning on the database over time.\n\nThis project helps you periodically add partitions to your Athena/Glue database for each day/month/year/region/account added to your CloudTrail log bucket.\n\n[Read more about why we built this, and how it can be used, in this blog post](https://www.gorillastack.com/news/cloudtrail-athena-query/).\n\n## Prerequisite - Enable CloudTrail\n\nCloudTrail is an audit log of every action to occur in your AWS Action. It should be on all the time.\n\nYou can now [enable CloudTrail at the AWS Organization level](https://docs.aws.amazon.com/organizations/latest/userguide/services-that-can-integrate-ct.html), which means that CloudTrail for each account will be centrally logged and automatically enabled for all new accounts.\n\nRead about how to [create your organization CloudTrail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/creating-trail-organization.html) here.\n\n## Installation\n\nInstall the Athena CloudTrail Partitioner through CloudFormation, either through the AWSCLI:\n\n```\naws cloudformation deploy \\\n  --stack-name athena-cloudtrail-partitioner \\\n  --region ${AWS_DEFAULT_REGION} \\\n  --template-file cf/template.yml \\\n  --force-upload \\\n  --parameter-overrides \\\n    \"OrganizationId=${ORGANIZATION_ID}\" \\\n    \"S3BucketName=${S3_BUCKET_NAME}\" \\\n  --capabilities CAPABILITY_NAMED_IAM \\\n  --no-fail-on-empty-changeset\n```\n\nor click this button to deploy throught the AWS Console:\n\n[![Launch Stack](https://cdn.rawgit.com/buildkite/cloudformation-launch-stack-button-svg/master/launch-stack.svg)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/quickcreate?stackName=athena-cloudtrail-partitioner\u0026templateUrl=https%3A%2F%2Fgorillastack-cloudformation-templates.s3.amazonaws.com%2Fathena-cloudtrail-partitioner.yml)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSWO-GS%2Fathena-cloudtrail-partitioner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FSWO-GS%2Fathena-cloudtrail-partitioner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSWO-GS%2Fathena-cloudtrail-partitioner/lists"}