Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/fivexl/terraform-aws-s3-access-logs-athena-table
This Terraform module creates an AWS Glue table designed specifically to query Amazon S3 server access logs in the new date-based partitioning format.
https://github.com/fivexl/terraform-aws-s3-access-logs-athena-table
Last synced: 8 days ago
JSON representation
This Terraform module creates an AWS Glue table designed specifically to query Amazon S3 server access logs in the new date-based partitioning format.
- Host: GitHub
- URL: https://github.com/fivexl/terraform-aws-s3-access-logs-athena-table
- Owner: fivexl
- License: apache-2.0
- Created: 2023-11-24T18:31:58.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-02-29T10:37:17.000Z (10 months ago)
- Last Synced: 2024-11-06T19:30:23.260Z (about 2 months ago)
- Language: HCL
- Homepage: https://registry.terraform.io/modules/fivexl/s3-access-logs-athena-table/aws
- Size: 17.6 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# terraform-aws-s3-access-logs-athena-table
[](https://fivexl.io/)
This Terraform module creates an AWS Glue table designed specifically to query Amazon S3 server access logs in the new [date-based partitioning](https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-s3-server-access-logging-date-partitioning/) format [^1].
Date-based partitioning improves performance and cost-efficiency of downstream log processing systems by limiting the scanning of logs to only the desired time range.
## Usage
```hcl
module "s3_access_logs_glue_table" {
source = "github.com/fivexl/terraform-aws-s3-access-logs-athena-table/?ref=main"
name = "s3_server_access_logs"
database_name = aws_glue_catalog_database.s3_access_logs_db.name
location = "s3://${local.logging_bucket_name}/s3-server-access-logs"
}
```
## Links that may be useful
- [How do I enable log delivery?](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ServerLogs.html#server-access-logging-overview)
- [Querying access logs for requests by using Amazon Athena](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-s3-access-logs-to-identify-requests.html#querying-s3-access-logs-for-requests)
## Requirements
| Name | Version |
|------|---------|
| [terraform](#requirement\_terraform) | ~> 1.0 |
| [aws](#requirement\_aws) | >= 4.9.0 |
## Providers
| Name | Version |
|------|---------|
| [aws](#provider\_aws) | >= 4.9.0 |
## Resources
| Name | Type |
|------|------|
| [aws_glue_catalog_table.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/glue_catalog_table) | resource |
## Inputs
| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| [database\_name](#input\_database\_name) | Name of AWS Glue database to create table in | `string` | n/a | yes |
| [date\_based\_partitioning](#input\_date\_based\_partitioning) | Whether to enable date-based partitioning | `bool` | `true` | no |
| [location](#input\_location) | S3 location of S3 access logs. This is the same url you specify in the S3 bucket when you enable S3 server access logging - e.g. s3://my-bucket-name/s3-access-logs | `string` | n/a | yes |
| [name](#input\_name) | Name of AWS Glue table to create | `string` | `"s3_access_logs"` | no |
## Outputs
| Name | Description |
|------|-------------|
| [aws\_glue\_catalog\_table\_arn](#output\_aws\_glue\_catalog\_table\_arn) | The ARN of the Glue Table. |
| [aws\_glue\_catalog\_table\_id](#output\_aws\_glue\_catalog\_table\_id) | The ID of the Glue Table. |
---
[^1]: [Amazon S3 server access logging now supports automatic date-based partitioning](https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-s3-server-access-logging-date-partitioning/)
## Weekly review link
- [Review](https://github.com/fivexl/terraform-aws-s3-access-logs-athena-table/compare/main@%7B7day%7D...main)
- [Review branch-based review](https://github.com/fivexl/terraform-aws-s3-access-logs-athena-table/compare/review...main)