https://github.com/getindata/terraform-snowflake-stage
Terraform module for managing Snowflake stages
https://github.com/getindata/terraform-snowflake-stage
Last synced: 5 months ago
JSON representation
Terraform module for managing Snowflake stages
- Host: GitHub
- URL: https://github.com/getindata/terraform-snowflake-stage
- Owner: getindata
- License: apache-2.0
- Created: 2023-01-12T09:27:20.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2025-01-13T01:59:35.000Z (over 1 year ago)
- Last Synced: 2025-01-24T02:30:20.664Z (over 1 year ago)
- Language: HCL
- Size: 64.5 KB
- Stars: 1
- Watchers: 6
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# Snowflake Stage Terraform Module
We help companies turn their data into assets
---
Terraform module for Snowflake stage management.
- Creates Snowflake stage
- Can create custom Snowflake database-roles with role-to-role assignments
- Can create a set of default roles to simplify access management:
- `READONLY` - granted `USAGE` or `READ` privilages
- `READWRITE` - granted `WRITE` privileges
- `ADMIN` - granted `ALL PRIVILEGES`
## USAGE
```terraform
module "snowflake_stage" {
source = "getindata/stage/snowflake"
# version = "x.x.x"
name = "my_stage"
schema = "my_schema"
database = "my_db"
url = "s3://com.example.bucket/prefix"
credentials = "AWS_KEY_ID='${var.example_aws_key_id}' AWS_SECRET_KEY='${var.example_aws_secret_key}'"
create_default_database_roles = true
}
```
## EXAMPLES
- [Simple](examples/simple) - Basic usage of the module
- [Complete](examples/complete) - Advanced usage of the module
## Breaking changes in v2.x of the module
Due to breaking changes in Snowflake provider and additional code optimizations, **breaking changes** were introduced in `v2.0.0` version of this module.
List of code and variable (API) changes:
- Switched to `snowflake_grant_ownership` resource instead of provider-removed `snowflake_role_ownership_grant`
- Switched to `snowflake_database_role` module to leverage new `database_roles` mechanism
- `default_roles` and `custom_roles` are now combined and managed by single module
- `roles` variable map received following additions:
- `all_privileges` - optional, bool
- `on_all` - optional, bool, defaults to false
- `on_future` - optional, bool, defaults to false
- `with_grant_option` - optional, bool
- `granted_to_database_roles` - optional, string
- `granted_database_roles` - optional, list of strings
- and got following items removed:
- `enabled`
- `comment`
- `role_ownership_grant`
- `granted_roles`
- `granted_to_users`
When upgrading from `v1.x`, expect most of the resources to be recreated - if recreation is impossible, then it is possible to import some existing resources.
For more information, refer to [variables.tf](variables.tf), list of inputs below and Snowflake provider documentation
## Breaking changes in v3.x of the module
Due to replacement of nulllabel (`context.tf`) with context provider, some **breaking changes** were introduced in `v3.0.0` version of this module.
List od code and variable (API) changes:
- Removed `context.tf` file (a single-file module with additonal variables), which implied a removal of all its variables (except `name`):
- `descriptor_formats`
- `label_value_case`
- `label_key_case`
- `id_length_limit`
- `regex_replace_chars`
- `label_order`
- `additional_tag_map`
- `tags`
- `labels_as_tags`
- `attributes`
- `delimiter`
- `stage`
- `environment`
- `tenant`
- `namespace`
- `enabled`
- `context`
- Remove support `enabled` flag - that might cause some backward compatibility issues with terraform state (please take into account that proper `move` clauses were added to minimize the impact), but proceed with caution
- Additional `context` provider configuration
- New variables were added, to allow naming configuration via `context` provider:
- `context_templates`
- `name_schema`
## Breaking changes in v4.x of the module
- Due to rename of Snowflake terraform provider source, all `versions.tf` files were updated accordingly.
Please keep in mind to mirror this change in your own repos also.
For more information about provider rename, refer to [Snowflake documentation](https://github.com/snowflakedb/terraform-provider-snowflake/blob/main/SNOWFLAKEDB_MIGRATION.md).
- Maximal version of supported provider was also unblocked in version `v4.1.x` , so keep in mind that, starting with Snowflake provider version `1.x`, the `snowflake_stage` resource is considered a preview feature and must be explicitly enabled in the provider configuration.
**Required Provider Configuration:**
```terraform
provider "snowflake" {
preview_features_enabled = ["snowflake_stage_resource"]
}
```
Without this configuration, you will encounter the following error:
```shell
Error: snowflake_stage_resource is currently a preview feature, and must be enabled by adding snowflake_stage_resource to preview_features_enabled in Terraform configuration.
```
For more information about preview features, refer to the [Snowflake provider documentation](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/stage#preview-features) and [Snowflake stage resource documentation](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/stage).
## Inputs
| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| [aws\_external\_id](#input\_aws\_external\_id) | ID of the customer AWS account | `string` | `null` | no |
| [comment](#input\_comment) | Specifies a comment for the stage | `string` | `null` | no |
| [context\_templates](#input\_context\_templates) | Map of context templates used for naming conventions - this variable supersedes `naming_scheme.properties` and `naming_scheme.delimiter` configuration | `map(string)` | `{}` | no |
| [copy\_options](#input\_copy\_options) | Specifies the copy options for the stage | `string` | `null` | no |
| [create\_default\_roles](#input\_create\_default\_roles) | Whether the default database roles should be created | `bool` | `false` | no |
| [credentials](#input\_credentials) | Specifies the credentials for the stage | `string` | `null` | no |
| [database](#input\_database) | The database in which to create the stage | `string` | n/a | yes |
| [directory](#input\_directory) | Specifies the directory settings for the stage | `string` | `null` | no |
| [encryption](#input\_encryption) | Specifies the encryption settings for the stage | `string` | `null` | no |
| [file\_format](#input\_file\_format) | Specifies the file format for the stage | `string` | `null` | no |
| [name](#input\_name) | Name of the resource | `string` | n/a | yes |
| [name\_scheme](#input\_name\_scheme) | Naming scheme configuration for the resource. This configuration is used to generate names using context provider:
- `properties` - list of properties to use when creating the name - is superseded by `var.context_templates`
- `delimiter` - delimited used to create the name from `properties` - is superseded by `var.context_templates`
- `context_template_name` - name of the context template used to create the name
- `replace_chars_regex` - regex to use for replacing characters in property-values created by the provider - any characters that match the regex will be removed from the name
- `extra_values` - map of extra label-value pairs, used to create a name
- `uppercase` - convert name to uppercase |
object({
properties = optional(list(string), ["name"])
delimiter = optional(string, "_")
context_template_name = optional(string, "snowflake-stage")
replace_chars_regex = optional(string, "[^a-zA-Z0-9_]")
extra_values = optional(map(string))
uppercase = optional(bool, true)
}) | `{}` | no |
| [roles](#input\_roles) | Database roles created in the stage scope | map(object({
name_scheme = optional(object({
properties = optional(list(string))
delimiter = optional(string)
context_template_name = optional(string)
replace_chars_regex = optional(string)
extra_labels = optional(map(string))
uppercase = optional(bool)
}))
comment = optional(string)
with_grant_option = optional(bool)
granted_to_roles = optional(list(string))
granted_to_database_roles = optional(list(string))
granted_database_roles = optional(list(string))
stage_grants = optional(list(string))
all_privileges = optional(bool)
})) | `{}` | no |
| [schema](#input\_schema) | The schema in which to create the stage | `string` | n/a | yes |
| [snowflake\_iam\_user](#input\_snowflake\_iam\_user) | Specifies the Snowflake IAM user | `string` | `null` | no |
| [stage\_ownership\_grant](#input\_stage\_ownership\_grant) | To which account role the stage ownership should be granted | `string` | `null` | no |
| [storage\_integration](#input\_storage\_integration) | Specifies the name of the storage integration used to delegate authentication responsibility for external cloud storage to a Snowflake identity and access management (IAM) entity | `string` | `null` | no |
| [url](#input\_url) | Specifies the URL for the stage | `string` | `null` | no |
## Modules
| Name | Source | Version |
|------|--------|---------|
| [roles\_deep\_merge](#module\_roles\_deep\_merge) | Invicton-Labs/deepmerge/null | 0.1.5 |
| [snowflake\_custom\_role](#module\_snowflake\_custom\_role) | getindata/database-role/snowflake | 3.0.0 |
| [snowflake\_default\_role](#module\_snowflake\_default\_role) | getindata/database-role/snowflake | 3.0.0 |
## Outputs
| Name | Description |
|------|-------------|
| [database\_roles](#output\_database\_roles) | This stage access roles |
| [fully\_qualified\_name](#output\_fully\_qualified\_name) | Fully Qualified Name of the stage |
| [name](#output\_name) | Name of the stage |
## Providers
| Name | Version |
|------|---------|
| [context](#provider\_context) | >=0.4.0 |
| [snowflake](#provider\_snowflake) | >= 0.95 |
## Requirements
| Name | Version |
|------|---------|
| [terraform](#requirement\_terraform) | >= 1.3 |
| [context](#requirement\_context) | >=0.4.0 |
| [snowflake](#requirement\_snowflake) | >= 0.95 |
## Resources
| Name | Type |
|------|------|
| [snowflake_grant_ownership.stage_ownership](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/grant_ownership) | resource |
| [snowflake_stage.this](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/stage) | resource |
| [context_label.this](https://registry.terraform.io/providers/cloudposse/context/latest/docs/data-sources/label) | data source |
## CONTRIBUTING
Contributions are very welcomed!
Start by reviewing [contribution guide](CONTRIBUTING.md) and our [code of conduct](CODE_OF_CONDUCT.md). After that, start coding and ship your changes by creating a new PR.
## LICENSE
Apache 2 Licensed. See [LICENSE](LICENSE) for full details.
## AUTHORS
Made with [contrib.rocks](https://contrib.rocks).