Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/kwame-mintah/terraform-azure-ai-engineer-associate
Deploy all the necessary Microsoft Azure resources for the self-paced learning pathway for exam AI-102 via Terraform.
https://github.com/kwame-mintah/terraform-azure-ai-engineer-associate
ai-102 azure azure-terraform azurerm cognitive-services terraform
Last synced: 29 days ago
JSON representation
Deploy all the necessary Microsoft Azure resources for the self-paced learning pathway for exam AI-102 via Terraform.
- Host: GitHub
- URL: https://github.com/kwame-mintah/terraform-azure-ai-engineer-associate
- Owner: kwame-mintah
- Created: 2023-08-01T17:39:41.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2023-08-30T12:48:12.000Z (over 1 year ago)
- Last Synced: 2024-04-24T02:57:15.800Z (8 months ago)
- Topics: ai-102, azure, azure-terraform, azurerm, cognitive-services, terraform
- Language: HCL
- Homepage:
- Size: 193 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Terraform Azure Ai Engineer Associate
The main purpose of this repository is to terraform all the resources needed for [Exam AI-102: Designing and Implementing a Microsoft Azure AI Solution - Certification](https://learn.microsoft.com/en-us/certifications/exams/ai-102/?ns-enrollment-type=Collection&ns-enrollment-id=63rjhrqoe512d3).
The end goal is to be easily deploy all the resources needed for the [self-paced learning](https://learn.microsoft.com/en-us/users/kwame-mintah/collections/63rjhrqoe512d3) modules.
As I have created resources following the instructions in the lab exercises when using the Azure Portal UI. Please note your your mileage may (or might) vary, as these resources
were deployed using my personal account which has no restrictions.
Table of contents
- [Terraform Azure Ai Engineer Associate](#terraform-azure-ai-engineer-associate)
- [Dependencies](#dependencies)
- [Prerequisites](#prerequisites)
- [To-do list](#to-do-list)
- [Usage](#usage)
- [Cost](#cost)
- [Pre-Commit hooks](#pre-commit-hooks)
- [Documentation Generation](#documentation-generation)
- [Requirements](#requirements)
- [Providers](#providers)
- [Modules](#modules)
- [Resources](#resources)
- [Inputs](#inputs)
- [Outputs](#outputs)
## Dependencies
- [terraform](https://www.terraform.io/)
- [terragrunt](https://terragrunt.gruntwork.io/)
- [terraform-docs](https://terraform-docs.io/) this is required for `terraform_docs` hooks
- [pre-commit](https://pre-commit.com/)
## Prerequisites
1. Have a [Azure Portal](https://portal.azure.com/) account.
2. You will need to create a Service Principal with a Client Secret [follow instructions](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_client_secret#creating-a-service-principal-in-the-azure-portal).
## To-do list
- [ ] Optionally link storage account created for cognitive services
- [ ] Create scripts that will give the search services the correct Azure IAM roles via API calls
- [ ] Move the creation of storage accounts into separate module
- [ ] Terraform creation of Azure Kubernetes Service for machine learning
## Usage
1. Navigate to the environment you would like to deploy,
2. Plan your changes with `terragrunt plan` to see what changes will be made,
3. If you're happy with the changes `terragrunt apply`.
> **IMPORTANT**
>
> Please note that `.tfstate` files are stored locally on your machine on first apply, an Azure Storage account is created as part of the Terraform.
> However you will be required to migrate to it after the tfstate storage account has been created. Please see comments in `backend.tf` or any of the environment `terragrunt.hcl`.
>
## Cost
A majority of the resources created will have either the 'Standard' or 'Free' tier used, however this does not mean that it will be cheap. Please be mindful of the cost for each tier,
for example the Azure Container Instance is always running and you will be charged for it's up-time during the month. [Infracost](https://www.infracost.io/) has been used to help indicate how much it will cost
you to have all these resources created.
Predicted Infracost as of 24/08/2023
```markdown
Name Monthly Qty Unit Monthly Cost
azurerm_key_vault_key.tfstate_key_vault_key
├─ Secrets operations Monthly cost depends on usage: $0.03 per 10K transactions
├─ Storage key rotations Monthly cost depends on usage: $1.00 per renewals
└─ Software-protected keys Monthly cost depends on usage: $0.03 per 10K transactions
azurerm_log_analytics_workspace.tfstate_analytics_workspace
├─ Log data ingestion Monthly cost depends on usage: $2.99 per GB
├─ Log data export Monthly cost depends on usage: $0.13 per GB
├─ Basic log data ingestion Monthly cost depends on usage: $0.65 per GB
├─ Basic log search queries Monthly cost depends on usage: $0.0065 per GB searched
├─ Archive data Monthly cost depends on usage: $0.026 per GB
├─ Archive data restored Monthly cost depends on usage: $0.13 per GB
└─ Archive data searched Monthly cost depends on usage: $0.0065 per GB
azurerm_search_service.cognitive_search_service
├─ Search usage (Basic, 1 unit) 730 hours $73.73
└─ Image extraction (first 1M) Monthly cost depends on usage: $1.00 per 1000 images
azurerm_storage_account.tfstate
├─ Capacity Monthly cost depends on usage: $0.0392 per GB
├─ Write operations Monthly cost depends on usage: $0.11 per 10k operations
├─ List and create container operations Monthly cost depends on usage: $0.11 per 10k operations
├─ Read operations Monthly cost depends on usage: $0.0043 per 10k operations
├─ All other operations Monthly cost depends on usage: $0.0043 per 10k operations
└─ Blob index Monthly cost depends on usage: $0.075 per 10k tags
module.cognitive_services.azurerm_storage_account.cognitive_service_storage[0]
├─ Capacity Monthly cost depends on usage: $0.0392 per GB
├─ Write operations Monthly cost depends on usage: $0.11 per 10k operations
├─ List and create container operations Monthly cost depends on usage: $0.11 per 10k operations
├─ Read operations Monthly cost depends on usage: $0.0043 per 10k operations
├─ All other operations Monthly cost depends on usage: $0.0043 per 10k operations
└─ Blob index Monthly cost depends on usage: $0.075 per 10k tags
module.form_recognizer.azurerm_storage_account.cognitive_service_storage[0]
├─ Capacity Monthly cost depends on usage: $0.0392 per GB
├─ Write operations Monthly cost depends on usage: $0.11 per 10k operations
├─ List and create container operations Monthly cost depends on usage: $0.11 per 10k operations
├─ Read operations Monthly cost depends on usage: $0.0043 per 10k operations
├─ All other operations Monthly cost depends on usage: $0.0043 per 10k operations
└─ Blob index Monthly cost depends on usage: $0.075 per 10k tags
module.machine_learning.azurerm_application_insights.machine_learning_key_insights
└─ Data ingested Monthly cost depends on usage: $2.30 per GB
module.machine_learning.azurerm_container_registry.machine_learning_container_registry
├─ Registry usage (Basic) 30 days $5.00
├─ Storage (over 10GB) Monthly cost depends on usage: $0.10 per GB
└─ Build vCPU Monthly cost depends on usage: $0.0001 per seconds
module.machine_learning.azurerm_storage_account.machine_learning_storage
├─ Capacity Monthly cost depends on usage: $0.0392 per GB
├─ Write operations Monthly cost depends on usage: $0.11 per 10k operations
├─ List and create container operations Monthly cost depends on usage: $0.11 per 10k operations
├─ Read operations Monthly cost depends on usage: $0.0043 per 10k operations
├─ All other operations Monthly cost depends on usage: $0.0043 per 10k operations
└─ Blob index Monthly cost depends on usage: $0.075 per 10k tags
module.video_indexer_media_services.azurerm_storage_account.media_storage
├─ Capacity Monthly cost depends on usage: $0.0392 per GB
├─ Write operations Monthly cost depends on usage: $0.11 per 10k operations
├─ List and create container operations Monthly cost depends on usage: $0.11 per 10k operations
├─ Read operations Monthly cost depends on usage: $0.0043 per 10k operations
├─ All other operations Monthly cost depends on usage: $0.0043 per 10k operations
└─ Blob index Monthly cost depends on usage: $0.075 per 10k tags
OVERALL TOTAL $78.73
──────────────────────────────────
64 cloud resources were detected:
∙ 11 were estimated, all of which include usage-based costs, see https://infracost.io/usage-file
∙ 41 were free:
∙ 14 x azurerm_key_vault_secret
∙ 9 x azurerm_key_vault
∙ 7 x azurerm_monitor_activity_log_alert
∙ 3 x azurerm_storage_container
∙ 1 x azurerm_key_vault_access_policy
∙ 1 x azurerm_log_analytics_storage_insights
∙ 1 x azurerm_resource_group
∙ 1 x azurerm_role_assignment
∙ 1 x azurerm_search_service
∙ 1 x azurerm_storage_account_customer_managed_key
∙ 1 x azurerm_storage_account_network_rules
∙ 1 x azurerm_user_assigned_identity
∙ 12 are not supported yet, see https://infracost.io/requested-resources:
∙ 7 x azurerm_cognitive_account
∙ 1 x azurerm_container_group
∙ 1 x azurerm_machine_learning_compute_cluster
∙ 1 x azurerm_machine_learning_workspace
∙ 1 x azurerm_media_services_account
∙ 1 x azurerm_resource_group_template_deployment
```
## Pre-Commit hooks
Git hook scripts are very helpful for identifying simple issues before pushing any changes. Hooks will run on every commit automatically pointing out issues in the code e.g. trailing whitespace.
To help with the maintenance of these hooks, [pre-commit](https://pre-commit.com/) is used, along with [pre-commit-hooks](https://pre-commit.com/#install).
Please following [these instructions](https://pre-commit.com/#install) to install `pre-commit` locally and ensure that you have run `pre-commit install` to install the hooks for this project.
Additionally, once installed, the hooks can be updated to the latest available version with `pre-commit autoupdate`.
## Documentation Generation
Code formatting and documentation for `variables` and `outputs` is generated using [pre-commit-terraform](https://github.com/antonbabenko/pre-commit-terraform/releases) hooks that in turn uses [terraform-docs](https://github.com/terraform-docs/terraform-docs) that will insert/update documentation. The following markers have been added to the `README.md`:
```
```
## Requirements
| Name | Version |
|------|---------|
| [terraform](#requirement\_terraform) | = 1.5.4 |
| [azurerm](#requirement\_azurerm) | 3.67.0 |
| [random](#requirement\_random) | 3.5.1 |
## Providers
| Name | Version |
|------|---------|
| [azurerm](#provider\_azurerm) | 3.67.0 |
| [random](#provider\_random) | 3.5.1 |
## Modules
| Name | Source | Version |
|------|--------|---------|
| [cognitive\_services](#module\_cognitive\_services) | ./modules/cognitive_services | n/a |
| [cognitive\_services\_container\_language](#module\_cognitive\_services\_container\_language) | ./modules/container_instances | n/a |
| [custom\_question\_answer\_service](#module\_custom\_question\_answer\_service) | ./modules/cognitive_services | n/a |
| [custom\_vision\_service\_prediction](#module\_custom\_vision\_service\_prediction) | ./modules/cognitive_services | n/a |
| [custom\_vision\_service\_training](#module\_custom\_vision\_service\_training) | ./modules/cognitive_services | n/a |
| [form\_recognizer](#module\_form\_recognizer) | ./modules/cognitive_services | n/a |
| [language\_service](#module\_language\_service) | ./modules/cognitive_services | n/a |
| [machine\_learning](#module\_machine\_learning) | ./modules/machine_learning | n/a |
| [open\_ai](#module\_open\_ai) | ./modules/cognitive_services | n/a |
| [video\_indexer\_media\_services](#module\_video\_indexer\_media\_services) | ./modules/video_indexers | n/a |
## Resources
| Name | Type |
|------|------|
| [azurerm_key_vault.tfstate_key_vault](https://registry.terraform.io/providers/hashicorp/azurerm/3.67.0/docs/resources/key_vault) | resource |
| [azurerm_key_vault_access_policy.tfstate_storage](https://registry.terraform.io/providers/hashicorp/azurerm/3.67.0/docs/resources/key_vault_access_policy) | resource |
| [azurerm_key_vault_key.tfstate_key_vault_key](https://registry.terraform.io/providers/hashicorp/azurerm/3.67.0/docs/resources/key_vault_key) | resource |
| [azurerm_log_analytics_storage_insights.tfstate_analytics_storage_insights](https://registry.terraform.io/providers/hashicorp/azurerm/3.67.0/docs/resources/log_analytics_storage_insights) | resource |
| [azurerm_log_analytics_workspace.tfstate_analytics_workspace](https://registry.terraform.io/providers/hashicorp/azurerm/3.67.0/docs/resources/log_analytics_workspace) | resource |
| [azurerm_monitor_diagnostic_setting.tfstate_diagnostic_setting](https://registry.terraform.io/providers/hashicorp/azurerm/3.67.0/docs/resources/monitor_diagnostic_setting) | resource |
| [azurerm_resource_group.environment_rg](https://registry.terraform.io/providers/hashicorp/azurerm/3.67.0/docs/resources/resource_group) | resource |
| [azurerm_search_service.cognitive_search_service](https://registry.terraform.io/providers/hashicorp/azurerm/3.67.0/docs/resources/search_service) | resource |
| [azurerm_search_service.qna_search_service](https://registry.terraform.io/providers/hashicorp/azurerm/3.67.0/docs/resources/search_service) | resource |
| [azurerm_storage_account.tfstate](https://registry.terraform.io/providers/hashicorp/azurerm/3.67.0/docs/resources/storage_account) | resource |
| [azurerm_storage_account_customer_managed_key.tfstate_cmk](https://registry.terraform.io/providers/hashicorp/azurerm/3.67.0/docs/resources/storage_account_customer_managed_key) | resource |
| [azurerm_storage_container.tfstate](https://registry.terraform.io/providers/hashicorp/azurerm/3.67.0/docs/resources/storage_container) | resource |
| [random_string.resource_code](https://registry.terraform.io/providers/hashicorp/random/3.5.1/docs/resources/string) | resource |
| [azurerm_client_config.current](https://registry.terraform.io/providers/hashicorp/azurerm/3.67.0/docs/data-sources/client_config) | data source |
## Inputs
| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| [arm\_client\_id](#input\_arm\_client\_id) | The Client ID which should be used. This can also be sourced
from the ARM\_CLIENT\_ID Environment Variable. | `string` | n/a | yes |
| [arm\_client\_secret](#input\_arm\_client\_secret) | The Client Secret which should be used. This can also be sourced
from the ARM\_CLIENT\_SECRET Environment Variable. | `string` | n/a | yes |
| [arm\_subscription\_id](#input\_arm\_subscription\_id) | The Subscription ID which should be used. This can also be sourced
from the ARM\_SUBSCRIPTION\_ID Environment Variable. | `string` | n/a | yes |
| [arm\_tenant\_id](#input\_arm\_tenant\_id) | The Tenant ID which should be used. This can also be sourced
from the ARM\_TENANT\_ID Environment Variable. | `string` | n/a | yes |
| [cloud\_enviornment](#input\_cloud\_enviornment) | The Cloud Environment which should be used. Possible values are public,
`usgovernment`, `german`, and `china`. Defaults to `public`. This can also be
sourced from the ARM\_ENVIRONMENT Environment Variable. | `string` | `"public"` | no |
| [environment](#input\_environment) | The name of the _environment_ to help identify resources. | `string` | n/a | yes |
| [location](#input\_location) | The Azure Region where the Resource Group should exist.
Changing this forces a new Resource Group to be created. | `string` | `"West Europe"` | no |
| [personal\_ip\_address](#input\_personal\_ip\_address) | Add your client IP address to the networking to allow access. | `string` | n/a | yes |
| [tags](#input\_tags) | Tags to be added to resources created. | `map(string)` | `{}` | no |
## Outputs
| Name | Description |
|------|-------------|
| [cognitive\_service\_endpoint](#output\_cognitive\_service\_endpoint) | The endpoint used to connect to the Cognitive Service
Account. |
| [cognitive\_service\_key\_vault\_name](#output\_cognitive\_service\_key\_vault\_name) | The name of the key vault created to contain cognitive service
secrets. |
| [cognitive\_service\_primary\_access\_key](#output\_cognitive\_service\_primary\_access\_key) | The primary access key which can be used to connect to
the Cognitive Service Account. |
| [cognitive\_service\_secondary\_access\_key](#output\_cognitive\_service\_secondary\_access\_key) | The secondary access key which can be used to connect
to the Cognitive Service Account. |
| [cognitive\_services\_container\_language\_fdqn](#output\_cognitive\_services\_container\_language\_fdqn) | The FDQN to connect to the container instance. |
| [custom\_vision\_service\_prediction\_endpoint](#output\_custom\_vision\_service\_prediction\_endpoint) | The endpoint used to connect to the custom vision
prediction service Account. |
| [custom\_vision\_service\_prediction\_key\_vault\_name](#output\_custom\_vision\_service\_prediction\_key\_vault\_name) | The name of the key vault created to contain custom vision
secrets. |
| [custom\_vision\_service\_prediction\_primary\_access\_key](#output\_custom\_vision\_service\_prediction\_primary\_access\_key) | The primary access key which can be used to connect to
the Cognitive Service Account. |
| [custom\_vision\_service\_prediction\_secondary\_access\_key](#output\_custom\_vision\_service\_prediction\_secondary\_access\_key) | The secondary access key which can be used to connect
to the Cognitive Service Account. |
| [custom\_vision\_service\_training\_endpoint](#output\_custom\_vision\_service\_training\_endpoint) | The endpoint used to connect to the custom vision
training service Account. |
| [custom\_vision\_service\_training\_key\_vault\_name](#output\_custom\_vision\_service\_training\_key\_vault\_name) | The name of the key vault created to contain custom vision
secrets. |
| [custom\_vision\_service\_training\_primary\_access\_key](#output\_custom\_vision\_service\_training\_primary\_access\_key) | The primary access key which can be used to connect to
the Cognitive Service Account. |
| [custom\_vision\_service\_training\_secondary\_access\_key](#output\_custom\_vision\_service\_training\_secondary\_access\_key) | The secondary access key which can be used to connect
to the Cognitive Service Account. |
| [form\_recognizer\_endpoint](#output\_form\_recognizer\_endpoint) | The endpoint used to connect to the form recognizer
Account. |
| [form\_recognizer\_key\_vault\_name](#output\_form\_recognizer\_key\_vault\_name) | The name of the key vault created to contain form recognizer
secrets. |
| [form\_recognizer\_primary\_access\_key](#output\_form\_recognizer\_primary\_access\_key) | The primary access key which can be used to connect to
the Cognitive Service Account. |
| [form\_recognizer\_secondary\_access\_key](#output\_form\_recognizer\_secondary\_access\_key) | The secondary access key which can be used to connect
to the Cognitive Service Account. |
| [language\_service\_endpoint](#output\_language\_service\_endpoint) | The endpoint used to connect to the Language Service
Account. |
| [language\_service\_key\_vault\_name](#output\_language\_service\_key\_vault\_name) | The name of the key vault created to contain language service
secrets. |
| [language\_service\_primary\_access\_key](#output\_language\_service\_primary\_access\_key) | The primary access key which can be used to connect to
the Language Service Account. |
| [language\_service\_secondary\_access\_key](#output\_language\_service\_secondary\_access\_key) | The secondary access key which can be used to connect
to the Language Service Account. |
| [machine\_learning\_discovery\_url](#output\_machine\_learning\_discovery\_url) | The url for the discovery service to identify regional endpoints
for machine learning experimentation services. |
| [machine\_learning\_workspace\_id](#output\_machine\_learning\_workspace\_id) | The immutable id associated with this workspace. |
| [service\_principal\_client\_id](#output\_service\_principal\_client\_id) | The principal being used to apply terraform changes
for this subscription. |
| [tenant\_id](#output\_tenant\_id) | The tenant ID used for this subscription. |
| [tfstate\_resource\_group\_name](#output\_tfstate\_resource\_group\_name) | The name of the resource group created for the
Terraform tfstate. |
| [tfstate\_storage\_account\_key](#output\_tfstate\_storage\_account\_key) | The storage account key created for the
Terraform tfstate. |
| [tfstate\_storage\_account\_name](#output\_tfstate\_storage\_account\_name) | The name of the storage account created for the
Terraform tfstate. |
| [tfstate\_storage\_container\_name](#output\_tfstate\_storage\_container\_name) | The name of the storage container created for the
Terraform tfstate. |