{"id":28212918,"url":"https://github.com/emvaldes/terraform-prototype","last_synced_at":"2026-04-12T13:47:19.848Z","repository":{"id":284512465,"uuid":"953422820","full_name":"emvaldes/terraform-prototype","owner":"emvaldes","description":"DevSecOps - Terraform Prototype","archived":false,"fork":false,"pushed_at":"2025-04-23T06:02:45.000Z","size":147,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-17T19:11:08.011Z","etag":null,"topics":["automation","aws","azure","configuration-management","deployment-as-a-service","devsecops","gcp","github-actions","github-workflows","infrastructure-as-code","json","terraform","zero-config","zero-trust"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/emvaldes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-03-23T10:34:51.000Z","updated_at":"2025-05-13T11:48:29.000Z","dependencies_parsed_at":"2025-04-23T06:34:31.147Z","dependency_job_id":null,"html_url":"https://github.com/emvaldes/terraform-prototype","commit_stats":null,"previous_names":["emvaldes/couchsurfing-terraform","emvaldes/terraform-prototype"],"tags_count":1,"template":true,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emvaldes%2Fterraform-prototype","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emvaldes%2Fterraform-prototype/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emvaldes%2Fterraform-prototype/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emvaldes%2Fterraform-prototype/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/emvaldes","download_url":"https://codeload.github.com/emvaldes/terraform-prototype/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emvaldes%2Fterraform-prototype/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259264920,"owners_count":22831045,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","aws","azure","configuration-management","deployment-as-a-service","devsecops","gcp","github-actions","github-workflows","infrastructure-as-code","json","terraform","zero-config","zero-trust"],"created_at":"2025-05-17T19:10:04.732Z","updated_at":"2026-04-12T13:47:14.786Z","avatar_url":"https://github.com/emvaldes.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Multi-Cloud Terraform Automation Framework\n\n[![Terraform GCP CI/CD Pipeline](https://github.com/emvaldes/terraform-prototype/actions/workflows/terraform.yaml/badge.svg)](https://github.com/emvaldes/terraform-prototype/actions/workflows/terraform.yaml)\n\n## Overview\n\nThis project is a **fully automated, GitHub Actions-driven infrastructure-as-code (IaC) framework** built with **Terraform** for **Google Cloud Platform (GCP)**, designed to be easily extended to support **multi-cloud deployments** (AWS, Azure). It demonstrates deep proficiency in:\n\n- Scalable, reusable, and modular Terraform design patterns\n- Advanced JSON-driven configuration management\n- Secure cloud resource provisioning and network controls\n- CI/CD automation through declarative GitHub Actions workflows\n- Automated infrastructure diagnostics using hardened shell scripts\n- Environment lifecycle management and resource hygiene\n\nThis system automates the provisioning and management of cloud infrastructure on **Google Cloud Platform (GCP)** using **Terraform modules**, **Bash automation scripts**, and a suite of centralized **JSON configuration files**. It is **modular**, **policy-driven**, and designed to be **safe**, **repeatable**, and **fully inspectable**.\n\nThe framework supports **ephemeral, per-environment deployments** (e.g., `dev`, `staging`, `prod`) by enforcing **workspace-specific configurations** and dynamically wiring resources based on a declarative JSON model. All configuration values—such as instance types, scaling policies, service mappings, and RBAC settings—are loaded from structured inputs, making the system environment-agnostic and easy to extend.\n\nSensitive data and secrets are **never hardcoded**; they are **injected externally** via secure channels, ensuring compliance with best practices. The system emphasizes **traceability** through tagging, **visibility** through inspection scripts, and **control** through scoped policy definitions, enabling teams to manage infrastructure confidently and securely across all lifecycle phases.\n\n---\n\n**Objective**: This document provides a unified, high-level explanation of how all project components—modules, configurations, scripts, workflows—interact to form a robust, modular, multi-cloud infrastructure framework. It serves as a reference map for engineers and operators to understand the system architecture, flow of control, and integration dependencies.\n\nThis framework is designed to enable **automated, cloud-agnostic infrastructure delivery** with clean separation of logic, data, and orchestration. It allows seamless expansion, multi-environment testing, and integrates deeply with developer workflows via scripts and CI/CD.\n\n---\n\n## Architecture Layers\n\nThis section describes the layered structure of the automation framework. Each layer encapsulates a role in defining, orchestrating, provisioning, or validating cloud infrastructure on GCP.\n\n---\n\n### 1. **Configuration Layer – Declarative Control Plane**\n\n- **Source:**\n [`project.json`](../project.json),\n [`configs/providers/*.json`](../configs/providers/),\n [`configs/targets/*.json`](../configs/targets/),\n [`configs/policies.json`](../configs/policies.json),\n [`configs/allowed.json`](../configs/allowed.json),\n [`configs/services/gcp/*.json`](../configs/services/gcp/)\n\n- **Purpose:**\n Central source of truth for environment settings, regional constraints, resource counts, tagging, IAM policies, service mappings, RBAC rules, and access control.\n\n- **Function:**\n These JSON files eliminate the need for `.tfvars` or hardcoded values. The Terraform root module dynamically pulls from these files using locals and conditionals, enabling environment-aware behavior without modifying `.tf` code.\n\n---\n\n### 2. **Orchestration Layer – Terraform Root Engine**\n\n- **Source:**\n [`main.tf`](../main.tf), [`variables.tf`](../variables.tf), [`backend.tf`](../backend.tf), [`providers.tf`](../providers.tf), [`outputs.tf`](../outputs.tf)\n\n- **Purpose:**\n Bridges configuration files to reusable modules. Injects environment-specific values into Terraform using dynamically computed locals.\n\n- **Function:**\n Each `terraform apply` run loads all JSON configurations, sets the active workspace, and conditionally provisions modules such as compute, storage, IAM, load balancer, and cloud functions depending on `enabled` flags in `project.json` or `policies.json`.\n\n---\n\n### 3. **Module Layer – Infrastructure Constructs**\n\n- **Source:**\n [`modules/gcp/*`](../modules/gcp/)\n\n- **Modules:**\n - `compute`: VM instances and managed instance groups\n - `networking`: VPCs, subnets, routers, NATs\n - `firewall`: Ingress/egress rules defined in `allowed.json`\n - `load_balancer`: HTTP(S) global load balancer stack\n - `cloud_function`: Serverless workloads packaged via automation\n - `profiles`: IAM identity definitions and service account provisioning\n - `storage`: GCS buckets for Terraform state and other services\n\n- **Function:**\n These are reusable, input-driven Terraform modules designed to emit rich outputs for inspection, diagnostics, and downstream use. Modules read only from parsed `locals`, not `var.*`.\n\n---\n\n### 4. **Automation Layer – Shell \u0026 Python Control Scripts**\n\n- **Source:**\n [`scripts/manage/`](../scripts/manage), [`scripts/configure/`](../scripts/configure), [`scripts/stressload/`](../scripts/stressload), [`scripts/docs/`](../scripts/docs)\n\n- **Purpose:**\n Provides lifecycle tooling, backend management, inspection interfaces, destruction routines, packaging flows, and integration with `terraform output`.\n\n- **Key Scripts Overview:**\n\n| Script | Responsibility |\n|------------------------------|------------------------------------------------------------------|\n| `configure-backend.shell` | Create/download/destroy Terraform state buckets |\n| `configure-profiles.shell` | Create/delete IAM service accounts and credentials |\n| `configure-terraform.shelll` | Initialize Terraform and modules with custom backends |\n| `configure-workspaces.shell` | Constructs all worspaces and activates target workspace |\n| `inspect-services.shell` | Full inspection of load balancer (IP, proxies, backends, health) |\n| `inspect-autoscaling.shell` | Stress testing with `hey`; measures autoscaler behavior |\n| `package-functions.shell` | Zips and deploys Python Cloud Function using outputs |\n| `destroy-services.shell` | Controlled teardown of GCP stack in dependency order |\n| `apache-webserver.shell` | Instance startup script for VM webservers |\n| `docs/*.md` | Human-readable documentation helpers |\n\n---\n\n### 5. **CI/CD Layer – GitHub Actions Integration**\n\n- **Source:**\n [`.github/workflows/terraform.yaml`](../.github/workflows/terraform.yaml)\n\n- **Purpose:**\n Enforces automation pipelines that apply, inspect, validate, and teardown infrastructure with full GitHub Actions support.\n\n- **Function:**\n Environment-specific workflows validate plans, apply infrastructure conditionally, export outputs to artifacts, and invoke post-deploy scripts such as stress tests.\n\n---\n\n## Data Flow \u0026 Evaluation Sequence\n\n```text\n1. GitHub Actions or user CLI triggers Terraform plan/apply\n2. Active workspace selected (e.g., dev, staging, prod)\n3. Configuration JSONs loaded: project, policies, targets, allowed\n4. Terraform root composes locals → passes values to modules\n5. Modules provision infrastructure → emit structured outputs\n6. Outputs saved to outputs.json → consumed by scripts\n7. Scripts handle post-deploy tasks: packaging, deployment, inspection\n```\n\n---\n\n## Terraform Outputs\n\n- Each module defines standard Terraform outputs.\n- `terraform output -json` is saved to `outputs.json` post-deploy.\n- Scripts use this file to extract IPs, names, URLs, IAM identities.\n- A reduced `config.json` is generated for Cloud Function payloads.\n- Secrets are never stored; output files contain metadata only.\n\n---\n\n## Interaction Summary\n\n| Component | Depends On | Consumes |\n|----------------------------|-------------------------------|--------------------------------|\n| `main.tf` | `project.json`, `locals.tf` | Terraform modules |\n| `modules/gcp/*` | locals, conditionals | JSON configs |\n| `compute` | `networking`, `firewall` | Startup scripts, zones |\n| `load_balancer` | `compute` | MIGs, named ports |\n| `cloud_function` | outputs.json, archive.zip | Target IP, IAM account |\n| `package-functions.shell` | outputs.json | Python zip + config.json |\n| `inspect-services.shell` | outputs.json | GCP service inspection |\n| `terraform.yaml` | All layers | Full lifecycle orchestration |\n\n---\n\n## Optional CI/CD Modes\n\n- **Standard Deploy:** Full provision of infrastructure\n- **Test-only Mode:** Create → Run `inspect-services` → Auto-teardown\n- **Stress Test Mode:** Uses `inspect-autoscaling.shell` to verify scale-up/down\n- **Artifact Mode:** Outputs archived via `terraform output` and ZIP bundles\n\n---\n\n## Cloud Function Security\n\n- Cloud Functions use ephemeral service accounts with scoped permissions\n- `config.json` contains only non-sensitive keys required by the function\n- Bucket uploads are temporary unless flagged for persistence\n- All functions can be auto-destroyed after test workflows\n\n---\n\n## Extensibility\n\n| Task | How to Extend |\n|------------------------|---------------------------------------------------------------------------|\n| Add environment | Add new file in `configs/targets/*.json` |\n| Add cloud provider | Update `project.json`, add to `configs/providers/*.json` |\n| Add new service/module | Add new Terraform module, reference from `main.tf`, create JSON interface |\n| Add CI/CD logic | Extend GitHub workflow or shell wrapper |\n\n---\n\n## Key Features\n\n### Environment Isolation\n- Workspaces: `dev`, `staging`, `prod`\n- Per-environment resource naming and counts\n- No shared global resources unless explicitly scoped\n\n### Secure State Management\n- GCS-backed remote state with backend validation\n- Bucket created dynamically per workspace\n- `.tfstate` downloaded on `--delete` operations for audit trace\n\n### JSON-Driven Input Model\n- Eliminates `*.tfvars` and hardcoding\n- Everything driven by environment JSONs\n- Inputs parsed into Terraform locals for full reusability\n\n### Zero Trust Firewall Defaults\n- No public internet by default (`0.0.0.0/0` blocked)\n- DevOps IP, Google Console subnet, and private ranges allowed\n- Controlled entirely via `allowed.json`\n\n### Full Lifecycle Automation\n- Scripts manage backend, inspection, teardown, and packaging\n- Terraform outputs inform scripts dynamically\n- Inspection tooling validates configuration correctness post-deploy\n\n---\n\n### 🔧 Shell Script Automation\n\nThe following scripts are used to automate and standardize infrastructure tasks across GCP environments. Each script supports modular argument parsing, dry-run support, and optional verbose/debug tracing.\n\n#### `./scripts/configure/apache-webserver.shell`\nAutomates the provisioning of compute VM instances configured to simulate HTTP traffic under various network load conditions.\n\n- Dynamically deploys GCP compute instances using pre-defined profiles.\n- Designed for benchmarking, load testing, or validating autoscaling policies.\n- Config-driven: aligns with defined environment parameters and VM sizing.\n- Automatically applies network tagging, firewall rules, and metadata configs.\n\n#### `./scripts/manage/configure-backend.shell`\nHandles the provisioning, inspection, download, and destruction of the Terraform remote state backend hosted on GCS, based on environment-targeted and JSON-driven configurations.\n\n- Supports operations: `--list`, `--create`, `--download`, `--destroy`\n- Applies GCS bucket naming patterns using `\u003cenv\u003e--\u003cpurpose\u003e--\u003cproject_id\u003e`\n- Automatically detects GCP project ID and region to resolve bucket location\n- Downloads .tfstate files per workspace and converts them to .json using terraform show\n- Reads configuration from project.json and configs/policies.json with optional overrides\n- Performs safe and confirmed deletion with pre-download of all state files\n- Creates .local/ directory for inspected/downloaded state snapshots\n- Fully compatible with multi-workspace Terraform environments\n\n#### `./scripts/manage/configure-profiles.shell`\nManages the creation and deletion of GCP IAM Service Accounts and credentials defined in a centralized profiles.json configuration.\n\n- Supports operations: `--create`, `--delete`\n- Automates service account creation, key generation, and credential cleanup\n- Reads account metadata (name, description, filename) from `./configs/profiles.json`\n- Derives full email and key paths based on active GCP project context\n- Safely handles overwrites, prompting confirmation before credential deletion\n- Outputs consistent and readable information about each account\n- Ensures service account presence before performing key actions\n- Requires gcloud and jq for GCP and JSON handling\n- Fully idempotent and safe to re-run with dry-run and verbose modes\n\n#### `./scripts/manage/configure-terraform.shell`\nBootstraps and orchestrates Terraform initialization, validation, and plan execution across multiple provider targets and workspaces using JSON-defined project context.\n\n- Dynamically sets Terraform working directory and provider configuration\n- Reads `project.json`, `configs/providers/*.json`, and `configs/targets/*.json`\n- Supports operations: `--init`, `--plan`, `--show`, `--validate`, `--refresh`\n- Handles workspace creation and selection automatically\n- Applies external variables like `TF_VAR_*` and backend settings per workspace\n- Provides clear formatted output logs for each phase\n- Supports GCP authentication and regional targeting using environment variables\n- Uses safe default fallback paths and Terraform automation flags\n\n#### `./scripts/manage/configure-workspaces.shell`\nManages Terraform workspace lifecycle actions such as creation, listing, selection, and deletion in alignment with JSON-based configuration sets.\n\n- Supports operations: `--list`, `--create`, `--select`, `--delete`\n- Retrieves workspace names from `configs/targets/*.json` or `project.json`\n- Can perform bulk creation or deletion across all configured workspaces\n- Automates workspace initialization with minimal Terraform CLI assumptions\n- Provides descriptive logging of active, new, or missing workspaces\n- Detects backend availability and validates initialization context\n- Ensures idempotent operations with fallback safety checks\n\n#### `./scripts/manage/inspect-autoscaling.shell`\nPerforms a configurable stress test against a GCP HTTP Load Balancer and inspects autoscaling behavior of the associated Managed Instance Group (MIG).\n\n- Executes multi-phase load testing (burst, sustained, cooldown, recovery) using hey\n- Reads stress level and autoscaling profiles from centralized JSON configuration files\n- Resolves MIG name, region, and target from Terraform outputs\n- Captures real-time instance counts and lists active VMs per test phase\n- Supports dynamic adjustment of thread concurrency, duration, and request intervals\n- Validates existence of target and policy configs before execution\n- Optional logic to reset MIG size (disabled when autoscaler is active)\n- Outputs clear logs for each phase, instance state, and scaling response\n\n#### `./scripts/manage/inspect-services.shell`\nInspects GCP infrastructure health, IAM configurations, and load balancer components in real-time using live gcloud, curl, and jq calls.\n\n- Validates and traces HTTP Load Balancer components: forwarding rules, proxies, URL maps, and backend services\n- Checks health checks, instance group membership, and autoscaler configuration\n- Evaluates IAM bindings, roles, keys, and custom role usage across Terraform-managed identities\n- Compares Terraform IAM outputs with current GCP policies to detect drift\n- Scans for expiring IAM keys and misconfigured roles or unbound identities\n- Fetches recent activity logs for IAM identities and autoscaling events\n- Supports environment detection via Terraform or override variables\n- Outputs all results as formatted JSON with sectioned summaries and key exports\n\n#### `./scripts/manage/package-functions.shell`\nBuilds and prepares a Python-based Cloud Function package, extracting dynamic configuration from Terraform outputs and uploading artifacts to a target GCS bucket.\n\n- Parses Terraform output and policies.json to construct environment-aware config files\n- Dynamically injects autoscaling and stressload settings into config.json\n- Packages source files and dependencies into a .zip archive for Cloud Function deployment\n- Verifies required project ID, function name, region, and service account details\n- Uploads the final archive to a pre-defined GCS bucket used by Terraform deployment\n- Optionally triggers Terraform -target apply to complete archive upload without full plan\n- Supports inspection of GCS object metadata, bucket size, and content listing after upload\n\n---\n\n#### `./scripts/others/destroy-services.shell`\nTears down all GCP resources related to a web application deployment in a controlled and dependency-aware sequence.\u003cbr /\u003e\nIt handles network, compute, and routing teardown with minimal interaction.\n\n- Deletes HTTP Load Balancer components: forwarding rules, proxies, URL maps, backend services, and health checks\n- Removes compute resources including MIGs, autoscalers, instance templates, and firewall rules\n- Destroys networking components: NAT configs, routers, subnets, VPCs, and VPC peering ranges\n- Requires project ID and region via --project-id and --region flags or fallback to gcloud/env\n- Validates required inputs before execution to prevent accidental deletions\n- All operations are fully non-interactive (--quiet) for automation use\n\n#### `./scripts/others/gcloud-presets.shell`\nBootstraps and enforces consistent gcloud CLI environment settings across all systems.\n\n- Applies default region, zone, and active account/project.\n- Loads secure credentials and activates service accounts from managed config.\n- Helps normalize local, CI/CD, or ephemeral environments before infrastructure interaction.\n\n### Shared Features\n\n- **Standardized Execution Flags**\n All scripts support modular CLI flags for consistent usage:\n `--create`, `--list`, `--delete`, etc.\n\n- **Optional Modes for Safe Execution**\n Enable enhanced control with:\n `--dry-run`, `--verbose`, `--debug`\n\n- **Config-Driven Behavior**\n Automatically loads input from:\n `project.json`, `policies.json`, or user-defined paths\n\n- **Safe and Predictable Operations**\n Scripts are non-destructive by default, requiring explicit confirmation for deletions or changes\n\n- **Readable and Traceable Output**\n Error messages are clean, informative, and optimized for debugging and automation logs\n\n---\n\n- **CI/CD GitHub Workflow Integration:**\n - Full Terraform lifecycle steps: `validate`, `plan`, `apply`, `destroy`\n - Dynamic switching of workspaces during workflow execution\n - Diagnostic logging (`gcloud`, `terraform`, `jq`) included\n\n- **Traceable Deployments \u0026 State Artifacts:**\n - Logs, install traces, and state backups uploaded via `actions/upload-artifact`\n - Useful for postmortem analysis and rollback capability\n\n- **Multi-Cloud Ready Architecture:**\n - Modular decomposition of compute, networking, and firewall layers\n - Supports future `aws` and `azure` modules via interface-compatible design\n\n---\n\n## Infrastructure Components\n\n#### 1. **Compute Layer**\n\n**Files:** `modules/gcp/compute/`, `scripts/configure/apache-webserver.shell`, `configs/services/gcp/compute_resources.json`\n- Provisions GCE instances using custom instance templates and managed instance groups (MIGs)\n- Instance types, replica counts, and metadata are sourced dynamically via `dev.json`, `policies.json`, and `compute_resources.json`\n- Tagged using centralized `tagging.json` for traceability (e.g., `ssh-access`, `http-server`)\n- Fully compatible with autoscaler modules and integrated health checks\n- Web server initialization handled by a custom Apache setup script\n\n#### 2. **Networking \u0026 Firewall**\n\n**Files:** `modules/gcp/networking/`, `modules/gcp/firewall/`, `configs/allowed.json`, `tagging.json`\n- Defines isolated, per-environment VPCs with regional subnets\n- Establishes DNS resolution, IP ranges, NAT routing, and reserved PSA ranges\n- Ingress/egress rules enforced using allowlists from `allowed.json` (e.g., `allow-ssh-restricted`, `allow-http-https`)\n- Tags applied conditionally to firewall rules (e.g., `ssh-access`) for role-based enforcement\n- Network components are automatically wired to Load Balancer backends and MIGs\n\n#### 3. **Load Balancing**\n\n**Files:** `modules/gcp/load_balancer/`, `scripts/manage/inspect-services.shell`, `configs/services/gcp/load_balancer.json`, `http_forwarding.json`, `web_backend.json`\n- Implements a Global HTTP(S) Load Balancer stack with end-to-end Terraform automation\n - Global forwarding rules\n - Target HTTP proxies\n - URL maps (with default service bindings)\n - Regional backend service with health checks and autoscaling support\n- `inspect-services.shell` provides live introspection of all components, including forwarding IPs, MIG health, proxy mappings, and instance group state\n\n#### 4. **Routing \u0026 NAT Configuration**\n\n**Files:** `modules/gcp/networking.router.tf`, `router.tf`, `configs/services/gcp/networking.json`\n- Provisions GCP Cloud Routers and regional NAT gateways for internet access without public IPs\n- Enables outbound traffic for instance patching, telemetry, and package installation\n- NAT gateway configuration is driven by Terraform variables and policies for full reproducibility\n- VPC peering and PSA allocation handled via automation scripts and `vpc-peerings` inspection\n\n#### 5. **State Management**\n\n**Files:** `backend.tf`, `scripts/manage/configure-backend.shell`, `project.json`, `policies.json`\n- Manages remote Terraform state in GCS using a dynamic, workspace-prefixed bucket naming convention\n- `configure-backend.shell` performs bucket creation, validation, state download, and `.tfstate` to `.json` transformation\n- State files can be inspected under `.local/` during teardown or troubleshooting\n- Supports RBAC-controlled provisioning based on `policies.json`, only creating backend resources when enabled\n- Safe to re-run, fully idempotent, and supports dry-run/verbose modes\n\n#### 6. **Storage \u0026 Bucket Provisioning**\n\n**Files:** `modules/gcp/storage/`, `backend.tf`, `configs/policies.json`, `tagging.json`\n- Creates GCS buckets conditionally based on environment-level policies defined in `policies.json`\n- Bucket naming follows the convention: `\u003cenv\u003e--\u003cbucket-name\u003e--\u003cproject_id\u003e`\n- Supports optional RBAC-based IAM binding if enabled via `storage.bucket.rbac`\n- Retention policies, versioning, and access controls are defined through centralized JSON and applied through Terraform\n- Tagging standards from `tagging.json` are applied to ensure service-level traceability (`storage`, `service-accounts`)\n- Used by both backend state and Cloud Function deployments (via upload targets and artifact storage)\n\n#### 7. **Terraform Setup \u0026 Workspace Automation**\n\n**Files:** `scripts/manage/configure-terraform.shell`, `scripts/manage/configure-workspaces.shell`, `project.json`, `configs/providers/*.json`, `configs/targets/*.json`\n\n- `configure-terraform.shell` prepares Terraform for execution by setting up CLI paths, validating tool presence, selecting the correct provider, and exporting relevant environment variables dynamically\n- `configure-workspaces.shell` manages Terraform workspaces across environments (`dev`, `staging`, `prod`), supporting creation, selection, deletion, and inspection\n- Both scripts are driven by JSON inputs and detect settings like project ID, region, provider type, and environment name automatically\n- Ensures consistency and pre-validation before running any Terraform operations\n- Designed for CI/CD pipelines and repeatable local runs\n- Fully idempotent, supports dry-run/verbose/debug flags, and avoids redundant actions when workspace or CLI context is already valid\n\n---\n\n## 1. Configuration: The Source of Truth\n\nBefore any cloud resource is created, this system **loads configuration from JSON files**.\nThese configurations live in the `configs/` folder:\n\n- **`project.json`**\n Defines which environment (like `dev`, `prod`, or `staging`) is being targeted, and where to find provider-specific configuration files.\n\n- **`configs/providers/gcp.json`**\n Contains details about the GCP provider (such as regions, credentials, or defaults).\n\n- **`configs/targets/dev.json`**\n Contains all environment-specific overrides—such as compute instance sizes or stress testing levels for the development environment.\n\n- **`configs/policies.json`**\n Controls which features are turned on or off. For example:\n - Should storage buckets be created?\n - Should access controls (RBAC) be applied?\n\n- **`configs/allowed.json`**\n Lists allowed firewall rules, like what IP ranges can SSH or connect via HTTP.\n\nThese JSON files are used instead of hardcoding variables, making the system easy to reuse across environments.\n\n---\n\n## 2. Terraform: Infrastructure as Code\n\nTerraform is the core tool that **turns configuration into actual GCP resources**. It reads `.tf` files (like `compute.tf`, `networking.tf`, etc.) and provisions real infrastructure in GCP.\nTerraform modules are organized like this:\n\n```\nmodules/gcp/\n├── compute/\n├── storage/\n├── firewall/\n├── networking/\n├── load_balancer/\n├── cloud_function/\n├── profiles/\n```\n\nEach module handles one part of the cloud environment.\n\n- Before Terraform runs, the `scripts/manage/configure-terraform.shell` script ensures the CLI, environment variables, and provider paths are set up correctly. It validates tools, exports config values, and makes the environment Terraform-ready.\n\n---\n\n## 3. Modules and What They Do\n\n### Compute (`modules/gcp/compute`)\n- Provisions **virtual machines** (VMs) in GCP.\n- Uses **managed instance groups (MIGs)** so Google can scale up or down based on demand.\n- All configuration (how many VMs, what type, etc.) is pulled from `dev.json`, `compute_resources.json`, and `policies.json`.\n- VM startup scripts are managed by `scripts/configure/apache-webserver.shell`.\n\n---\n\n### Networking \u0026 Firewall (`modules/gcp/networking`, `modules/gcp/firewall`)\n- Creates an **isolated Virtual Private Cloud (VPC)** per environment.\n- Defines **subnets**, or IP address ranges per region.\n- Adds **firewall rules** like:\n - Who can SSH in\n - Whether HTTP/HTTPS is allowed\n- These rules are configured using `allowed.json` and applied automatically.\n\n---\n\n### Load Balancer (`modules/gcp/load_balancer`)\n- Sets up a **Global HTTP(S) Load Balancer**.\n- Incoming internet traffic is routed through:\n - A **global forwarding rule** (receives traffic)\n - A **target HTTP proxy** (sends it to the correct app)\n - A **URL map** (defines backend behavior)\n - A **backend service** (connects to your VM instances)\n- A health check is continuously monitoring the backend service.\n\nYou can inspect the full setup live using the script:\n`scripts/manage/inspect-services.shell`\n\n---\n\n### Routing \u0026 NAT (`modules/gcp/networking.router.tf`)\n- Creates **Cloud Routers** and **NAT gateways**.\n- This allows your internal VMs (which don’t have public IPs) to:\n - Download software\n - Install updates\n - Communicate externally without exposing themselves to the internet\n\n---\n\n### Storage (`modules/gcp/storage`)\n- Creates **Google Cloud Storage (GCS) buckets** for:\n - Terraform state files (the record of your infrastructure)\n - Uploading archives used by Cloud Functions\n- Bucket behavior is controlled by `policies.json` and `tagging.json`\n- If RBAC (role-based access control) is enabled, IAM permissions are applied automatically\n\n---\n\n### Cloud Functions (`modules/gcp/cloud_function`)\n- Deploys **serverless functions** that run in response to HTTP requests.\n- These functions:\n - Read configuration from a generated `config.json`\n - Simulate traffic (for autoscaling tests)\n - Run on demand\n- All code and packages are zipped and uploaded using:\n `scripts/manage/package-functions.shell`\n\n---\n\n### Profiles / Service Accounts (`modules/gcp/profiles`)\n- Creates and manages **IAM service accounts** based on `profiles.json`\n- Automatically:\n - Generates keys\n - Binds roles\n - Deletes old accounts safely\n- Service accounts are grouped using a `group` key so you can apply access controls to whole teams at once.\n\n---\n\n## 4. Terraform State Management\n\nTerraform needs to **keep track of what it built**, and it does that via a `.tfstate` file.\n\nThis state file is:\n- Stored in a **GCS bucket**\n- Managed by `backend.tf`\n- Automatically created or validated by the script:\n `scripts/manage/configure-backend.shell`\n- The `scripts/manage/configure-workspaces.shell` script automates creation, selection, and inspection of Terraform workspaces based on JSON config. It ensures each environment maps cleanly to an isolated state scope.\n\nIf you ever destroy the environment, this script can download the `.tfstate` file and convert it to readable `.json` for inspection (in the `.local/` directory).\n\n---\n\n## 5. Inspections, Health Checks \u0026 Automation Scripts\n\nSeveral custom **Bash scripts** help you validate and operate the environment consistently:\n\n- `inspect-services.shell`: Shows the entire load balancer setup and health status\n- `inspect-autoscaling.shell`: Runs a stress test and watches how autoscaling responds\n- `configure-backend.shell`: Manages Terraform’s backend GCS bucket\n- `configure-profiles.shell`: Creates and deletes IAM accounts and credentials\n- `configure-terraform.shell`: Bootstraps Terraform execution across providers and workspaces\n- `configure-workspaces.shell`: Manages Terraform workspace lifecycle actions\n- `package-functions.shell`: Packages and uploads Python functions to GCS\n\nEach script:\n- Accepts flags like `--dry-run`, `--debug`, `--verbose`\n- Reads shared config like `project.json`, `policies.json`, and provider-specific JSON files\n- Is idempotent (safe to re-run)\n- Designed for automated use in CI/CD pipelines and local operations\n\n---\n\n## 6. Execution Flow (What Happens When)\n\nHere’s what happens when you set up a new environment from scratch:\n\n1. You define or select the target environment in `project.json` (e.g., `dev`)\n\n2. JSON configuration is loaded dynamically:\n - `dev.json` → environment-specific inputs\n - `policies.json` → which features are enabled\n - `profiles.json` → which credentials should exist\n\n3. Terraform reads modules (`*.tf`) and builds:\n - VPC and subnets\n - VM instances in MIGs\n - Cloud Router and NAT\n - Global Load Balancer\n - Cloud Functions (if enabled)\n\n4. Terraform backend state is stored in a GCS bucket named like:\n `\u003cterraform-workspace\u003e--\u003cterraform-bucket-name\u003e--\u003cgcp-project-id\u003e`\n\n5. Post-deployment:\n - You can run inspection scripts to validate health and performance\n - Cloud Functions can simulate traffic\n - IAM accounts and credentials are auto-created\n\n6. When tearing down:\n - Scripts handle state backup, safe deletion, and introspection\n - IAM keys and buckets are cleaned up conditionally\n\n---\n\n## Summary\n\nThis infrastructure system is:\n- **Policy-driven**: JSON files decide what gets created\n- **Modular**: Each feature (compute, networking, IAM, etc.) is isolated\n- **Repeatable and Safe**: Every script has dry-run mode and safety checks\n- **Extensible**: Supports multi-cloud in the future via config abstraction\n- **Auditable**: All components can be inspected, described, and reported in JSON\n\nThis system empowers teams to manage cloud environments **confidently**, **transparently**, and **at scale**—even if you’re new to cloud or Terraform.\n\n---\n\n#### Download \u0026 install (latest version):\n\n```bash\n\u003e target_package=\"google-cloud-cli-457.0.0-linux-x86_64.tar.gz\" ;\n\u003e curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/${target_package} ;\n\u003e tar -xf google-cloud-cli-*.tar.gz ;\n\u003e ./google-cloud-sdk/install.sh ;\n```\n\n---\n\n#### Backup Local Configuration\n\n```bash\n\u003e GCP_HOME=\"${HOME}/.gcp\" ;\n\u003e GCP_BACKUPS=\"${GCP_HOME}/backups\" ;\n\n\u003e mkdir -p ${GCP_BACKUPS}/ ;\n\u003e cp -prv ${HOME}/.config/gcloud ${GCP_BACKUPS}/ ;\n\n\u003e gcloud config configurations \\\n list --format=json \u003e ${HOME}/.gcp/backups/configurations.json ;\n\u003e gcloud config configurations \\\n describe default \u003e ${HOME}/.gcp/backups/default-configs.yaml ;\n\u003e gcloud iam service-accounts \\\n list --format=json \u003e ${HOME}/.gcp/backups/service-accounts.json ;\n```\n\n#### Purging GCP Project (default/current)\n\n```bash\n\u003e gcloud projects delete $( gcloud config get-value project --quiet )\nYour project will be deleted.\n\nDo you want to continue (Y/n)? Y\n\nDeleted [https://cloudresourcemanager.googleapis.com/v1/projects/\u003cgcp-project-name\u003e].\n\nYou can undo this operation for a limited period by running the command below.\n $ gcloud projects undelete $( gcloud config get-value project --quiet )\n\nSee https://cloud.google.com/resource-manager/docs/creating-managing-projects\nfor information on shutting down projects.\n```\n\n---\n\n#### Initializing GCP Configurations\n\n##### Follow the prompts to:\n\n- Authenticate with your Google account\n- Choose a GCP project or create a new one\n- Set a default region and zone\n\n```bash\n\u003e gcloud init --console-only ;\n```\n\n##### This will allow Google Cloud SDK to:\n\n- See, edit, configure, and delete your Google Cloud data and see the email address for your Google Account.\n- View and sign in to your Google Cloud SQL instances\n- View and manage your Google Compute Engine resources\n- View and manage your applications deployed on Google App Engine\n\n##### Copy the URL, open in browser manually, paste the code back in the terminal.\n\n```bash\n\u003e gcloud init --console-only ;\n\nWelcome! This command will take you through the configuration of gcloud.\n\nYour current configuration has been set to: [default]\n\nYou can skip diagnostics next time by using the following flag:\n gcloud init --skip-diagnostics\n\nNetwork diagnostic detects and fixes local network connection issues.\nChecking network connection...done.\nReachability Check passed.\nNetwork diagnostic passed (1/1 checks passed).\n\nYou must sign in to continue. Would you like to sign in (Y/n)? Y\n\nGo to the following link in your browser, and complete the sign-in prompts:\n\n https://accounts.google.com/o/oauth2/auth?response_type=code\n \u0026client_id=\u003cclient-id\u003e.apps.googleusercontent.com\n \u0026redirect_uri=https://sdk.cloud.google.com/authcode.html\n \u0026scope=openid+\n https://www.googleapis.com/auth/userinfo.email+\n https://www.googleapis.com/auth/cloud-platform+\n https://www.googleapis.com/auth/appengine.admin+\n https://www.googleapis.com/auth/sqlservice.login+\n https://www.googleapis.com/auth/compute+\n https://www.googleapis.com/auth/accounts.reauth\n \u0026state=\u003caccount-state\u003e\n \u0026prompt=consent\n \u0026token_usage=remote\n \u0026access_type=offline\n \u0026code_challenge=\u003ccode-challenge-query\u003e\n \u0026code_challenge_method=S256\n\nOnce finished, enter the verification code provided in your browser: \u003ccode-challenge-response\u003e\nYou are signed in as: [\u003cgcp-account-email\u003e].\n\nThis account has no projects.\n\nWould you like to create one? (Y/n)? Y\n\nEnter a Project ID. Note that a Project ID CANNOT be changed later.\nProject IDs must be 6-30 characters (lowercase ASCII, digits, or\nhyphens) in length and start with a lowercase letter. \u003cgcp-project-name\u003e\nWaiting for [operations/create_project.global.\u003cgcp-account-number\u003e] to finish...done.\nYour current project has been set to: [\u003cgcp-project-name\u003e].\n\nNot setting default zone/region (this feature makes it easier to use\n[gcloud compute] by setting an appropriate default value for the\n--zone and --region flag).\nSee https://cloud.google.com/compute/docs/gcloud-compute section on how to set\ndefault compute region and zone manually. If you would like [gcloud init] to be\nable to do this for you the next time you run it, make sure the\nCompute Engine API is enabled for your project on the\nhttps://console.developers.google.com/apis page.\n\nCreated a default .boto configuration file at [${HOME}/.boto]. See this file and\n[https://cloud.google.com/storage/docs/gsutil/commands/config] for more\ninformation about configuring Google Cloud Storage.\nThe Google Cloud CLI is configured and ready to use!\n\n* Commands that require authentication will use \u003cgcp-account-email\u003e by default\n* Commands will reference project `\u003cgcp-project-name\u003e` by default\nRun `gcloud help config` to learn how to change individual settings\n\nThis gcloud configuration is called [default].\nYou can create additional configurations if you work with multiple accounts and/or projects.\nRun `gcloud topic configurations` to learn more.\n\nSome things to try next:\n\n* Run `gcloud --help` to see the Cloud Platform services you can interact with.\n And run `gcloud help COMMAND` to get help on any gcloud command.\n* Run `gcloud topic --help` to learn about advanced features of the CLI like arg files and output formatting\n* Run `gcloud cheat-sheet` to see a roster of go-to `gcloud` commands.\n```\n\n```bash\n\u003e gcloud services enable iam.googleapis.com compute.googleapis.com ;\n\nERROR: (gcloud.services.enable) FAILED_PRECONDITION:\n Billing account for project '\u003cgcp-account-number\u003e' is not found.\n Billing must be enabled for activation of service(s)\n 'compute.googleapis.com,compute.googleapis.com,compute.googleapis.com' to proceed.\nHelp Token: \u003cgcp-service-token\u003e\n\n- '@type': type.googleapis.com/google.rpc.PreconditionFailure\n violations:\n - subject: ?error_code=390001\n \u0026project=\u003cgcp-account-number\u003e\n \u0026services=compute.googleapis.com\n \u0026services=compute.googleapis.com\n \u0026services=compute.googleapis.com\n type: googleapis.com/billing-enabled\n\n- '@type': type.googleapis.com/google.rpc.ErrorInfo\n domain: serviceusage.googleapis.com/billing-enabled\n metadata:\n project: '\u003cgcp-account-number\u003e'\n services: compute.googleapis.com,compute.googleapis.com,compute.googleapis.com\n reason: UREQ_PROJECT_BILLING_NOT_FOUND\n```\n\n```bash\n\u003e gcloud beta billing accounts list ;\n\nYou do not currently have this command group installed. Using it\nrequires the installation of components: [beta]\n\n\nYour current Google Cloud CLI version is: 517.0.0\nInstalling components from version: 517.0.0\n\n┌─────────────────────────────────────────────┐\n│ These components will be installed. │\n├──────────────────────┬────────────┬─────────┤\n│ Name │ Version │ Size │\n├──────────────────────┼────────────┼─────────┤\n│ gcloud Beta Commands │ 2025.03.29 │ \u003c 1 MiB │\n└──────────────────────┴────────────┴─────────┘\n\nFor the latest full release notes, please visit:\n https://cloud.google.com/sdk/release_notes\n\nOnce started, canceling this operation may leave your SDK installation in an inconsistent state.\n\nDo you want to continue (Y/n)? Y\n\nPerforming in place update...\n\n╔════════════════════════════════════════════════════════════╗\n╠═ Downloading: gcloud Beta Commands ═╣\n╠════════════════════════════════════════════════════════════╣\n╠═ Installing: gcloud Beta Commands ═╣\n╚════════════════════════════════════════════════════════════╝\n\nPerforming post processing steps...done.\n\nUpdate done!\n\nRestarting command:\n $ gcloud beta billing accounts list\n\nAPI [cloudbilling.googleapis.com] not enabled on project [\u003cgcp-project-name\u003e].\nWould you like to enable and retry (this will take a few minutes)? (y/N)? y\n\nEnabling service [cloudbilling.googleapis.com] on project [\u003cgcp-project-name\u003e]...\n\nERROR: (gcloud.beta.billing.accounts.list) PERMISSION_DENIED:\nService Usage API has not been used in project \u003cgcp-project-name\u003e before or it is disabled.\nEnable it by visiting https://console.developers.google.com/apis/api/serviceusage.googleapis.com/overview?project=\u003cgcp-project-name\u003e then retry.\nIf you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.\nThis command is authenticated as \u003cgcp-account-email\u003e which is the active account specified by the [core/account] property.\n\nService Usage API has not been used in project \u003cgcp-project-name\u003e before or it is disabled.\nEnable it by visiting https://console.developers.google.com/apis/api/serviceusage.googleapis.com/overview?project=\u003cgcp-project-name\u003e then retry.\nIf you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.\n\nGoogle developers console API activation\nhttps://console.developers.google.com/apis/api/serviceusage.googleapis.com/overview?project=\u003cgcp-project-name\u003e\n\n- '@type': type.googleapis.com/google.rpc.ErrorInfo\n domain: googleapis.com\n metadata:\n activationUrl: https://console.developers.google.com/apis/api/serviceusage.googleapis.com/overview?project=\u003cgcp-project-name\u003e\n consumer: projects/\u003cgcp-project-name\u003e\n containerInfo: \u003cgcp-project-name\u003e\n service: serviceusage.googleapis.com\n serviceTitle: Service Usage API\n reason: SERVICE_DISABLED\n```\n\n```bash\n\u003e gcloud beta billing accounts list --format=json ;\n\n[\n {\n \"currencyCode\": \"USD\",\n \"displayName\": \"My Billing Account\",\n \"masterBillingAccount\": \"\",\n \"name\": \"billingAccounts/\u003cgcp-billing-account\u003e\",\n \"open\": true,\n \"parent\": \"\"\n }\n]\n```\n\n```bash\n\u003e gcloud beta billing projects \\\n link $( gcloud config get-value project --quiet ) \\\n --billing-account \u003cgcp-billing-account\u003e ;\n\n billingAccountName: billingAccounts/\u003cgcp-billing-account\u003e\n billingEnabled: true\n name: projects/\u003cgcp-project-name\u003e/billingInfo\n projectId: \u003cgcp-project-name\u003e\n```\n\n```bash\n\u003e gcloud beta billing projects describe $( gcloud config get-value project) ;\n\nAPI [cloudbilling.googleapis.com] not enabled on project [\u003cgcp-project-number\u003e].\nWould you like to enable and retry (this will take a few minutes)? (y/N)? y\n\nEnabling service [cloudbilling.googleapis.com] on project [\u003cgcp-project-number\u003e]...\nOperation \"operations/acat.p2-\u003cgcp-project-number\u003e-\u003coperation-unique-identifier\u003e\" finished successfully.\nbillingAccountName: billingAccounts/\u003cgcp-billing-account\u003e\nbillingEnabled: true\nname: projects/\u003cgcp-project-name\u003e/billingInfo\nprojectId: \u003cgcp-project-name\u003e\n```\n\n```bash\n\u003e gcloud services enable \\\n iam.googleapis.com compute.googleapis.com ;\n\n Operation \"operations/acf.p2-\u003cgcp-account-number\u003e-\u003cservice-serial-number\u003e\" finished successfully.\n```\n\n```bash\n\u003e gcloud iam service-accounts \\\n create gcp-cli-admin \\\n --display-name \"GCP CLI Admin\" ;\n\n Created service account [gcp-cli-admin].\n```\n\n```bash\n\u003e gcloud projects add-iam-policy-binding \u003cgcp-project-name\u003e \\\n --member=\"serviceAccount:gcp-cli-admin@\u003cgcp-project-name\u003e.iam.gserviceaccount.com\" \\\n --role=\"roles/owner\" ;\n\n Updated IAM policy for project [\u003cgcp-project-name\u003e].\n\n bindings:\n - members:\n - serviceAccount:service-\u003cgcp-account-number\u003e@compute-system.iam.gserviceaccount.com\n\n role: roles/compute.serviceAgent\n - members:\n - serviceAccount:\u003cgcp-account-number\u003e-compute@developer.gserviceaccount.com\n - serviceAccount:\u003cgcp-account-number\u003e@cloudservices.gserviceaccount.com\n\n role: roles/editor\n - members:\n - serviceAccount:gcp-cli-admin@\u003cgcp-project-name\u003e.iam.gserviceaccount.com\n - user:\u003cgcp-account-email\u003e\n\n role: roles/owner\n\n etag: BwYyRXUa4Ps=\n version: 1\n```\n\n```bash\n\u003e gcloud iam service-accounts keys \\\n create ${HOME}/.gcp/credentials.json \\\n --iam-account gcp-cli-admin@\u003cgcp-project-name\u003e.iam.gserviceaccount.com ;\n\n created key [\u003cgcp-private-keyid\u003e] of type [json] as [${HOME}/.gcp/credentials.json]\n for [gcp-cli-admin@\u003cgcp-project-name\u003e.iam.gserviceaccount.com]\n```\n\n```bash\n\u003e ls -al ${HOME}/.gcp/credentials.json ;\n -rw------- 1 \u003cuser-id\u003e staff 2380 Jan 1 00:00 ${HOME}/.gcp/credentials.json\n```\n\n```bash\n\u003e bat ${HOME}/.gcp/credentials.json ;\n │ File: ${HOME}/.gcp/credentials.json\n 1 │ {\n 2 │ \"type\": \"service_account\",\n 3 │ \"project_id\": \"\u003cgcp-project-name\u003e\",\n 4 │ \"private_key_id\": \"\u003cgcp-private-keyid\u003e\",\n 5 │ \"private_key\": \"-----BEGIN PRIVATE KEY-----\\nMIIEvQIBAD...mUziEzFz5s=\\n-----END PRIVATE KEY-----\\n\",\n 6 │ \"client_email\": \"gcp-cli-admin@\u003cgcp-project-name\u003e.iam.gserviceaccount.com\",\n 7 │ \"client_id\": \"\u003cgcp-client-id\u003e\",\n 8 │ \"auth_uri\": \"https://accounts.google.com/o/oauth2/auth\",\n 9 │ \"token_uri\": \"https://oauth2.googleapis.com/token\",\n 10 │ \"auth_provider_x509_cert_url\": \"https://www.googleapis.com/oauth2/v1/certs\",\n 11 │ \"client_x509_cert_url\": \"https://www.googleapis.com/robot/v1/metadata/x509/gcp-cli-admin%40\u003cgcp-project-name\u003e.iam.gserviceaccount.com\",\n 12 │ \"universe_domain\": \"googleapis.com\"\n 13 │ }\n```\n\n```bash\n\u003e gcloud auth activate-service-account --key-file=${HOME}/.gcp/credentials.json ;\n\n Activated service account credentials for: [gcp-cli-admin@\u003cgcp-project-name\u003e.iam.gserviceaccount.com]\n```\n\n```bash\n# Extract abstract region from target config\n\u003e abstract_region=$(\n jq -r '.region' \"${targets_config_path}\"\n ) ;\n\n# Resolve actual cloud region from project config\n\u003e target_region=$(\n jq -r --arg key \"${abstract_region}\" '.regions[$key]' \"${project_config}\"\n ) ;\n\n\u003e gcloud config set compute/region ${target_region} ;\n\n WARNING: Property validation for compute/region was skipped.\n Updated property [compute/region].\n\n\u003e gcloud config get-value compute/region ;\n \u003cgcp-compute-region\u003e\n```\n\n```bash\n\u003e target_zone=$(\n gcloud compute zones list \\\n --filter=\"region:(${target_region})\" \\\n --limit=1 \\\n --format=\"value(name)\"\n ) ;\n\n\u003e gcloud config set compute/zone ${target_zone} ;\n WARNING: Property validation for compute/zone was skipped.\n Updated property [compute/zone].\n\n\u003e gcloud config get-value compute/zone ;\n \u003cgpc-compute-zone\u003e\n```\n\n```bash\n\u003e gcloud iam service-accounts keys \\\n create ${HOME}/.gcp/credentials.json \\\n --iam-account $(\n gcloud auth list --filter=status:ACTIVE --format=\"value(account)\"\n ) ;\n # created key [\u003cgcp-created-keyid\u003e] of type [json]\n # as [${HOME}/.gcp/credentials.json] for [gcp-cli-admin@\u003cgcp-project-name\u003e.iam.gserviceaccount.com]\n```\n\n---\n\n```bash\n\u003e ./scripts/manage/configure-profiles.shell ;\n\n Usage: configure-profiles.shell [OPTIONS]\n\n Options:\n -c, --create Create the service account and its associated credentials key (file) if they do not exist\n -d, --delete Delete the service account and its associated credentials key (file)\n\n --help Show this help message and exit\n --dry-run Print actions without executing them\n --verbose Enable verbose output\n --debug Enable debug mode with trace output\n\n Examples:\n configure-profiles.shell --create ;\n configure-profiles.shell --delete ;\n\n\u003e ./scripts/manage/configure-profiles.shell --create ;\n\nAccount: dev-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com\nDescription: Development environment service account\nCredentials: ~/.config/gcloud/accounts/dev-credentials.json\n```\n\n```json\nCreated service account [dev-account].\n\n{\n \"displayName\": \"Development environment service account\",\n \"email\": \"dev-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com\",\n \"etag\": \"MDEwMjE5MjA=\",\n \"name\": \"projects/\u003cgcp-project-name\u003e/serviceAccounts/dev-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com\",\n \"oauth2ClientId\": \"\u003coauth2-client-id\u003e\",\n \"projectId\": \"\u003cgcp-project-name\u003e\",\n \"uniqueId\": \"\u003coauth2-client-id\u003e\"\n}\n```\n\n```bash\ncreated key [gcp-private-keyid] of type [json]\nas [~/.config/gcloud/accounts/dev-account--credentials.json]\nfor [dev-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com]\n-rw------- 1 \u003cuser-id\u003e staff 2376 Jan 1 00:00 ~/.config/gcloud/accounts/dev-account--credentials.json\n\nAccount: devops-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com\nDescription: DevOps service account\nCredentials: ~/.config/gcloud/accounts/devops-account--credentials.json\n```\n\n```json\nCreated service account [devops-account].\n\n{\n \"displayName\": \"DevOps service account\",\n \"email\": \"devops-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com\",\n \"etag\": \"MDEwMjE5MjA=\",\n \"name\": \"projects/\u003cgcp-project-name\u003e/serviceAccounts/devops-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com\",\n \"oauth2ClientId\": \"116957529764710061292\",\n \"projectId\": \"\u003cgcp-project-name\u003e\",\n \"uniqueId\": \"116957529764710061292\"\n}\n```\n\n```bash\ncreated key [gcp-private-keyid] of type [json]\nas [~/.config/gcloud/accounts/devops-account--credentials.json]\nfor [devops-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com]\n-rw------- 1 \u003cuser-id\u003e staff 2382 Jan 1 00:00 ~/.config/gcloud/accounts/devops-account--credentials.json\n\nAccount: prod-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com\nDescription: Production environment service account\nCredentials: ~/.config/gcloud/accounts/prod-account--credentials.json\n```\n\n```json\nCreated service account [prod-account].\n\n{\n \"displayName\": \"Production environment service account\",\n \"email\": \"prod-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com\",\n \"etag\": \"MDEwMjE5MjA=\",\n \"name\": \"projects/\u003cgcp-project-name\u003e/serviceAccounts/prod-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com\",\n \"oauth2ClientId\": \"117373193121071847420\",\n \"projectId\": \"\u003cgcp-project-name\u003e\",\n \"uniqueId\": \"117373193121071847420\"\n}\n```\n\n```bash\ncreated key [gcp-private-keyid] of type [json]\nas [~/.config/gcloud/accounts/prod-account--credentials.json]\nfor [prod-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com]\n-rw------- 1 \u003cuser-id\u003e staff 2382 Jan 1 00:00 ~/.config/gcloud/accounts/prod-account--credentials.json\n\nAccount: staging-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com\nDescription: Staging environment service account\nCredentials: ~/.config/gcloud/accounts/staging-account--credentials.json\n```\n\n```json\nCreated service account [staging-account].\n\n{\n \"displayName\": \"Staging environment service account\",\n \"email\": \"staging-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com\",\n \"etag\": \"MDEwMjE5MjA=\",\n \"name\": \"projects/\u003cgcp-project-name\u003e/serviceAccounts/staging-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com\",\n \"oauth2ClientId\": \"\u003coauth2-client-id\u003e\",\n \"projectId\": \"\u003cgcp-project-name\u003e\",\n \"uniqueId\": \"\u003coauth2-client-id\u003e\"\n}\n```\n\n```bash\ncreated key [\u003cgcp-private-keyid\u003e] of type [json]\nas [~/.config/gcloud/accounts/staging-account--credentials.json]\nfor [staging-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com]\n-rw------- 1 \u003cuser-id\u003e staff 2382 Jan 1 00:00 ~/.config/gcloud/accounts/staging-account--credentials.json\n```\n\n---\n\n```bash\n\u003e ls -al ~/.config/gcloud/accounts/ ;\n\ntotal 32\n-rw------- 1 \u003cuser-id\u003e staff 2376 Jan 1 00:00 dev-credentials.json\n-rw------- 1 \u003cuser-id\u003e staff 2382 Jan 1 00:00 devops-credentials.json\n-rw------- 1 \u003cuser-id\u003e staff 2378 Jan 1 00:00 prod-credentials.json\n-rw------- 1 \u003cuser-id\u003e staff 2384 Jan 1 00:00 staging-credentials.json\n```\n\n---\n\n```bash\n\u003e ./scripts/manage/configure-profiles.shell --delete ;\n\ndeleted service account [dev-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com]\nDeleting Credential: ~/.config/gcloud/accounts/dev-account--credentials.json\n\ndeleted service account [devops-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com]\nDeleting Credential: ~/.config/gcloud/accounts/devops-account--credentials.json\n\ndeleted service account [prod-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com]\nDeleting Credential: ~/.config/gcloud/accounts/prod-account--credentials.json\n\ndeleted service account [staging-account@\u003cgcp-project-name\u003e.iam.gserviceaccount.com]\nDeleting Credential: ~/.config/gcloud/accounts/staging-account--credentials.json\n```\n\n---\n\n```bash\nexport GOOGLE_APPLICATION_CREDENTIALS=\"${HOME}/.gcp/credentials.json\";\nexport GCP_CREDENTIALS=\"$( cat ${GOOGLE_APPLICATION_CREDENTIALS} | base64 )\";\n\nexport GCP_PROJECT_ID=$(\n jq -r .project_id \"${GOOGLE_APPLICATION_CREDENTIALS}\"\n) ; ## echo -e \"Project ID: ${GCP_PROJECT_ID}\" ;\n\nexport TF_VAR_gcp_project_id=\"${GCP_PROJECT_ID}\";\n```\n\n##### Note: This will become active in the /locals.tf file:\n\n```terraform\n# File: /locals.tf\n# Version: 0.1.0\n\n# Description: Contains all local values used across modules\n\nlocals {\n\n # Load dispatcher\n project = jsondecode(file(\"${path.root}/project.json\"))\n\n # Active provider ID\n provider_id = local.project.defaults.provider\n\n # Provider config (cloud-specific)\n provider_default = jsondecode(file(\"${path.root}/configs/providers/${local.provider_id}.json\"))\n\n # Final provider config, overriding project_id if passed via env\n provider = merge(\n local.provider_default,\n {\n project_id = var.gcp_project_id\n }\n )\n\n # Use the overridden project_id\n project_id = local.provider.project_id\n\n # Workspace/target config (env-specific)\n workspace = jsondecode(file(\"${path.root}/configs/targets/${terraform.workspace}.json\"))\n\n # Shared policies\n policies = jsondecode(file(\"${path.root}/configs/policies.json\"))\n\n # Profiles (Accounts, Groups, Credentials, RBAC \u0026 access roles)\n profiles = jsondecode(file(\"${path.root}/configs/profiles.json\"))\n\n # Abstracted region/type from provider map\n region = lookup(local.provider.regions, local.workspace.region)\n type = lookup(local.provider.types, local.workspace.type)\n\n # Load tagging map\n tagging = jsondecode(file(\"${path.root}/configs/tagging.json\"))\n\n # Allowed Access (White listing)\n allowed = jsondecode(file(\"${path.root}/configs/allowed.json\"))\n\n # GCP service naming map\n services = {\n for service in local.provider.services :\n service =\u003e jsondecode(\n file(\"${path.root}/configs/services/${local.provider.provider}/${service}.json\")\n )\n }\n\n # Compute Resources (inject tags at creation time)\n compute_resources = merge(\n try(local.services.compute_resources, {}),\n {\n tags = [\n for tag in try(local.tagging.providers[local.provider_id].compute.tags, []) :\n tag.fixed ? tag.value : \"${terraform.workspace}--${tag.value}\"\n ]\n }\n )\n\n # Firewall Rules\n firewall_rules = merge(\n try(local.services.firewall_rules, {}),\n {\n tags = {\n allow_ssh = [\n for tag in try(local.tagging.providers[local.provider_id].firewall.allow_ssh.tags, []) :\n tag.fixed ? tag.value : \"${terraform.workspace}--${tag.value}\"\n ]\n allow_ssh_iap = [\n for tag in try(local.tagging.providers[local.provider_id].firewall.allow_ssh_iap.tags, []) :\n tag.fixed ? tag.value : \"${terraform.workspace}--${tag.value}\"\n ]\n }\n }\n )\n\n # GCP - Cloud Function (Stress-Load Testing)\n cloud_function = merge(\n try(local.services.cloud_function, {}),\n {\n tags = [\n for tag in try(local.tagging.providers[local.provider_id].cloud_function.tags, []) :\n tag.fixed ? tag.value : \"${terraform.workspace}--${tag.value}\"\n ]\n }\n )\n\n # Load-Balancer\n load_balancer = merge(\n try(local.services.load_balancer, {}),\n {\n tags = [\n for tag in try(local.tagging.providers[local.provider_id].load_balancer.tags, []) :\n tag.fixed ? tag.value : \"${terraform.workspace}--${tag.value}\"\n ]\n }\n )\n\n # Networking\n networking = merge(\n try(local.services.networking, {}),\n {\n tags = [\n for tag in try(local.tagging.providers[local.provider_id].networking.tags, []) :\n tag.fixed ? tag.value : \"${terraform.workspace}--${tag.value}\"\n ]\n }\n )\n\n # Profiles (Accounts, Groups, Credentials, RBAC \u0026 access roles)\n accounts = merge(\n try(local.profiles.accounts, {}),\n {\n tags = [\n for tag in try(local.tagging.providers[local.provider_id].accounts.tags, []) :\n tag.fixed ? tag.value : \"${terraform.workspace}--${tag.value}\"\n ]\n }\n )\n\n # Group Access Credentials (mapping): e.g.: local.group_credentials[\"devops\"]\n group_credentials = {\n for group_key in distinct([\n for profile_key, profile in local.profiles.credentials :\n try(profile.group, null)\n if try(profile.group, null) != null\n ]) :\n group_key =\u003e {\n for profile_key, profile in local.profiles.credentials :\n profile_key =\u003e profile\n if try(profile.group, null) == group_key\n }\n }\n\n # Compute final backend config\n backend = merge(\n local.policies.storage.bucket,\n {\n name = (\n local.policies.storage.bucket.rbac ?\n \"${terraform.workspace}--${local.policies.storage.bucket.name}--${local.project_id}\" :\n local.policies.storage.bucket.name\n )\n }\n )\n\n # Autoscaling\n autoscaler = try(\n local.policies.autoscaling.profiles[local.workspace.policies.autoscaling],\n {}\n )\n\n}\n```\n\n---\n\n```bash\n\u003e gcloud auth list ;\n Credentialed Accounts\nACTIVE ACCOUNT\n \u003cgcp-account-email\u003e@gmail.com\n* gcp-cli-admin@\u003cgcp-project-name\u003e.iam.gserviceaccount.com\n\nTo set the active account, run:\n $ gcloud config set account `ACCOUNT`\n\n```\n\n---\n\n```bash\n\u003e gcloud services \\\n enable cloudresourcemanager.googleapis.com \\\n --format=json ;\n Operation \"operations/acat.p2-\u003cgcp-project-number\u003e-\u003coperation-unique-identifier\u003e\" finished successfully.\n []\n```\n\n```json\n\u003e gcloud services \\\n list --enabled \\\n --filter=\"config.name=cloudresourcemanager.googleapis.com\" \\\n --format=json ;\n\n[\n {\n \"config\": {\n \"authentication\": {},\n \"documentation\": {\n \"summary\": \"Creates, reads, and updates metadata for Google Cloud Platform resource containers.\"\n },\n \"monitoring\": {},\n \"name\": \"cloudresourcemanager.googleapis.com\",\n \"quota\": {},\n \"title\": \"Cloud Resource Manager API\",\n \"usage\": {\n \"requirements\": [\n \"serviceusage.googleapis.com/tos/cloud\"\n ]\n }\n },\n \"name\": \"projects/\u003cgcp-project-number\u003e/services/cloudresourcemanager.googleapis.com\",\n \"parent\": \"projects/\u003cgcp-project-number\u003e\",\n \"state\": \"ENABLED\"\n }\n]\n```\n\n---\n\n```bash\n\u003e gcloud services enable logging.googleapis.com --format=json ;\n Operation \"operations/acat.p2-\u003cgcp-project-number\u003e-60cd72ae-54c4-4fe2-ac6b-3409e3b08058\" finished successfully.\n []\n```\n\n```json\n\u003e gcloud services \\\n list --enabled \\\n --filter=\"config.name=logging.googleapis.com\" \\\n --format=json ;\n\n[\n {\n \"config\": {\n \"authentication\": {},\n \"documentation\": {\n \"summary\": \"Writes log entries and manages your Cloud Logging configuration.\"\n },\n \"monitoredResources\": [\n {\n \"description\": \"A cloud logging specialization target schema of cloud.ChargedProject.\",\n \"displayName\": \"Cloud logging target\",\n \"labels\": [\n {\n \"description\": \"The monitored resource container. Could be project, workspace, etc.\",\n \"key\": \"resource_container\"\n },\n {\n \"description\": \"The service-specific notion of location.\",\n \"key\": \"location\"\n },\n {\n \"description\": \"The name of the API service with which the data is associated (e.g.,'logging.googleapis.com').\",\n \"key\": \"service\"\n }\n ],\n \"launchStage\": \"ALPHA\",\n \"type\": \"logging.googleapis.com/ChargedProject\"\n }\n ],\n \"monitoring\": {\n \"consumerDestinations\": [\n {\n \"metrics\": [\n \"logging.googleapis.com/billing/ingested_bytes\",\n \"logging.googleapis.com/billing/stored_bytes\"\n ],\n \"monitoredResource\": \"logging.googleapis.com/ChargedProject\"\n }\n ]\n },\n \"name\": \"logging.googleapis.com\",\n \"quota\": {},\n \"title\": \"Cloud Logging API\",\n \"usage\": {\n \"requirements\": [\n \"serviceusage.googleapis.com/tos/cloud\"\n ]\n }\n },\n \"name\": \"projects/\u003cgcp-project-number\u003e/services/logging.googleapis.com\",\n \"parent\": \"projects/\u003cgcp-project-number\u003e\",\n \"state\": \"ENABLED\"\n }\n]\n```\n\n---\n\n##### Grant your gcp-cli-admin service account the necessary permissions on it:\n\n```bash\n\u003e gcp_cli_admin=\"gcp-cli-admin@$( gcloud config get-value project --quiet )\" ;\n\u003e gsutil iam ch serviceAccount:${gcp_cli_admin}.iam.gserviceaccount.com:roles/storage.admin \\\n gs://${TERRAFORM_BACKEND_BUCKET} ;\n```\n\n---\n\n##### Terraform State Backend setup\n\n**Warning**: Terraform cannot create its own backend storage (GCS bucket) because its a pre-existing requirement to run. The bucket must:\n\n- Already exist\n- Be accessible by the service account\n\n```bash\nPOLICIES_FILE=\"./configs/policies.json\" ;\nexport TERRAFORM_BACKEND_BUCKET=$( jq -r '.storage.bucket.name' \"${POLICIES_FILE}\" ) ;\n```\n\n**Note**: You have the option to use a native request or use the ./scripts/manage/configure-backend.shell script to manage this process.\n\n```bash\n\u003e gsutil mb -p \u003cgcp-project-name\u003e \\\n -l us-west2 \\\n -b on gs://${TERRAFORM_BACKEND_BUCKET} ;\n```\n\nor\n\n```bash\n\u003e ./scripts/manage/configure-backend.shell ;\n\n Usage: configure-backend.shell [OPTIONS]\n\n Options:\n -l, --list List the current bucket status and configuration\n -c, --create Create the bucket if it does not exist\n -w, --download Download and convert remote Terraform state to local JSON\n -d, --destroy Destroy the bucket and optionally backup state locally\n\n -j, --project=PATH Path to the project configuration file (default: ./project.json)\n -t, --target=NAME Target workspace/environment name\n -p, --policies=PATH Path to the policies configuration file (default: ./configs/policies.json)\n -n, --name=NAME Name of the GCS bucket to manage\n -x, --prefix=NAME State prefix path within the bucket\n\n --help Show this help message and exit\n --dry-run Print actions without executing them\n --verbose Enable verbose output\n --debug Enable debug mode with trace output\n\n Examples:\n configure-backend.shell --create --target=testing\n configure-backend.shell --list --target=prod --name=bucket-name\n configure-backend.shell --download --dry-run --verbose\n\n\u003e ./scripts/manage/configure-backend.shell --create ;\n\nCreating gs://\u003cterraform-workspace\u003e--\u003cterraform-bucket-name\u003e--\u003cgcp-project-name\u003e/...\nBucket gs://\u003cterraform-workspace\u003e--\u003cterraform-bucket-name\u003e--\u003cgcp-project-name\u003e was created and confirmed!\n```\n\n```json\nBucket gs://\u003cterraform-workspace\u003e--\u003cterraform-bucket-name\u003e--\u003cgcp-project-name\u003e exists.\nBucket configuration:\n\n{\n \"creation_time\": \"2025-04-11T01:35:21+0000\",\n \"default_storage_class\": \"STANDARD\",\n \"generation\": 1744335321634073103,\n \"location\": \"US\",\n \"location_type\": \"multi-region\",\n \"metageneration\": 1,\n \"name\": \"\u003cterraform-workspace\u003e--\u003cterraform-bucket-name\u003e--\u003cgcp-project-name\u003e\",\n \"public_access_prevention\": \"inherited\",\n \"rpo\": \"DEFAULT\",\n \"soft_delete_policy\": {\n \"effectiveTime\": \"2025-04-11T01:35:21.888000+00:00\",\n \"retentionDurationSeconds\": \"604800\"\n },\n \"storage_url\": \"gs://\u003cterraform-workspace\u003e--\u003cterraform-bucket-name\u003e--\u003cgcp-project-name\u003e/\",\n \"uniform_bucket_level_access\": true,\n \"update_time\": \"2025-04-11T01:35:21+0000\"\n}\n\nDone.\n```\n\n```bash\n\u003e ./scripts/manage/configure-terraform.shell ;\n\n Usage: configure-terraform.shell [OPTIONS]\n\n Options:\n -i, --init Initialize Terraform Engine \u0026 Modules\n -p, --policies=PATH Path to the policies configuration file (default: ./configs/policies.json)\n -j, --project=PATH Path to the project configuration file (default: ./project.json)\n -w, --workspace Terraform target workspace: dev, staging, prod\n\n --help Show this help message and exit\n --dry-run Print actions without executing them\n --verbose Enable verbose output\n --debug Enable debug mode with trace output\n\n Examples:\n configure-terraform.shell --init --workspace='dev' ;\n\n\u003e ./scripts/manage/configure-terraform.shell --init ;\n\nTerraform Bucket name: \u003cterraform-workspace\u003e--\u003cterraform-bucket-name\u003e--\u003cgcp-project-name\u003e\nTerraform Bucket prefix: terraform/state\n```\n\n```hcl\nInitializing the backend...\nInitializing modules...\nInitializing provider plugins...\n- Reusing previous version of hashicorp/google from the dependency lock file\n- Using previously-installed hashicorp/google v6.29.0\n\nTerraform has been successfully initialized!\n\nYou may now begin working with Terraform. Try running \"terraform plan\" to see\nany changes that are required for your infrastructure. All Terraform commands\nshould now work.\n\nIf you ever set or change modules or backend configuration for Terraform,\nrerun this command to reinitialize your working directory. If you forget, other\ncommands will detect it and remind you to do so if necessary.\n```\n\n```json\n\u003e jq -r . .terraform/terraform.tfstate;\n\n{\n \"version\": 3,\n \"terraform_version\": \"1.11.4\",\n \"backend\": {\n \"type\": \"gcs\",\n \"config\": {\n \"access_token\": null,\n \"bucket\": \"\u003cterraform-workspace\u003e--\u003cterraform-bucket-name\u003e--\u003cgcp-project-name\u003e\",\n \"credentials\": null,\n \"encryption_key\": null,\n \"impersonate_service_account\": null,\n \"impersonate_service_account_delegates\": null,\n \"kms_encryption_key\": null,\n \"prefix\": \"terraform/state\",\n \"storage_custom_endpoint\": null\n },\n \"hash\": 1543364987\n }\n}\n\ngs://\u003cterraform-workspace\u003e--\u003cterraform-bucket-name\u003e--\u003cgcp-project-name\u003e/terraform/state/default.tfstate\n\nDone.\n```\n\n```bash\n\u003e gsutil ls gs://${terraform_bucket_name}/terraform/state/;\n gs://\u003cterraform-workspace\u003e--\u003cterraform-bucket-name\u003e--\u003cgcp-project-name\u003e/terraform/state/default.tfstate\n gs://\u003cterraform-workspace\u003e--\u003cterraform-bucket-name\u003e--\u003cgcp-project-name\u003e/terraform/state/dev.tfstate\n gs://\u003cterraform-workspace\u003e--\u003cterraform-bucket-name\u003e--\u003cgcp-project-name\u003e/terraform/state/prod.tfstate\n gs://\u003cterraform-workspace\u003e--\u003cterraform-bucket-name\u003e--\u003cgcp-project-name\u003e/terraform/state/staging.tfstate\n```\n\n```bash\n\u003e ./scripts/manage/configure-workspaces.shell ;\n\n Usage: configure-workspaces.shell [OPTIONS]\n\n Options:\n -c, --create Create Terraform workspaces: dev, staging, prod, ...\n -j, --project=PATH Path to the project configuration file (default: ./project.json)\n -w, --workspace Terraform target workspace: dev, staging, prod, ...\n\n --help Show this help message and exit\n --dry-run Print actions without executing them\n --verbose Enable verbose output\n --debug Enable debug mode with trace output\n\n Examples:\n configure-workspaces.shell --create --workspace='dev' ;\n\n\u003e ./scripts/manage/configure-workspaces.shell --create ;\n```\n\n```hcl\nCreating workspace: dev\nCreated and switched to workspace \"dev\"!\n\nYou're now on a new, empty workspace. Workspaces isolate their state,\nso if you run \"terraform plan\" Terraform will not see any existing state\nfor this configuration.\nCreating workspace: prod\nCreated and switched to workspace \"prod\"!\n\nYou're now on a new, empty workspace. Workspaces isolate their state,\nso if you run \"terraform plan\" Terraform will not see any existing state\nfor this configuration.\nCreating workspace: staging\nCreated and switched to workspace \"staging\"!\n\nYou're now on a new, empty workspace. Workspaces isolate their state,\nso if you run \"terraform plan\" Terraform will not see any existing state\nfor this configuration.\nSwitched to workspace \"dev\".\n\nCurrent Terraform Workspace: dev\n\nDone.\n```\n\n```terraform\n\u003e terraform validate ;\n Success! The configuration is valid.\n```\n\n---\n\n## Configuration Files\n\n### `/backend.tf`\n\n```hcl\n# File: /backend.tf\n# Version: 0.1.0\n\nterraform {\n backend \"gcs\" {}\n}\n```\n\n### `/providers.tf`\n\n```hcl\n# File: /providers.tf\n# Version: 0.1.0\n\nterraform {\n required_version = \"\u003e= 1.3.0\"\n\n ## Optional version pinning — uncomment if stability is required\n # required_providers {\n # google = {\n # source = \"hashicorp/google\"\n # version = \"\u003e= 6.29.0\"\n # }\n # }\n\n}\n\nprovider \"google\" {\n project = local.project_id\n region = local.region\n}\n```\n\n### `/project.json`\nDefines the project's defaults settings, configurations paths and scripting resources for all automed services.\n\n**Ownership**: Application Management\n\n```json\n{\n \"defaults\": {\n \"provider\": \"gcp\",\n \"target\": \"dev\"\n },\n \"configs\": {\n \"providers\": {\n \"path\": \"./configs/providers\",\n \"sets\": {\n \"aws\": \"aws.json\",\n \"azure\": \"azure.json\",\n \"gcp\": \"gcp.json\"\n }\n },\n \"targets\": {\n \"path\": \"./configs/targets\",\n \"sets\": {\n \"dev\": \"dev.json\",\n \"prod\": \"prod.json\",\n \"staging\": \"staging.json\"\n }\n }\n },\n \"scripts\": {\n \"configure\": {\n \"apache_webserver\": {\n \"path\": \"./scripts/configure\",\n \"script\": \"apache-webserver.shell\"\n }\n },\n \"manage\": {\n \"configure_backend\": {\n \"path\": \"./scripts/manage\",\n \"script\": \"configure-backend.shell\"\n },\n \"configure_profiles\": {\n \"path\": \"./scripts/manage\",\n \"script\": \"configure-profiles.shell\"\n },\n \"configure_terraform\": {\n \"path\": \"./scripts/manage\",\n \"script\": \"configure-terraform.shell\"\n },\n \"configure_workspaces\": {\n \"path\": \"./scripts/manage\",\n \"script\": \"configure-workspaces.shell\"\n },\n \"inspect_autoscaling\": {\n \"path\": \"./scripts/manage\",\n \"script\": \"inspect-autoscaling.shell\"\n },\n \"inspect_services\": {\n \"path\": \"./scripts/manage\",\n \"script\": \"inspect-services.shell\"\n },\n \"package_functions\": {\n \"path\": \"./scripts/manage\",\n \"script\": \"package-functions.shell\"\n }\n },\n \"stressload\": {\n \"webservers\": {\n \"path\": \"./scripts/stressload\",\n \"script\": \"stressload-webservers.zip\"\n }\n }\n }\n}\n```\n\n---\n\n### `/configs/providers/gcp.json`\nDefines: provider, project, regions, compute types, services (cloud function, compute, firewall,load balancing, networking, and other infrastructure services.\n\n**Ownership**: Infrastructure Administrators\n\n```json\nConfig-File: /configs/providers/gcp.json\n\n{\n \"provider\": \"gcp\",\n \"project_id\": \"\",\n \"credentials\": \"\",\n \"regions\": {\n \"west\": \"us-west2\",\n \"central\": \"us-central2\",\n \"east\": \"us-east2\"\n },\n \"types\": {\n \"micro\": \"e2-micro\",\n \"medium\": \"e2-medium\",\n \"standard\": \"n1-standard-1\"\n },\n \"services\": [\n \"cloud_function\",\n \"compute_resources\",\n \"firewall_rules\",\n \"health_check\",\n \"http_forwarding\",\n \"load_balancer\",\n \"networking\",\n \"web_autoscaling\",\n \"web_backend\"\n ]\n}\n```\n\n```json\nConfig-File: /configs/services/gcp/cloud_function.json\n\n{\n \"enable\": true,\n \"auto_deploy\": false,\n \"name\": \"webapp-stress-tester\",\n \"description\": \"Stub Cloud Function for stress testing framework\",\n \"entry_point\": \"main\",\n \"runtime\": \"python311\",\n \"memory\": \"256M\",\n \"timeout\": 60,\n \"bucket_name\": \"cloud-function-bucket\",\n \"archive_path\": \"./packages\",\n \"archive_name\": \"stressload-webservers.zip\",\n \"force_destroy\": true,\n \"env\": {\n \"TARGET_URL\": \"\"\n },\n \"event_type\": \"google.cloud.functions.v2.eventTypes.EVENT_TRIGGERED\",\n \"pubsub_topic\": null,\n \"invoker_role\": \"roles/cloudfunctions.invoker\",\n \"invoker_member\": \"allUsers\",\n \"tags\": []\n}\n```\n\n```json\nConfig-File: /configs/services/gcp/compute_resources.json\n\n{\n \"instance_template_name_prefix\": \"web-server-template--\",\n \"instance_group_name\": \"web-servers-group\",\n \"base_instance_name\": \"web-server\",\n \"source_image\": \"ubuntu-os-cloud/ubuntu-2004-lts\",\n \"startup_script_path\": \"./scripts/configure/apache-webserver.shell\",\n \"health_check\": {\n \"name\": \"http-health-check\",\n \"interval\": 5,\n \"timeout\": 5,\n \"port\": 80\n },\n \"tags\": []\n}\n```\n\n```json\nConfig-File: /configs/services/gcp/firewall_rules.json\n\n{\n \"allow_ssh\": {\n \"name\": \"allow-ssh-restricted\",\n \"protocol\": \"tcp\",\n \"ports\": [\n \"22\"\n ],\n \"target_tags\": [\n \"ssh-access\"\n ]\n },\n \"allow_ssh_iap\": {\n \"name\": \"allow-ssh-iap\",\n \"protocol\": \"tcp\",\n \"ports\": [\n \"22\"\n ],\n \"target_tags\": [\n \"ssh-access\"\n ]\n },\n \"allow_http_https\": {\n \"name\": \"allow-http-https\",\n \"protocol\": \"tcp\",\n \"ports\": [\n \"80\",\n \"443\"\n ]\n },\n \"public_http_ranges\": [\n \"0.0.0.0/0\"\n ],\n \"tags\": []\n}\n```\n\n```json\nConfig-File: /configs/services/gcp/health_check.json\n\n{\n \"name\": \"http-health-check\",\n \"tags\": []\n}\n```\n\n```json\nConfig-File: /configs/services/gcp/http_forwarding.json\n\n{\n \"name\": \"http-forwarding-rule\",\n \"tags\": []\n}\n```\n\n```json\nConfig-File: /configs/services/gcp/load_balancer.json\n\n{\n \"http_forwarding\": {\n \"name\": \"http-forwarding-rule\",\n \"port_range\": \"80\",\n \"scheme\": \"EXTERNAL\"\n },\n \"http_proxy\": {\n \"name\": \"web-http-proxy\"\n },\n \"url_map\": {\n \"name\": \"web-url-map\"\n },\n \"web_backend\": {\n \"name\": \"web-backend-service\",\n \"protocol\": \"HTTP\",\n \"timeout\": 30\n },\n \"health_check\": {\n \"name\": \"http-health-check\",\n \"interval\": 5,\n \"timeout\": 5,\n \"port\": 80\n },\n \"tags\": []\n}\n```\n\n```json\nConfig-File: /configs/services/gcp/networking.json\n\n{\n \"vpc_network_name\": \"webapp-vpc\",\n \"subnet_name\": \"webapp-subnet\",\n \"subnet_cidr\": \"10.100.0.0/24\",\n \"psa_range_name\": \"cloudsql-psa-range\",\n \"psa_range_prefix_length\": 16,\n \"nat\": {\n \"router_name\": \"webapp-router\",\n \"config_name\": \"webapp-nat-config\",\n \"nat_logging_filter\": \"ERRORS_ONLY\",\n \"enable_nat_logging\": true,\n \"timeouts\": {\n \"tcp_established_sec\": 1200,\n \"tcp_transitory_sec\": 30,\n \"udp_idle_sec\": 30,\n \"icmp_idle_sec\": 30\n }\n },\n \"management\": {\n \"enable\": false,\n \"vpc_name\": \"mgmt-vpc\",\n \"subnet_name\": \"mgmt-subnet\",\n \"subnet_cidr\": \"10.90.0.0/24\",\n \"private_ip_google_access\": true\n },\n \"tags\": []\n}\n```\n\n```json\nConfig-File: /configs/services/gcp/web_autoscaling.json\n\n{\n \"name\": \"web-autoscaling\",\n \"tags\": []\n}\n```\n\n```json\nConfig-File: /configs/services/gcp/web_backend.json\n\n{\n \"name\": \"web-backend-service\",\n \"tags\": []\n}\n```\n\n---\n\n### `/configs/targets/dev.json`\nThis is an abstraction mechanism to allow multipe environments to define implementation agnostic configurations.\n\n**Ownership**: DevOps Engineers\n\n```json\n{\n \"id\": \"dev\",\n \"name\": \"development\",\n \"description\": \"Development environment\",\n \"region\": \"west\",\n \"type\": \"micro\",\n \"policies\": {\n \"autoscaling\": \"basic\",\n \"stressload\": \"low\"\n }\n}\n```\n\n---\n\n### `/configs/policies.json`\nThis is an abstraction mechanism to define services configurations.\n\n**Ownership**: DevSecOps/DevNetOps Engieners\n\n```json\n{\n \"autoscaling\": {\n \"profiles\": {\n \"basic\": {\n \"min\": 1,\n \"max\": 2,\n \"threshold\": 0.6,\n \"cooldown\": 60\n },\n \"medium\": {\n \"min\": 2,\n \"max\": 4,\n \"threshold\": 0.65,\n \"cooldown\": 90\n },\n \"advanced\": {\n \"min\": 3,\n \"max\": 6,\n \"threshold\": 0.7,\n \"cooldown\": 120\n }\n },\n \"logging\": {\n \"log_file\": \"./logs/autoscaling.log\",\n \"log_format\": \"%(asctime)s - %(levelname)s - %(message)s\"\n }\n },\n \"profiles\": {\n \"service\": {\n \"read_only\": {\n \"name\": \"ro--service-account\",\n \"caption\": \"Service Account (Read Only)\"\n }\n },\n \"cloud_function\": {\n \"read_only\": {\n \"name\": \"ro--cloud-function\",\n \"caption\": \"Cloud Function SA (Stress Test)\"\n }\n }\n },\n \"storage\": {\n \"bucket\": {\n \"name\": \"terraform-prototype\",\n \"prefix\": \"terraform/state\",\n \"rbac\": true,\n \"lifecycle\": {\n \"rules\": [\n {\n \"action\": {\n \"type\": \"Delete\"\n },\n \"condition\": {\n \"age\": 90,\n \"matchesStorageClass\": [\n \"STANDARD\"\n ]\n }\n }\n ]\n }\n }\n },\n \"stressload\": {\n \"levels\": {\n \"low\": {\n \"duration\": 60,\n \"threads\": 250,\n \"interval\": 0.04,\n \"requests\": 10000\n },\n \"medium\": {\n \"duration\": 60,\n \"threads\": 500,\n \"interval\": 0.02,\n \"requests\": 30000\n },\n \"high\": {\n \"duration\": 60,\n \"threads\": 1000,\n \"interval\": 0.01,\n \"requests\": 1000000\n }\n },\n \"logging\": {\n \"log_file\": \"./logs/stressload.log\",\n \"log_format\": \"%(asctime)s - %(levelname)s - %(message)s\"\n }\n }\n}\n```\n\n---\n\n### `/configs/profiles.json`\nThis is an abstraction mechanism to define services configurations.\n\n**Ownership**: DevSecOps/DevNetOps Engieners\n\n```json\n{\n \"accounts\": {\n \"users\": {},\n \"groups\": {},\n \"service\": {\n \"read_only\": {\n \"name\": \"ro--service-account\",\n \"caption\": \"Service Account (Read Only)\"\n }\n },\n \"cloud_function\": {\n \"read_only\": {\n \"name\": \"ro--cloud-function\",\n \"caption\": \"Cloud Function SA (Stress Test)\"\n }\n }\n },\n \"credentials\": {\n \"dev\": {\n \"id\": \"Development\",\n \"description\": \"Development environment service account\",\n \"name\": \"dev-account\",\n \"filename\": \"dev-credentials.json\",\n \"group\": \"devs\",\n \"environments\": [\n \"dev\"\n ],\n \"roles\": [\n {\n \"resource\": \"bucket\",\n \"role\": \"roles/storage.objectAdmin\"\n },\n {\n \"resource\": \"project\",\n \"role\": \"roles/viewer\"\n }\n ]\n },\n \"staging\": {\n \"id\": \"Staging\",\n \"description\": \"Staging environment service account\",\n \"name\": \"staging-account\",\n \"filename\": \"staging-credentials.json\",\n \"group\": \"staging\",\n \"environments\": [\n \"dev\",\n \"staging\"\n ],\n \"roles\": [\n {\n \"resource\": \"bucket\",\n \"role\": \"roles/storage.objectAdmin\"\n },\n {\n \"resource\": \"project\",\n \"role\": \"roles/logging.viewer\"\n }\n ]\n },\n \"prod\": {\n \"id\": \"Production\",\n \"description\": \"Production environment service account\",\n \"name\": \"prod-account\",\n \"filename\": \"prod-credentials.json\",\n \"group\": \"prod\",\n \"environments\": [\n \"prod\"\n ],\n \"roles\": [\n {\n \"resource\": \"bucket\",\n \"role\": \"roles/storage.objectAdmin\"\n },\n {\n \"resource\": \"project\",\n \"role\": \"roles/monitor.viewer\"\n }\n ]\n },\n \"devops\": {\n \"id\": \"DevOps\",\n \"description\": \"DevOps service account\",\n \"name\": \"devops-account\",\n \"filename\": \"devops-credentials.json\",\n \"group\": \"devops\",\n \"environments\": [\n \"dev\",\n \"staging\",\n \"prod\"\n ],\n \"roles\": [\n {\n \"resource\": \"bucket\",\n \"role\": \"roles/storage.objectAdmin\"\n },\n {\n \"resource\": \"project\",\n \"role\": \"roles/logging.logWriter\"\n }\n ]\n }\n }\n}\n```\n\n---\n\n### `/configs/tagging.json`\nThis is an abstraction mechanism to define resource tagging within a unified control pane.\u003cbr /\u003e\nNote: Tags could be fixed (as it's: nesting JSON structures, etc.) or dynamically tailored by well defined policies.\n\n**Ownership**: Operations Management team\n\n```json\n{\n \"providers\": {\n \"gcp\": {\n \"cloud_function\": {\n },\n \"compute\": {\n \"tags\": [\n {\n \"value\": \"ssh-access\",\n \"fixed\": true\n },\n {\n \"value\": \"http-server\",\n \"fixed\": false\n }\n ]\n },\n \"firewall\": {\n \"allow_ssh\": {\n \"tags\": [\n {\n \"value\": \"ssh-access\",\n \"fixed\": true\n }\n ]\n },\n \"allow_ssh_iap\": {\n \"tags\": [\n {\n \"value\": \"ssh-access\",\n \"fixed\": true\n }\n ]\n }\n },\n \"load_balancer\": {\n \"tags\": [\n {\n \"value\": \"load-balancer\",\n \"fixed\": false\n }\n ]\n },\n \"accounts\": {\n \"tags\": [\n {\n \"value\": \"service-accounts\",\n \"fixed\": false\n }\n ]\n },\n \"networking\": {\n \"tags\": [\n {\n \"value\": \"networking\",\n \"fixed\": false\n }\n ]\n }\n }\n },\n \"globals\": {\n }\n}\n```\n\n---\n\n### `/configs/allowed.json`\nSpecifies IP ranges allowed through firewall ingress rules.\n* DevOps IPS: Whitelisting remote IP adddresses.\n* Private IPS: Determine Allowed VPC Peering traffic.\n* Console IPS: Allows for the GCP SSH Console access.\n\n```json\n{\n \"devops_ips\": [\n \"\u003cremote-ip-address\u003e\"\n ],\n \"private_ips\": [\n \"10.0.0.0/8\"\n ],\n \"console_ips\": [\n \"35.235.240.0/20\"\n ]\n}\n```\n\n---\n\n## GitHub Actions CI/CD Pipeline (`terraform.yaml`)\n\n### Trigger\nThe workflow is manually triggered using `workflow_dispatch` and accepts two parameters:\n- `target_environment`: one of `dev`, `staging`, or `prod`\n- `action`: `validate`, `plan`, `apply`, or `destroy`\n\n### Key Phases\n- **Environment Setup:**\n - Decodes credentials from GitHub Secrets → JSON file\n - Extracts region and forwarding rule name from JSON\n- **Diagnostics Phase:**\n - Lists active authentication, configurations, projects, zones, services, and networks using `gcloud`\n- **Terraform Lifecycle:**\n - Initializes backend\n - Switches to correct workspace or creates it\n - Validates syntax, plans, and applies changes\n- **Safe Destruction:**\n - Destroy allowed only in `dev`\n - Downloads and uploads Terraform state before deletion\n\n---\n\n## Testing Strategy\n\nThe project includes built-in testing hooks:\n1. `terraform validate`: Static validation for every execution\n2. `terraform plan`: Logged in GitHub workflow output\n3. (Optional) Load balancer HTTP check using public IP and `curl`\n4. Diagnostic shell commands for inspecting real-time cloud state\n\nAll logs are captured and uploaded to GitHub Artifacts to assist in post-deploy analysis or CI traceability.\n\n---\n\n## Terraform Outputs\n\nDefined in `outputs.tf` and printed after `apply`, these include:\n- `instance_ips`: List of internal or external IPs of all compute instances\n- `forwarding_rule_ip`: Public IP of the load balancer’s forwarding rule\n- `backend_service_name`: Name of the created backend service used by the load balancer\n\n---\n\n## Directory \u0026 File Layout\n\n```console\n├── .github/\n│   └── workflows/\n│   └── terraform.yaml\n├── .gitignore\n├── .terraform/\n│   ├── environment\n│   ├── modules/\n│   │   └── modules.json\n│   ├── providers/\n│   │   └── registry.terraform.io/\n│   │   └── hashicorp/\n│   │   └── google/\n│   │   └── 6.29.0/\n│   │   └── darwin_amd64/\n│   │   ├── LICENSE.txt\n│   │   └── terraform-provider-google_v6.29.0_x5*\n│   └── terraform.tfstate\n├── .terraform.lock.hcl\n├── backend.tf\n├── configs/\n│   ├── allowed.json\n│   ├── policies.json\n│   ├── profiles.json\n│   ├── providers/\n│   │   ├── aws.json\n│   │   ├── azure.json\n│   │   └── gcp.json\n│   ├── README.md\n│   ├── services/\n│   │   └── gcp/\n│   │   ├── cloud_function.json\n│   │   ├── compute_resources.json\n│   │   ├── firewall_rules.json\n│   │   ├── health_check.json\n│   │   ├── http_forwarding.json\n│   │   ├── load_balancer.json\n│   │   ├── networking.json\n│   │   ├── web_autoscaling.json\n│   │   └── web_backend.json\n│   ├── tagging.json\n│   └── targets/\n│   ├── dev.json\n│   ├── prod.json\n│   └── staging.json\n├── errata.md\n├── LICENSE\n├── locals.tf\n├── logs/\n│   └── README.md\n├── main.tf\n├── modules/\n│   └── gcp/\n│   ├── cloud_function/\n│   │   ├── cloud_function.outputs.tf\n│   │   ├── cloud_function.tf\n│   │   ├── cloud_function.variables.tf\n│   │   └── README.md\n│   ├── compute/\n│   │   ├── compute.outputs.tf\n│   │   ├── compute.tf\n│   │   ├── compute.variables.tf\n│   │   └── README.md\n│   ├── firewall/\n│   │   ├── firewall.outputs.tf\n│   │   ├── firewall.tf\n│   │   ├── firewall.variables.tf\n│   │   └── README.md\n│   ├── load_balancer/\n│   │   ├── load_balancer.outputs.tf\n│   │   ├── load_balancer.tf\n│   │   ├── load_balancer.variables.tf\n│   │   └── README.md\n│   ├── networking/\n│   │   ├── networking.manage.tf\n│   │   ├── networking.outputs.tf\n│   │   ├── networking.router.tf\n│   │   ├── networking.tf\n│   │   ├── networking.variables.tf\n│   │   └── README.md\n│   ├── profiles/\n│   │   ├── profiles.outputs.tf\n│   │   ├── profiles.tf\n│   │   ├── profiles.variables.tf\n│   │   └── README.md\n│   ├── README.md\n│   └── storage/\n│   ├── README.md\n│   ├── storage.outputs.tf\n│   ├── storage.tf\n│   ├── storage.variables.tf\n│   └── validating.steps\n├── outputs.json\n├── outputs.tf\n├── packages/\n│   ├── REDME.md\n│   └── stressload-webservers.zip\n├── project.json\n├── providers.tf\n├── README.md\n├── reports/\n│   ├── inspect.console\n│   ├── queries/\n│   │   ├── backend-services.json\n│   │   ├── compute-instances.json\n│   │   ├── firewall-rules-describe.json\n│   │   ├── firewall-rules.json\n│   │   ├── forwarding-rules.json\n│   │   ├── health-checks.json\n│   │   ├── instance-groups.json\n│   │   ├── instance-template.json\n│   │   ├── networks-listing.json\n│   │   ├── operations-listing.json\n│   │   ├── project-policies.json\n│   │   ├── proxies-listing.json\n│   │   ├── service-account.json\n│   │   ├── storage-buckets.json\n│   │   └── subnets-listing.json\n│   ├── README.md\n│   ├── terraform-apply.md\n│   ├── terraform-destroy.md\n│   ├── terraform-plan.md\n│   └── webserver.console\n├── scripts/\n│   ├── configure/\n│   │   └── apache-webserver.shell*\n│   ├── docs/\n│   │   ├── apache-webserver.md\n│   │   ├── destroy-services.md\n│   │   ├── gcloud-presets.md\n│   │   ├── inspect-autoscaling.md\n│   │   ├── inspect-services.md\n│   │   ├── package-functions.md\n│   │   └── terraform-backend.md\n│   ├── manage/\n│   │   ├── configure-backend.shell*\n│   │   ├── configure-profiles.shell*\n│   │   ├── configure-terraform.shell*\n│   │   ├── configure-workspaces.shell*\n│   │   ├── inspect-autoscaling.shell*\n│   │   ├── inspect-services.shell*\n│   │   └── package-functions.shell*\n│   ├── others/\n│   │   ├── destroy-resources.shell*\n│   │   ├── destroy-services.shell*\n│   │   ├── gcloud-presets.shell*\n│   │   └── inspect-resources.shell\n│   ├── README.md\n│   └── stressload/\n│   └── webservers/\n│   ├── config.json\n│   ├── main.py*\n│   └── requirements.txt\n└── variables.tf\n\n36 directories, 118 files\n```\n\n---\n\n## Design Philosophy\n\nThis project is engineered around the following principles:\n\n- **Zero Touch Configuration**\n All infrastructure logic is driven by structured JSON files (`project.json`, `configs/`, `targets/`). No hardcoded values or inline definitions are allowed in Terraform files or scripts.\n\n- **Multi-Environment Repeatability**\n The same codebase supports infinite environments (`dev`, `staging`, `prod`, etc.), each defined by its own target JSON config and Terraform workspace.\n\n- **Security-First Posture**\n - No default public exposure (`0.0.0.0/0`)\n - IP-restricted firewall access\n - Service accounts scoped to minimal privilege\n - State managed remotely in private buckets\n\n- **Automation-First Workflow**\n Shell scripts manage the full lifecycle: backend setup, provisioning, teardown, inspection, and Cloud Function deployment. CI/CD pipelines validate and enforce correct flows.\n\n- **Cloud-Native Diagnostics**\n GCP-native introspection tools (e.g., `gcloud`, `gsutil`) are embedded in `inspect-services.shell`, `inspect-autoscaling.shell`, and `destroy-services.shell` to validate runtime infrastructure.\n\n- **Terraform Best Practices**\n - State isolation via workspaces\n - Remote backend with auto-bucket creation\n - Strict separation between config, variables, and logic\n - Dynamic locals and centralized outputs for module interop\n\n---\n\n## Reviewer Evaluation Guide\n\n### What to Evaluate\n\n- **Security Discipline**\n Confirm firewall ingress is locked down. Inspect `allowed.json` and IAM bindings. Verify state bucket is private and access-controlled.\n\n- **Automation Scope**\n End-to-end flow should be repeatable with `./scripts/manage/*.shell` or CI trigger. Validate dry-run behavior and error fallback handling.\n\n- **Infrastructure Quality**\n Inspect module reuse, input/output separation, and config-driven behavior. Check if services are appropriately decoupled (compute, load balancer, function, etc.).\n\n- **Error Handling**\n Validate safe destruct conditions, missing dependency warnings, and interactive prompts in destructive actions.\n\n- **Real-World Readiness**\n Can the system be deployed to real GCP projects with minimal change? Are boundaries in place for destructive actions? Is monitoring/logging viable?\n\n### Suggested Scenarios to Test\n\n- Deploy `dev`, `staging`, and `prod` targets with different instance types and policies\n- Attempt `terraform destroy` in `staging` or `prod` (should be blocked or gated)\n- Modify `allowed.json` and validate new firewall rules are reflected\n- Rotate credentials via `configure-profiles.shell` and verify IAM keys\n- Run `inspect-autoscaling.shell` after applying `stressload` and confirm scale-out behavior\n- Review outputs generated in `outputs.json` and their use in `package-functions.shell`\n\n---\n\n## Future Enhancements\n\n- Multi-environment provisioning with JSON-driven isolation\n- Provider-agnostic module support (AWS, Azure)\n- DNS + certificate management via ACM + Cloud DNS\n- GitHub OIDC for eliminating static credentials\n- Multi-region failover for backend and load balancing\n- Prometheus or GCP Ops integration for metrics and alerting\n- Pre-commit hook integration (`tflint`, `tfsec`, `terraform fmt`)\n- Terraform module publishing for VPC, MIG, Firewall, etc.\n- Add custom CI stages for cost estimation and drift detection\n\n---\n\n## Project Components and Core Structure\n\nThis repository is structured for clarity, traceability, and reusability across cloud providers and deployment targets.\n\n- **Terraform Modules (`modules/gcp/*`)**\n Define infrastructure components (e.g., networking, compute, firewall, load balancer, cloud functions) as reusable units.\n\n- **JSON Configs (`configs/`)**\n Drive behavior dynamically. Includes environment targets, provider definitions, IAM profiles, policies, allowed networks, and tagging.\n\n- **Shell Automation (`scripts/`)**\n Declarative entrypoints for operations including:\n - State backend management\n - IAM credential lifecycle\n - Stress testing\n - Service teardown\n - Diagnostics and reporting\n\n- **CI/CD Pipeline (`.github/workflows/terraform.yaml`)**\n Handles plan, apply, teardown, artifact upload, and inspection logs using GitHub Actions.\n\n---\n\n## Core Terraform Files\n\n- **`main.tf`**\n Entrypoint for module orchestration. Passes environment-aware locals into downstream modules.\n\n- **`backend.tf`**\n Dynamically resolved backend configuration (GCS). Auto-created and validated via `configure-backend.shell`.\n\n- **`providers.tf`**\n Declares cloud provider plugins based on variables passed from config.\n\n- **`variables.tf`**\n Defines all inputs needed by `main.tf` and modules.\n\n- **`outputs.tf`**\n Captures environment outputs (e.g., IPs, URLs, names) and feeds downstream automation.\n\n---\n\n## Meta Files\n\n- **`.gitignore`**\n Ignores sensitive or local state files.\n\n- **`.terraform.lock.hcl`**\n Ensures consistent provider versions.\n\n- **`LICENSE`**\n Current licensing scheme (open source or proprietary).\n\n---\n\n## State and Logs\n\n- **`.local/`**\n Locally stored state backups and downloaded `.tfstate` for debugging or destruction flows.\n\n- **`outputs.json`**\n Terraform output exported in JSON format. Consumed by scripts for deployment and testing.\n\n- **`logs/` + `reports/`**\n Execution traces, introspection results, and command snapshots stored for each deployment.\n\n---\n\n## Directory Hierarchy Summary\n\n```bash\nmodules/ # Cloud-specific, reusable Terraform modules\nconfigs/ # JSON-based environment and policy configurations\nscripts/ # Automation shell and Python tools\n.github/ # CI/CD GitHub Actions workflows and metadata\npackages/ # Packaged artifacts (e.g., stressload function zips)\n.local/ # Terraform state backups and downloads\nlogs/ # CLI tool outputs and internal logs\nreports/ # GCP CLI JSON inspections and TF apply traces\n```\n\n---\n\n## Inter-component Relationships\n\n| Component | Depends On | Consumes |\n|---------------------------|---------------------------------|----------------------------------|\n| `main.tf` | `project.json`, `locals.tf` | Passes inputs to modules |\n| `gcp/compute` | `gcp/networking`, firewall | Tags, startup scripts |\n| `gcp/load_balancer` | `gcp/compute` | MIG, backend service, named port |\n| `gcp/cloud_function` | outputs.json, archive.zip | Target URL, IAM email |\n| `configure-backend.shell` | `project.json`, `policies.json` | Creates GCS state bucket |\n| `configure-profiles.shell`| `profiles.json` | IAM keys and service accounts |\n| `inspect-services.shell` | Terraform outputs | IPs, health checks, MIGs |\n| `terraform.yaml` | Everything | Orchestrates CI/CD flows |\n\n---\n\n## Summary\n\nThis project delivers a fully modular, secure, and automation-driven infrastructure framework for Google Cloud. Every component—from IAM to compute to serverless—is driven by JSON-based configuration and managed through traceable workflows. Terraform modules are st","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femvaldes%2Fterraform-prototype","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Femvaldes%2Fterraform-prototype","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femvaldes%2Fterraform-prototype/lists"}