{"id":35050058,"url":"https://github.com/elisiariocouto/specular","last_synced_at":"2025-12-27T09:12:16.789Z","repository":{"id":329139763,"uuid":"1118301628","full_name":"elisiariocouto/specular","owner":"elisiariocouto","description":"🪞 Specular is an open-source Terraform provider network mirror. This might evolve into a generic proxy mirror for other packages/artifacts.","archived":false,"fork":false,"pushed_at":"2025-12-20T00:27:55.000Z","size":134,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-21T22:54:58.969Z","etag":null,"topics":["cache","mirror","proxy","pull-through-cache","terraform","terraform-providers"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elisiariocouto.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2025-12-17T14:53:23.000Z","updated_at":"2025-12-20T18:56:16.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/elisiariocouto/specular","commit_stats":null,"previous_names":["elisiariocouto/speculum"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/elisiariocouto/specular","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elisiariocouto%2Fspecular","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elisiariocouto%2Fspecular/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elisiariocouto%2Fspecular/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elisiariocouto%2Fspecular/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elisiariocouto","download_url":"https://codeload.github.com/elisiariocouto/specular/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elisiariocouto%2Fspecular/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28076689,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-27T02:00:05.897Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cache","mirror","proxy","pull-through-cache","terraform","terraform-providers"],"created_at":"2025-12-27T09:12:16.030Z","updated_at":"2025-12-27T09:12:16.783Z","avatar_url":"https://github.com/elisiariocouto.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🪞 Specular\n\nSpecular is an open-source Terraform provider network mirror. This might evolve into a generic proxy mirror for other packages/artifacts.\n\nSpecular implements the [Terraform Provider Network Mirror Protocol](https://developer.hashicorp.com/terraform/internals/provider-network-mirror-protocol) to intercept provider requests, cache them locally, and serve subsequent requests from cache. This reduces dependency on upstream registries and improves deployment speeds.\n\n## Features\n\n- **Caching Proxy**: Cache Terraform providers locally to reduce upstream traffic\n- **Simple Configuration**: Environment variable-based configuration\n- **Observability**: Prometheus metrics and structured logging\n- **Extensible Storage**: Filesystem storage with interface for future S3 support\n\n## Quick Start\n\n\u003e **⚠️ Important**: Terraform requires network mirrors to be served over HTTPS with a valid certificate. Running Specular on plain HTTP (localhost development excluded) will not work with Terraform. Use a reverse proxy like [Caddy](https://caddyserver.com/), [Traefik](https://traefik.io/), or [nginx](https://nginx.org/) to handle TLS termination.\n\n### Installation\n\nDownload the latest release from the [releases page](https://github.com/elisiariocouto/specular/releases).\n\nAlternatively, run Specular using Docker:\n\n```bash\n# Docker Hub\ndocker run -p 8080:8080 \\\n  -e SPECULAR_BASE_URL=https://specular.example.com \\\n  elisiariocouto/specular:latest\n\n# GitHub Container Registry\ndocker run -p 8080:8080 \\\n  -e SPECULAR_BASE_URL=https://specular.example.com \\\n  ghcr.io/elisiariocouto/specular:latest\n```\n\nThe default images are based on distroless. Alpine-based images are available with the `-alpine` suffix (e.g., `latest-alpine`, `2025.12.3-alpine`) for a lightweight footprint. These are suitable for resource-constrained environments.\n\n### Using with Terraform\n\nConfigure Terraform to use the mirror by adding to `~/.terraformrc`:\n\n```hcl\nprovider_installation {\n  network_mirror {\n    url = \"https://specular.example.com/terraform/providers/\"\n  }\n}\n```\n\nThen run `terraform init` in any Terraform project and it will use your local mirror.\n\n\u003e **Note**: The URL must end with `/terraform/providers/` to match Specular's routing structure.\n\n## Configuration\n\nAll configuration is via environment variables:\n\n### Server Configuration\n- `SPECULAR_PORT` (default: `8080`) - HTTP server port\n- `SPECULAR_HOST` (default: `0.0.0.0`) - Bind address\n- `SPECULAR_READ_TIMEOUT` (default: `30s`) - HTTP read timeout\n- `SPECULAR_WRITE_TIMEOUT` (default: `30s`) - HTTP write timeout\n- `SPECULAR_SHUTDOWN_TIMEOUT` (default: `30s`) - Graceful shutdown timeout\n\n### Storage Configuration\n- `SPECULAR_STORAGE_TYPE` (default: `filesystem`) - Storage backend\n- `SPECULAR_CACHE_DIR` (default: `/var/cache/specular`) - Cache directory\n\n### Upstream Configuration\n- `SPECULAR_UPSTREAM_TIMEOUT` (default: `60s`) - Upstream request timeout\n- `SPECULAR_UPSTREAM_MAX_RETRIES` (default: `3`) - Max retry attempts\n\n### Mirror Configuration\n- `SPECULAR_BASE_URL` (default: `https://specular.example.com`) - Public base URL of mirror (without trailing slash). Supports hosting at subpaths (e.g., `https://example.com/mirror`). Note that Terraform provider endpoints are served under `/terraform/providers`, and observability endpoints (`/health`, `/metrics`) are served at the root of this base URL.\n\n### Observability Configuration\n- `SPECULAR_LOG_LEVEL` (default: `info`) - Log level: debug, info, warn, error\n- `SPECULAR_LOG_FORMAT` (default: `json`) - Log format: json, text\n- `SPECULAR_METRICS_ENABLED` (default: `true`) - Enable Prometheus metrics\n\n## API Endpoints\n\n\u003e **Note**: All Terraform provider endpoints are served under the `/terraform/providers` path prefix. This structure allows Specular to potentially support other package registries in the future (e.g., `/docker/registries`, `/npm`, `/pypi`, `/maven`) as a multi-ecosystem pull-through cache.\n\n### Terraform Provider Endpoints\n\n#### List Versions\n```\nGET $SPECULAR_BASE_URL/terraform/providers/:hostname/:namespace/:type/index.json\n```\n\nReturns available versions of a provider.\n\n**Example:**\n```\nhttps://specular.example.com/terraform/providers/registry.terraform.io/hashicorp/aws/index.json\n```\n\n#### List Packages\n```\nGET $SPECULAR_BASE_URL/terraform/providers/:hostname/:namespace/:type/:version.json\n```\n\nReturns available installation packages for a specific version.\n\n**Example:**\n```\nhttps://specular.example.com/terraform/providers/registry.terraform.io/hashicorp/aws/5.70.0.json\n```\n\n### Observability Endpoints\n\n#### Health\n```\nGET $SPECULAR_BASE_URL/health\n```\n\nHealth check endpoint. Returns `{\"status\":\"ok\"}` when the service is running.\n\n#### Metrics\n```\nGET $SPECULAR_BASE_URL/metrics\n```\n\nPrometheus metrics endpoint (returns 404 if metrics are disabled via `SPECULAR_METRICS_ENABLED=false`).\n\n## Contributing\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, running locally, and release procedures.\n\n## Architecture\n\nThe mirror consists of several layers:\n\n- **HTTP Server** - Handles requests and routing\n- **Mirror Service** - Core cache-or-fetch business logic\n- **Storage Layer** - Abstract interface with filesystem implementation\n- **Upstream Client** - Fetches from provider registries, uses Terraform's [Remote Service Discovery Protocol](https://developer.hashicorp.com/terraform/internals/remote-service-discovery)\n- **Observability** - Prometheus metrics and structured logging\n\n## Future Enhancements\n\n- S3 storage backend\n- Cache invalidation API\n- Pre-warming cache (already supported since the filesystem structure is the same as `terraform providers mirror`)\n- Authentication and authorization\n- Rate limiting\n- Support for other ecosystems (Docker, npm, PyPI, nuget, maven)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felisiariocouto%2Fspecular","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felisiariocouto%2Fspecular","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felisiariocouto%2Fspecular/lists"}