https://github.com/adobe/kompos
https://github.com/adobe/kompos
Last synced: 5 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/adobe/kompos
- Owner: adobe
- License: apache-2.0
- Created: 2019-11-18T15:39:35.000Z (over 6 years ago)
- Default Branch: main
- Last Pushed: 2025-11-07T18:05:28.000Z (7 months ago)
- Last Synced: 2025-11-07T20:08:27.030Z (7 months ago)
- Language: Python
- Size: 214 KB
- Stars: 19
- Watchers: 12
- Forks: 12
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# kompos
[](https://www.travis-ci.com/adobe/kompos) [](https://hub.docker.com/r/adobe/kompos) [](https://microbadger.com/images/adobe/kompos "Get your own version badge on microbadger.com") [](https://github.com/adobe/kompos/blob/master/LICENSE) [](https://pypi.python.org/pypi/kompos/)
**Kompos** is a configuration source management tool for infrastructure platforms. It maintains a Git-based layered
configuration structure that generates environment-specific configurations for Terraform and other infrastructure tools.
**Perfect for:** Multi-environment deployments, cell-based architectures, platform engineering teams managing 10s-100s
of clusters.
**Workflow:** Layered config source in Git → Kompos generates → Commit → Terraform/TFE consumes → Deploy
## Key Features
- **Layered Configuration**: Organize configs by cloud/environment/region/cluster with automatic inheritance and merge
- **Generate Configurations**: Produce tfvars, workspace configs, and composition files for Terraform/TFE
- **Value Interpolation**: Dynamic config resolution with `{{key.path}}` placeholders
- **Configuration Analysis**: Trace value origins, visualize hierarchy, compare environments
- **Extensible Runners**: Built-in Terraform and TFE runners, extensible architecture for custom destinations
### Core Benefits
- ✅ **Version Controlled**: All changes tracked in Git with full audit trail
- ✅ **DRY at Scale**: Define once, inherit everywhere. Start with cluster-specific configs, refactor common patterns
upward to shared layers
- ✅ **Clear Precedence**: Specific configs override general ones automatically
- ✅ **Environment Variants**: Dev uses latest versions, prod uses stable - same codebase
- ✅ **Effortless Updates**: Change one shared config file → update 50+ clusters instantly
- ✅ **Clean Separation**: Source configs vs generated artifacts
Below is a graphical representation of the data flow, showing how hierarchical
configurations are merged and interpolated before being injected into runners:
## Installation
**Requirements:** Python 3.11 or higher
### PyPI (Recommended for Users)
```bash
# Install
pip install kompos
# Upgrade to latest version
pip install --upgrade kompos
```
### Using Virtual Environment (Recommended)
Using virtualenv for isolated installation:
```bash
pip install virtualenv
virtualenv kompos-env
source kompos-env/bin/activate
(kompos-env) pip install kompos
(kompos-env) pip install --upgrade kompos
(kompos-env) kompos --version
```
### Locally for Development
Using virtualenv with editable install:
```bash
pip install virtualenv
virtualenv .env
source .env/bin/activate
(env) cd kompos/
(env) pip install --editable .
```
## Documentation
- **[📖 Quick Guide](./docs/GUIDE.md)** - Get started in 5 minutes
- **[📚 Advanced Guide](./docs/ADVANCED.md)** - Architecture, runners, debugging, best practices
- **[💡 Examples](./examples/)** - Hands-on tutorials (beginner to advanced)
- **[🧪 Testing](./tests/README.md)** - Run tests and add new ones
## Testing
```bash
# Quick tests (no dependencies)
python tests/run_tests.py
# Comprehensive tests (requires pytest)
pip install pytest pyyaml
pytest tests/ -v
```
See [tests/README.md](./tests/README.md) for details.
## Layered Configuration
Kompos leverages [himl](https://github.com/adobe/himl) to provide a layered configuration structure as your **Git-based
source of truth**. Configuration files are organized hierarchically and automatically merged based on directory paths,
enabling powerful inheritance patterns.
**How it works:** Base configurations are defined at higher levels (e.g., cloud, region) and automatically inherited by
more specific levels (e.g., cell, cluster), with the ability to override any value at any level.
**Ideal for cell-based architectures:** Define shared cell configuration once (networking, security, compliance), then
override per-cell specifics (capacity, module versions, feature flags). One config change can update an entire cell
deployment pattern.
**The DRY workflow:**
1. **Start specific** - Define per-cluster configs as you build
2. **Spot patterns** - Notice repeated values across clusters
3. **Refactor upward** - Move common config to shared layers (region → env → cloud)
4. **Override when needed** - Keep only differences at cluster level
**Example:** All clusters use `terraform_version: "1.5.0"` → Move to `cloud=aws/defaults.yaml` → One change updates
everything.
This enables:
- **Git-Based Workflow**: Config changes via PRs → Generate → Terraform/TFE consumes
- **Config Inheritance**: Define once at base level, override only what differs
- **Value Interpolation**: Reference values with `{{key.path}}` syntax
- **Artifact Generation**: Produce tfvars, compositions, workspace configs for downstream tools
- **Config Analysis**: Trace origins, visualize hierarchy, compare environments
- **Scale**: Manage 50+ clusters from a handful of config files
## Examples
Comprehensive examples are available in [`examples/`](./examples/) with a progressive learning path:
1. **[Layered Configuration](./examples/01-hierarchical-config/)** - Learn configuration inheritance and merge behavior
2. **[Module Version Pinning](./examples/02-module-version-pinning/)** - Dynamic Terraform module versioning with
`.tf.versioned` files
3. **[Config Exploration](./examples/03-config-exploration/)** - Analyze and visualize configuration hierarchies
4. **[TFE Multi-Cluster](./examples/04-tfe-multi-cluster/)** - Terraform Enterprise workspace and composition generation
See the [Examples README](./examples/README.md) for a complete guide with difficulty levels, time estimates, and
learning paths.
## Usage
Generate configuration artifacts from layered config source for infrastructure tools:
```bash
# TFE: Generate workspace configs, tfvars, and compositions (primary)
kompos tfe generate
# Terraform: Generate configs and run Terraform locally
kompos terraform
# Explore: Analyze config hierarchy and trace values
kompos explore
# Config: View merged configuration
kompos config --format yaml
```
**Workflow:** Generate configs → Commit → Terraform/TFE consumes → Deploy
**Note:** helmfile runner is deprecated. For Kubernetes, use GitOps tools (ArgoCD/Flux) or the planned helm runner.
### Common Config Commands
The `config` command supports all HIML arguments natively for flexible configuration viewing and debugging:
```bash
# Example path from hierarchical example
CONFIG_PATH="examples/01-hierarchical-config/config/cloud=aws/env=dev/cluster=cluster1/composition=terraform/terraform=cluster"
# View full merged configuration
kompos $CONFIG_PATH config
# View as JSON
kompos $CONFIG_PATH config --format json
# Filter: show only specific keys
kompos $CONFIG_PATH config --filter cluster --filter vpc
# Exclude: hide specific keys
kompos $CONFIG_PATH config --exclude terraform --exclude composition
# Save to file
kompos $CONFIG_PATH config --output-file merged-config.yaml
# Wrap output under a key (useful for Terraform)
kompos $CONFIG_PATH config --enclosing-key config
# Skip interpolation validation (useful for templates with missing values)
kompos $CONFIG_PATH config --skip-interpolation-validation
# Skip secret resolution (faster for debugging)
kompos $CONFIG_PATH config --skip-secrets
# Combine multiple options
kompos $CONFIG_PATH config \
--filter cluster --filter vpc \
--format json \
--output-file cluster-vpc.json \
--skip-secrets
```
**Tip:** Use `--filter` to inspect specific sections of your configuration during development and debugging.
## Docker Image
Docker images are not currently maintained. Please use PyPI installation for the latest version.
Docker image [adobe/kompos](https://hub.docker.com/r/adobe/kompos)
## License
[Apache License 2.0](/LICENSE)