Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/ruben-sprengel/vigil-cd

VigilCD is a lightweight GitOps deployment agent that automatically monitors Git repositories and performs Docker Compose deployments on changes. Built for self-hosted environments with support for private repositories and Docker registries.
https://github.com/ruben-sprengel/vigil-cd

deployment docker docker-compose fastapi gitops selfhosted

Last synced: 5 months ago
JSON representation

VigilCD is a lightweight GitOps deployment agent that automatically monitors Git repositories and performs Docker Compose deployments on changes. Built for self-hosted environments with support for private repositories and Docker registries.

Awesome Lists containing this project

README 

          
# VigilCD - GitOps Deployment Agent



**VigilCD** is a lightweight GitOps deployment agent that automatically monitors Git repositories and performs Docker Compose deployments on changes. Built for self-hosted environments with support for private repositories and Docker registries.



[![VigilCD CI Checks](https://github.com/ruben-sprengel/vigil-cd/actions/workflows/vigilcd-ci.yml/badge.svg?branch=main)](https://github.com/ruben-sprengel/vigil-cd/actions/workflows/vigilcd-ci.yml)

[![VigilCD image build](https://github.com/ruben-sprengel/vigil-cd/actions/workflows/vigilcd-image-build.yml/badge.svg?&event=release)](https://github.com/ruben-sprengel/vigil-cd/actions/workflows/vigilcd-image-build.yml)

[![trivy-scan](https://github.com/ruben-sprengel/vigil-cd/actions/workflows/trivy-scan.yml/badge.svg)](https://github.com/ruben-sprengel/vigil-cd/actions/workflows/trivy-scan.yml)



[![Python 3.14](https://img.shields.io/badge/python-3.14-blue.svg?logo=python&logoColor=white)](https://www.python.org/downloads/)

[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv)

[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)

[![ty](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ty/main/assets/badge/v0.json)](https://github.com/astral-sh/ty)

[![pytest](https://img.shields.io/badge/pytest-green.svg?logo=pytest&logoColor=white)](https://github.com/pytest-dev/pytest)



[![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/rubensprengel)



## 🎯 Features



- ✅ **Automatic Git Sync** - Periodic polling or webhook-based

- ✅ **Parallel Processing** - Independent jobs per repo/branch for maximum performance

- ✅ **Multi-Repository** - Manage multiple repos and branches simultaneously

- ✅ **SSH & HTTPS Auth** - Private GitHub/GitLab repositories

- ✅ **Private Registries** - GHCR, Docker Hub, self-hosted with secure credential cleanup

- ✅ **Health Checks** - Auto-recovery on failures

- ✅ **RESTful API** - Status monitoring and manual triggers

- ✅ **Flexible Timeouts** - Configurable timeouts with float precision or disable entirely

- ✅ **Secure** - Non-root container (UID 1000) + guaranteed credential cleanup

- ✅ **Multi-Arch** - AMD64 and ARM64 (Raspberry Pi)



## 🚀 Quick Start



### Prerequisites



- Docker

- Docker Compose



### Installation



```bash

# 1. Clone repository

git clone https://github.com/ruben-sprengel/vigil-cd.git

cd vigil-cd



# 2. Run setup (auto-configures everything)

chmod +x setup.sh

./setup.sh



# 3. Edit configuration

vim config/config.yaml  # Add your repositories



# 4. Add secrets

vim .env  # Add authentication tokens



# 5. Start VigilCD

docker-compose up -d



# 6. Verify

curl http://localhost:8000/health

```



**That's it!** VigilCD is now monitoring your repositories.



## ⚙️ Configuration



### What `setup.sh` Does



The setup script automatically:

- ✅ Creates `config.yaml` from template

- ✅ **Auto-detects Docker socket GID** (required for deployments)

- ✅ Generates `.env` with correct permissions

- ✅ Sets up directories (`repos/`, `ssh-keys/`, `logs/`)

- ✅ (Optional) Generates SSH keys for private repos

- ✅ Validates configuration



### Repository Configuration



Edit `config/config.yaml`:



```yaml

repos:

  # Public HTTPS repository

  - name: "my-app"

    url: "https://github.com/username/my-app"

    auth_method: "https"

    branches:

      - name: "main"

        targets:

          - name: "app"

            file: "docker-compose.yml"

            deploy: true

            build_images: false



  # Private SSH repository

  - name: "private-app"

    url: "git@github.com:username/private-app.git"

    auth_method: "ssh"

    branches:

      - name: "production"

        targets:

          - name: "api"

            file: "docker-compose.yml"

            deploy: true

            build_images: true



  # With private Docker registry

  - name: "microservice"

    url: "git@github.com:company/microservice.git"

    auth_method: "ssh"

    registries:

      - url: "ghcr.io"

        username: "bot"

        password_env_var: "GHCR_TOKEN"

    branches:

      - name: "main"

        targets:

          - name: "service"

            file: "docker-compose.yml"

            deploy: true

```



### Environment Variables



Edit `.env` (auto-generated by `setup.sh`):



```bash

# Docker Configuration (auto-detected)

DOCKER_GID=999



# Git Authentication

VIGILCD_GITHUB_TOKEN=ghp_xxxxx



# Docker Registry Authentication

GHCR_TOKEN=ghp_xxxxx



# Scheduling (optional)

VIGILCD_CHECK_INTERVAL_MINUTES=5

VIGILCD_LOG_LEVEL=INFO

```



## 🔐 SSH Keys for Private Repos



### Automatic Setup



The `setup.sh` script prompts you to generate SSH keys. Alternatively:



```bash

# Generate key

ssh-keygen -t ed25519 -C "vigilcd-test" -f id_ed25519_vigilcd



# Show public key

cat ./ssh-keys/id_ed25519_vigilcd.pub



# Add to GitHub:

# Repository → Settings → Deploy Keys → Add deploy key

```



### Multiple Keys



For different repos with different keys:



```yaml

repos:

  - name: "service-a"

    url: "git@github.com:company/service-a.git"

    auth_method: "ssh"

    ssh_key_path: "/home/vigilcd/.ssh/service_a_key"

```



## 🐳 Private Docker Registries



### Configuration



1. **Add token to `.env`:**



```bash

GHCR_TOKEN=ghp_xxxxx

```



2. **Configure in `config.yaml`:**



```yaml

repos:

  - name: "my-app"

    registries:

      - url: "ghcr.io"

        username: "deployment-bot"

        password_env_var: "GHCR_TOKEN"

```



### Supported Registries



- GitHub Container Registry (ghcr.io)

- Docker Hub (docker.io)

- Self-hosted registries

- AWS ECR, GCR, etc.



## 📡 API Endpoints



Check out the interactive swagger docs here:



```

http://localhost:8000/docs

```



## 🏗️ Architecture



```

VigilCD Container (vigilcd user)

├── FastAPI (Web API)

├── APScheduler (Periodic checks)

├── GitPython (Git operations)

└── DeploymentService

    ├── Git sync (fetch + reset --hard)

    ├── Docker Compose deployment

    └── Health checks

```



### How It Works



1. **Scheduler** triggers sync check every N minutes

2. **Git Check** fetches remote commit hash

3. **Comparison** with local commit

4. **Update** if different: `git fetch` + `git reset --hard`

5. **Deploy** via `docker compose up -d`

6. **Health Check** monitors container status

7. **Recovery** redeploys failed containers



## 🐛 Troubleshooting



### Permission Issues



```bash

./fix-permissions.sh

docker-compose restart vigilcd

```



### Docker Socket Access



```bash

# Verify DOCKER_GID in .env

cat .env | grep DOCKER_GID



# Re-run setup if changed

./setup.sh

```



### SSH Authentication Failed



```bash

# Check key permissions

ls -la ./ssh-keys/



# Test connection

docker exec vigilcd ssh -T git@github.com

```



### Config Changes Not Applied



```bash

docker-compose restart vigilcd

docker-compose logs vigilcd

```



### View Logs



```bash

# Follow logs

docker-compose logs -f vigilcd



# Last 100 lines

docker-compose logs --tail=100 vigilcd



# Search for errors

docker-compose logs vigilcd | grep -i error

```



## 🔒 Security



### Files in Git ✅



```

✅ config/config.template.yaml

✅ .env.template

✅ docker-compose.yml

✅ Dockerfile

✅ setup.sh

```



### Never Commit ❌



```

❌ config/config.yaml (repo-specific)

❌ .env (secrets!)

❌ ssh-keys/ (private keys!)

❌ repos/ (cloned repos)

```



### Best Practices



- ✅ Use **Deploy Keys** (not personal tokens)

- ✅ Rotate credentials every 90 days

- ✅ Limit token scopes (`read:packages` only)

- ✅ Run container as non-root (UID 1000)

- ✅ Use HTTPS for webhooks (reverse proxy)



## 🔄 Updating



### Configuration Changes



```bash

vim config/config.yaml

docker-compose restart vigilcd

```



### Update Image



```bash

docker-compose pull

docker-compose up -d

```



### Add New Repository



1. Edit `config/config.yaml`

2. Restart: `docker-compose restart vigilcd`

3. Verify: `curl http://localhost:8000/api/status`



## 🎛️ Advanced Configuration



### Environment Variables



| Variable                         | Default       | Description                             |

|----------------------------------|---------------|-----------------------------------------|

| `DOCKER_GID`                     | Auto-detected | Docker socket group (required)          |

| `VIGILCD_CHECK_INTERVAL_MINUTES` | `5`           | Polling interval                        |

| `VIGILCD_GIT_RETRY_COUNT`        | `3`           | Git operation retries                   |

| `VIGILCD_DOCKER_TIMEOUT`         | `300`         | Docker Compose timeout (seconds)        |

| `VIGILCD_LOG_LEVEL`              | `INFO`        | Log level (DEBUG, INFO, WARNING, ERROR) |



### Repository Options



| Field          | Required | Description                               |

|----------------|----------|-------------------------------------------|

| `name`         | ✅        | Unique identifier                         |

| `url`          | ✅        | Git URL (HTTPS or SSH)                    |

| `auth_method`  | ❌        | `"https"` or `"ssh"` (default: `"https"`) |

| `ssh_key_path` | ❌        | Custom SSH key path                       |

| `registries`   | ❌        | Private Docker registries                 |



### Target Options



| Field          | Default | Description                     |

|----------------|---------|---------------------------------|

| `deploy`       | `true`  | Auto-deploy on changes          |

| `build_images` | `false` | Run `docker compose up --build` |



## 📊 Production Deployment



### Resource Limits



```yaml

# docker-compose.yml

deploy:

  resources:

    limits:

      cpus: '1.0'

      memory: 512M

```



### Logging



```yaml

logging:

  driver: "json-file"

  options:

    max-size: "10m"

    max-file: "3"

```



### Monitoring



- Health endpoint: `/health`

- Status endpoint: `/api/status`

- Metrics: (Coming soon)



### Backup



```bash

tar -czf vigilcd-backup.tar.gz config/ .env ssh-keys/

```



## 🤝 Contributing



Contributions welcome! Please:



1. Fork the repository

2. Create feature branch

3. Submit pull request



### Development Setup



```bash

pip install uv

uv sync --extra dev

uv run uvicorn src.app:app --reload

```



## 📝 License



MIT License - see [LICENSE](LICENSE) file



## 🙏 Acknowledgments



Built with:

- [FastAPI](https://fastapi.tiangolo.com/)

- [GitPython](https://gitpython.readthedocs.io/)

- [APScheduler](https://apscheduler.readthedocs.io/)

- [uv](https://github.com/astral-sh/uv)



## 📮 Support



- **Issues:** [GitHub Issues](https://github.com/ruben-sprengel/vigil-cd/issues)

- **Documentation:** This README

- **Sponsor:** [Ko-fi](https://ko-fi.com/rubensprengel)