{"id":29356901,"url":"https://github.com/cesnet/exafs-deploy","last_synced_at":"2025-08-30T20:13:58.835Z","repository":{"id":297595812,"uuid":"997254424","full_name":"CESNET/ExaFS-deploy","owner":"CESNET","description":"Ansible playbooks for ExaFS deployment","archived":false,"fork":false,"pushed_at":"2025-06-06T15:44:01.000Z","size":41,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-18T17:19:23.845Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CESNET.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-06T08:04:17.000Z","updated_at":"2025-06-06T15:44:03.000Z","dependencies_parsed_at":"2025-06-06T10:32:28.036Z","dependency_job_id":"343ab3a4-7b0e-463f-94c6-51f984990151","html_url":"https://github.com/CESNET/ExaFS-deploy","commit_stats":null,"previous_names":["cesnet/exafs-deploy"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/CESNET/ExaFS-deploy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CESNET%2FExaFS-deploy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CESNET%2FExaFS-deploy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CESNET%2FExaFS-deploy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CESNET%2FExaFS-deploy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CESNET","download_url":"https://codeload.github.com/CESNET/ExaFS-deploy/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CESNET%2FExaFS-deploy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272900163,"owners_count":25012034,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-30T02:00:09.474Z","response_time":77,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-09T05:40:47.966Z","updated_at":"2025-08-30T20:13:58.817Z","avatar_url":"https://github.com/CESNET.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deployment Scripts for ExaFS\n## with Docker Compose and Ansible\n\nThis repository contains Ansible playbooks for deploying [ExaFS](https://github.com/CESNET/) with Docker Compose on a target machine.\n\n## Introductory Notes\n\n* The Ansible playbooks currently expect the target OS to be RPM-based – RHEL 9 or Rocky Linux. Support for other operating systems is work in progress.\n* Services are installed under the root account, with possible privilege escalation to the `deploy` user.\n* You must set up an SSH key on the target server and have Ansible installed to run the playbook.\n\n## Prerequisites\n\nThe following tasks must be completed manually before starting the deployment:\n\n* **HTTP server installation and authentication method selection**\n  * Our recommended combination is Apache httpd + Shibboleth Auth\n  * Don't forget to set up HTTPS certificates and configure ProxyPass to Docker\n  * Proxy configuration should include the X-Forwarded-For header\n  * Configuration files `shib.conf` and `ssl.conf` are provided as examples in [./docs/apache_conf](./docs/apache_conf)\n* **Docker installation**\n  * Install [Docker + Docker Compose plugin](https://docs.docker.com/engine/install/rhel/) from the Docker repository\n* **Firewall configuration**\n* **RHEL-specific requirement**\n  * For RHEL, the `codeready-builder-for-rhel-9-x86_64-rpms` repository must be enabled due to dependencies (python3.x packaging)\n\n## Deployment Steps\n\n1. **Generate environment variables and secrets**\n   * Use the `generate_env_vars.py` script to generate the `secrets.yaml` file for installation\n   * The generated data is written to `inventory/host_vars/{hostname}/secrets.yaml`\n   * This file can be manually edited or created entirely using the provided template in the `example` directory\n   * The generation script has a CLI for specifying parameters – most importantly, the public IP addresses of the target machine\n   * Usage:\n     ```bash\n     python generate_env_vars.py --help\n     python generate_env_vars.py --hostname test.example.com --local-ip 192.168.1.2\n     ```\n   * Requirements: `click`, `PyYAML`. See `requirements.txt`\n\n2. **Configure inventory**\n   * Create an entry for the server in `inventory/host_vars/hosts.yaml`\n\n3. **Configure application files**\n   * In `roles/exafs/files/exafs_app/`, modify `config.py` and `run.py` if needed, according to your planned installation\n   * Key parameters are usually set via environment variables, so modifying `secrets.yaml` is often sufficient\n\n4. **Set ExaFS version**\n   * Currently the develop branch of [ExaFS repository](https://github.com/CESNET/exafs/tree/develop) is used\n   * This can be modified in the Dockerfile of the exafs role in `roles/exafs/files/exafs_app/`\n\n5. **Configure database**\n   * The directory `roles/exafs/files/database/` contains the files `01_app_tables_data.sql` and `02_rule_tables_empty.sql`, which create the basic database structure used in CESNET\n   * The content can be replaced by a database dump (e.g., when restoring from a backup)\n   * If the database doesn't have the ExaFS 1.1.x structure, migration is necessary (see 7. bellow)\n\n6. **Run deployment**\n   * Run `ansible-playbook site.yaml`\n   * This file defines the order of playbooks – database, RabbitMQ, application, ExaBGP + process\n   * If needed, you can run each playbook separately, for example:\n     ```bash\n     ansible-playbook site.yaml --tags exabgp\n     ```\n   * Or limit execution to a specific host:\n     ```bash\n     ansible-playbook site.yaml --limit hostname\n     ```\n\n7. **Database migration (if needed)**\n   * If database migration is needed, run the script with an environment variable that triggers the migration\n   * Migration is only necessary when the model changes – typically with a major application version change (see ExaFS changelog)\n   ```bash\n   ansible-playbook -i inventory site.yaml -e \"run_migrations=true\" --limit hostname\n   ```\n\n## What Gets Installed and Where?\n\n* **ExaFS application**\n  * Runs in Docker containers under the `deploy` user (Ansible creates this user if it doesn't exist)\n  * The application is installed in `/opt/exafs`, where you can manage the containers using Docker Compose\n* **Database backups**\n  * Daily database backups are configured at 3 AM to `/opt/exafs/backups`\n* **Scheduled tasks**\n  * A cron job is configured to call the endpoint `https://example.com/rules/withdraw_expired` every 10 minutes\n* **Services architecture**\n  * The `guarda` service is no longer used\n  * The endpoint `/rules/announce_all` is contacted directly by the ExaBGP service after restart (via `ExecStartPost`)\n  * RabbitMQ is used for sending messages from ExaFS to ExaBGP\n  * The communication process (formerly `exa_api`) is installed as the Python package `exabgp-process`\n* **Logging**\n  * Containers use the 'syslog' driver; logs are then written via syslog into three files in `/opt/exafs/logs/`\n  * Standard Docker Compose logging is also available\n  * `exabgp-process` logs together with ExaBGP to `/var/log/exabgp`\n\n## Post-Installation Configuration\n\n* **ExaBGP configuration**\n  * [ExaBGP package](https://github.com/Exa-Networks/exabgp) is installed by Ansible scripts, but need final configuration.\n  * At the beginning of `/etc/exabgp/exabgp.conf`, add:\n\n```\nprocess flowspec {\n    run /usr/local/bin/exabgp-process;\n    encoder json;\n}\n```\n  * Connect ExaBGP to your network","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcesnet%2Fexafs-deploy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcesnet%2Fexafs-deploy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcesnet%2Fexafs-deploy/lists"}