{"id":43171944,"url":"https://github.com/opencitations/infrastructure","last_synced_at":"2026-02-01T02:35:40.206Z","repository":{"id":273844425,"uuid":"918099092","full_name":"opencitations/infrastructure","owner":"opencitations","description":"This repo implements OpenCitations Kubernetes IaC, with all the orchestration manifests and deployment configs.","archived":false,"fork":false,"pushed_at":"2026-01-29T13:08:13.000Z","size":103,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2026-01-30T03:10:35.863Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/opencitations.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-01-17T08:48:59.000Z","updated_at":"2026-01-29T13:08:16.000Z","dependencies_parsed_at":"2025-01-23T10:26:09.079Z","dependency_job_id":"7a9ede5b-3ec3-46fe-afd5-c86b5031791b","html_url":"https://github.com/opencitations/infrastructure","commit_stats":null,"previous_names":["opencitations/infrastructure"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/opencitations/infrastructure","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencitations%2Finfrastructure","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencitations%2Finfrastructure/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencitations%2Finfrastructure/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencitations%2Finfrastructure/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/opencitations","download_url":"https://codeload.github.com/opencitations/infrastructure/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencitations%2Finfrastructure/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28965430,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-01T02:14:24.993Z","status":"ssl_error","status_checked_at":"2026-02-01T02:13:55.706Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-01T02:35:39.592Z","updated_at":"2026-02-01T02:35:40.195Z","avatar_url":"https://github.com/opencitations.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OpenCitations Kubernetes Infrastructure\n\nThis repository contains the Kubernetes manifests and deployment tools for deploying and managing OpenCitations services on a Kubernetes cluster. To begin working with this infrastructure, you'll need to either clone or fork this repository to your own GitHub account.\n\n## Prerequisites\n\n- Python 3.11\n- Running Kubernetes cluster with kubectl configured\n- Helm package manager\n- Git (for Fleet integration)\n- Storage system (NFS or alternative)\n\n## Installation Steps\n\n### 1. Database Preparation\n\nBefore beginning the deployment, you must prepare the OpenCitations databases:\n\n1. Download the Meta and Index databases from https://opencitations.net/download\n2. Place these databases in your storage system that will be used by the infrastructure\n3. Make note of the storage paths as they will be needed in the configuration\n\n### 2. Environment Configuration\n\nCreate and configure your environment file:\n\n```bash\ncp .env.example .env\n```\n\nEdit the `.env` file with your specific configurations for services, infrastructure, and Git integration.\n\n### 3. Storage Configuration\n\n#### For NFS Storage:\n1. Edit `preliminary/00-secrets.yaml` with your NFS configuration\n2. Modify `preliminary/02-storage.yaml` with your NFS paths and settings\n3. Ensure the following variables are properly set in `.env`:\n   - `NFS_SERVER`\n   - `NFS_CERT_PATH`\n   - `NFS_DATA_PATH`\n   - `NFS_LOG_PATH`\n   - `NFS_LOG_TRAEFIK_SUBPATH`\n\n#### For Alternative Storage:\n1. Modify both `preliminary/00-secrets.yaml` and `preliminary/02-storage.yaml` to match your storage system's requirements\n2. Update the corresponding storage variables in `.env`\n\n### 4. Traefik Configuration\n\n1. Edit `preliminary/03-traefik-values.yaml`:\n   - Modify `additionalArguments` section for HTTPS certificate configuration\n   - If not using MetalLB, for instance in a Cloud environment, remove the MetalLB-specific configurations\n\n### 5. Domain Configuration\n\nUpdate the domain addresses in .env configuration file:\n- Web service manifests\n- API service manifests\n- Any other services with web addresses\n\n### 6. Services system requirements\n\nConfigure your service requirements in the `.env` file. Each service has specific resource needs that should be adjusted based on your infrastructure capacity.\n\nImportant considerations:\n- CPU and memory requests should be set according to your cluster's available resources\n- Storage sizes should accommodate your data requirements plus growth\n- Service versions should match your deployment requirements\n- Ports should be available and not conflict with other services\n\n### 7. WordPress Configuration (OPTIONAL)\n\nThe OpenCitations infrastructure currently features a showcase website developed on WordPress. To set up a showcase site using this same technological approach, uncomment the 03 and 04 YAML sections and update the WordPress configuration parameters specified in the .env file.\n\nThe infrastructure includes an automated backup system for WordPress using rclone and pCloud storage (YAML 04). The system performs daily backups of:\n- WordPress database (SQL dump)\n- WordPress files\n- MariaDB raw data\n\nConfigure the backup system by setting the appropriate variables in your `.env`:\n```ini\nWORDPRESS_SUBPATH=wordpress_prod\nMARIADB_SUBPATH=mariadb_prod\nBACKUP_SCHEDULE=\"0 2 * * *\"\nBACKUP_RETENTION_DAYS=90\nPCLOUD_BACKUP_FOLDER=backup/wordpress\nRCLONE_CONFIG=your_base64_encoded_config\n```\n\nIf you install WordPress and/or the backup system, remove the `_OPTIONAL` suffix from the file names.\n\nWP backup info ---\u003e docs/wp-backup.md\nRedis token implementation info ---\u003e docs/oc-api-token.md\n\n### 8. Fleet Integration\n\nFleet provides automated deployment capabilities through Git repository monitoring.\n\n#### Fleet Repository Setup\n\n1. Create a **private** Git repository for your production manifests\n2. Configure Fleet variables in `.env`:\n```bash\nPRIVATE_REPO_URL=https://github.com/your-org/your-repo\nGIT_USERNAME=your-username\nGIT_TOKEN=your-personal-access-token\n```\n\n#### Fleet Configuration in Rancher\n\n##### If using Rancher:\n\n1. Navigate to Rancher UI → Continuous Delivery\n2. Create a new Git Repository with:\n   - Name: opencitations-fleet\n   - Repository URL: Your private repository URL\n   - Branch: main\n   - Paths: ./ (root directory)\n   - Target cluster: Your cluster name\n\n##### If using standalone Fleet:\n\n1. Create a Fleet configuration file `fleet.yaml`:\n```yaml\nnamespace: opencitations\ntargetCustomizations:\n  - name: production\n    clusterSelector:\n      matchLabels:\n        environment: production\n```\n\n2. Apply the configuration:\n```bash\nkubectl apply -f fleet.yaml\n```\n\n### 9. Deployment\n\nInstall Python dependencies:\n```bash\npip3.11 install -r requirements.txt\n```\n\nThe deployment script (`deploy.py`) provides several options:\n\n```bash\npython3.11 ./deploy.py -i    # Initialize infrastructure\npython3.11 ./deploy.py -p    # Preview manifest or preliminary files with variable substitution\npython3.11 ./deploy.py -f    # Create Fleet-ready production files\npython3.11 ./deploy.py       # Deploy all services\npython3.11 ./deploy.py \u003cmanifests/0x-manifest.yaml\u003e  # Deploy a specific manifest file\n```\n\n#### Initialize Infrastructure\n\n#### Initialize Infrastructure\n\nThe first time you deploy, you'll need to initialize the infrastructure by deploying all manifests in the `preliminary` folder.  Use the `-i` switch with the `deploy.py` script to do this:\n\n```bash\n./deploy.py -i\n```\n\nThis will guide you through:\n1. Creating Kubernetes secrets\n2. Setting up MetalLB (if used)\n3. Configuring storage\n4. Installing Traefik\n5. Setting up the Traefik dashboard (optional)\n\n#### Preview Configuration\n```bash\npython3.11 ./deploy.py -p \u003cmanifests/0x-manifest.yaml\u003e  # Preview how variables will be substituted\n```\nThis allows you to verify your configuration before deployment by showing how environment variables will be substituted in your YAML files.\n\n#### Deploy Using Fleet\n```bash\npython3.11 ./deploy.py -f  # For Fleet-managed deployments\n```\nThis will process all manifests and push them to your Fleet repository.\n\nTo configure Fleet, use the section in .env.example:\n\n```bash\nPRIVATE_REPO_URL=https://github.com/username/private-repo.git\nGIT_USERNAME=your-username\nGIT_TOKEN=your-personal-access-token\n```\n\n#### Direct Deployment\n\nIf you've cloned the repository within a server in the Kubernetes cluster, you can deploy the manifests directly using these commands (remember to initialize the infrastructure first).\n\n```bash\npython3.11 ./deploy.py                            # Deploy all manifest\npython3.11 ./deploy.py manifests/0x-manifest.yaml  # Deploy a specific service\n```\n\n## Troubleshooting\n\nIf you encounter issues during deployment:\n\n1. Check the logs of the deployment script\n2. Verify your environment variables in `.env`\n3. Ensure all prerequisites are properly installed\n4. Check your Kubernetes cluster's status and connectivity\n5. Verify storage system accessibility\n\nFor more detailed troubleshooting, consult the Kubernetes and Fleet documentation.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencitations%2Finfrastructure","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopencitations%2Finfrastructure","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencitations%2Finfrastructure/lists"}