{"id":31912229,"url":"https://github.com/correlaid/parrotpark","last_synced_at":"2026-02-19T17:31:12.858Z","repository":{"id":318093377,"uuid":"901876097","full_name":"CorrelAid/parrotpark","owner":"CorrelAid","description":null,"archived":false,"fork":false,"pushed_at":"2025-07-21T10:44:19.000Z","size":2356,"stargazers_count":0,"open_issues_count":7,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-01-19T00:27:15.891Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CorrelAid.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-12-11T13:33:56.000Z","updated_at":"2025-09-24T16:00:21.000Z","dependencies_parsed_at":"2025-10-31T04:00:18.262Z","dependency_job_id":null,"html_url":"https://github.com/CorrelAid/parrotpark","commit_stats":null,"previous_names":["correlaid/parrotpark"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/CorrelAid/parrotpark","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CorrelAid%2Fparrotpark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CorrelAid%2Fparrotpark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CorrelAid%2Fparrotpark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CorrelAid%2Fparrotpark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CorrelAid","download_url":"https://codeload.github.com/CorrelAid/parrotpark/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CorrelAid%2Fparrotpark/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29624493,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-19T13:04:20.082Z","status":"ssl_error","status_checked_at":"2026-02-19T13:03:33.775Z","response_time":117,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-13T17:59:34.416Z","updated_at":"2026-02-19T17:31:12.841Z","avatar_url":"https://github.com/CorrelAid.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Parrotpark\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\n      \u003cimg src=\"assets/image.png\" alt=\"alt text\"/\u003e\n    \u003c/td\u003e\n    \u003ctd\u003e\n      \u003cp\u003eParrotPark is a Infrastructure as Code solution for self-hosting a LLM assistant with all required components, including inference servers and a chat interface. It was developed by CorrelAid with the support of \u003ca href=\"https://d-64.org/\"\u003eD64\u003c/a\u003e. This project is currently not being worked on and ended with an evaluation (see evaluation directory).\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\n\n## Hosted (Exposed) Services on Server\n\n- **LiteLLM**: [api.parrotpark.correlaid.org](https://api.parrotpark.correlaid.org)\n  - Admin UI: [api.parrotpark.correlaid.org/ui](https://api.parrotpark.correlaid.org/ui)\n  - API Doc: [api.parrotpark.correlaid.org/docs](https://api.parrotpark.correlaid.org/docs)\n- **LibreChat**: [parrotpark.correlaid.org](https://parrotpark.correlaid.org)\n\n## User Documentation\n\n- Find Instructions on how to use Parrotpark [here](https://correlaid.gitbook.io/parrotpark-docs)\n\n## Architecture Description\n\n### High Level Overview\n\n```mermaid\n---\nconfig:\n  flowchart:\n    htmlLabels: false\n---\n%%{ init: { 'flowchart': { 'curve': '' } } }%%\nflowchart LR\n\n    subgraph proxy[\"`**Entrance Server**`\"]\n    direction TB\n      Caddy \n      Databases\n      Scheduler\n    end\n    subgraph gpu[\"`**Ephemeral GPU Server**`\"]\n    direction TB\n      vLLM\n      LiteLLM\n      LibreChat\n    end \n      LibreChat ---|SSO Auth| Keycloak\n      \n      buckets[\"`S3 Buckets`\"]\n      LibreChat ---|Storage| buckets\n   \n    LiteLLM \u0026 LibreChat --- Databases \n    Caddy --\u003e|Proxy| LiteLLM \u0026 LibreChat\n    Scheduler --\u003e|Creates Periodically| gpu\n\n```\n### Networking\n\n- Across servers, the services are connected via a netbird VPN\n- SSH access is restricted to the VPN\n- Internal networking is configured with docker networks\n\n### Metrics\n\n- The entrance server additionally runs a telegraf agent for scraping metrics from services and host systems\n- The GPU servers hosts a nvidia smi exporter container for scraping metrics from the GPU (nvidia smi)\n- Metrics are sent to a timescale database, to which a metabase instance has access to\n\n### Scheduling\n\n- The scheduler is a python script that runs on the entrance server. It executes opentofu and ansible commands to create and destroy the GPU server. It is packaged as a systemd service.\n\n\n## IaC Setup\n\n- IaC code refers to [OpenTofu](https://opentofu.org/) and [Ansible](https://www.ansible.com/) scripts.\n- This is a nested Infrastructure as Code project, because the initial IaC script creates an entrance server which will contain IaC code to automatically create a second ephemeral GPU server.\n\n### Requirements \n\n#### External SaaS/Cloud Provider services\n\nBecause this project grew dynamically, infrastructure is a bit all over the place (spread across multiple cloud providers), Theoretically, as long as its the same type of service/has the same functionality, most components can be swapped out. For example hetzner has object storage (buckets) as well, you could use [Tailscale](https://tailscale.com/) instead of netbird or Azure Entra instead of keycloak.\n\n- [Hetzner](https://www.hetzner.com/) account and Cloud project\n  - A domain in the Hetzner DNS console\n  - A mailbox in a Hetzner KonsoleH webhosting account\n\n- [Digital Ocean Account](https://www.digitalocean.com/) account for LibreChat asset and OpenTofu state storage\n- [Scaleway](https://www.scaleway.com/en/) account with access to creating L4 GPU instances (if you open an account you will have to write the support to request access)\n\n- [Netbird](https://netbird.io/) for connection via VPN. \n    - Set up one or multple [Groups](https://docs.netbird.io/how-to/manage-network-access) with access configured so that you can connect to the servers via SSH and the servers can connect to each other. \n-  [Keycloak](https://www.keycloak.org/) instance with a configured realm, e.g. on [cloud-iam](https://www.cloud-iam.com/), for user management\n- A domain configured on [Hetzner DNS](https://www.hetzner.com/dns-console/) \n- [Infisical](https://infisical.com/) instance for secret management for the IaC code.\n  - Contains secrests for accessing some of the existing infrastructure programatically \n  - Is used by the IaC code to create managed secrets that can be read automatically at other places in the code\n\n#### Software Requirements\n\n- [Netbird Client](https://app.netbird.io/install) \n- [OpenTofu](https://opentofu.org/)\n- [uv](https://docs.astral.sh/uv/) \n\n## IaC Setup\n\n1. `uv sync --all-groups` to install other dependencies\n\n2. Set up pre-commit\n    ```\n    uv run pre-commit install\n    ```\n\n3. Install ansible requirements\n    ```\n    uv run ansible-galaxy install -r ansible/requirements.yml\n    ```\n4. When you were added as a peer to the required Netbird Organisation, run `netbird up`\n\n5. Create a new set of SSH keys and adjust the entrance_server_settings.public_ssh_key variable in the `opentofu/variables.tf` file. Also adjust ansible/files/ansible.cfg depending on where you saved the private key.\n\n6. Adjust netbird_vps_group, infisical_workspace_id an  scaleway_project variables in the `opentofu/variables.tf` file.\n\n7. Manually create a bucket on scaleway and adjust the s3 information in `opentofu/meta.tf`. Create access credentials and save it as described in the next step.\n\n8. In your infisical project, create all vars you see at the top of `ansible/playbook.yml` and in the nested IaC in `ansible/files/ansible/group_vars/unmanaged.yml`\n\n9. Save these environemtn variables somewhere for easy copy and paste (but do not locally hard code them):\n    ```\n    export AWS_ACCESS_KEY_ID=\"\"\n    export AWS_SECRET_ACCESS_KEY=\"\"\n    export TF_VAR_infisical_client_secret=\"\" \n    #### MAKE SURE YOU HAVE DISABLED HISTORY FOR THESE VARS ###\n    ```\n10. Decide wether you want to scrape metrics with telegraf. Metric visualisation and alerting is not part of this IaC project, but you can set up a telegraf agent that can send metrics to a TimeScale DB. If you do not want this, just do not include the `deploy_telegraf.yml` in `ansible/playbook.yml`. If you do, you need to have a TimeScale DB available and adjust the vars in the main playbook.\n\n11. For the Keycloak Setup, follow this tutorial: https://www.librechat.ai/docs/configuration/authentication/OAuth2-OIDC/keycloak\n\n12. To initialize the OpenTofu backend, run `tofu init`\n\n## Running the IaC code\n\n- For both OpenTofu and Ansible, you need to have the environment variables set as described in 10. \n\n### OpenTofu\n\n- While in the `opentofu` directory, run `tofu apply` to create the entrance server\n\n### Ansible\n\n- Because Ansible requires SSH access to the servers, you need to have the Netbird client running\n- Run ansible with: `uv run ansible-playbook ansible/playbook.yml -vv`\n\n\n### Manually running the nested IaC code\n\nWhile the scheduler takes care of creating the GPU instances as specified in `scripts/scheduler.py`, you can also manually run the nested IaC code. \n\nFor this, you first need to SSH into the entrance server, have the same environment variables set as described in 9. and while in the `/home/correlaid/scheduler/opentofu` directory, run `tofu apply`, followed by 'ansible-playbook ansible/playbook.yml -vv' while in the `/home/correlaid/scheduler` directory.\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcorrelaid%2Fparrotpark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcorrelaid%2Fparrotpark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcorrelaid%2Fparrotpark/lists"}