{"id":28504678,"url":"https://github.com/controlplaneio/ai-security-architecture","last_synced_at":"2025-07-06T04:30:49.954Z","repository":{"id":282016059,"uuid":"945952706","full_name":"controlplaneio/ai-security-architecture","owner":"controlplaneio","description":"Reference security architecture for AI applications","archived":false,"fork":false,"pushed_at":"2025-03-28T21:38:19.000Z","size":42,"stargazers_count":13,"open_issues_count":2,"forks_count":4,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-06-08T18:47:44.882Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/controlplaneio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-10T11:39:04.000Z","updated_at":"2025-03-28T23:03:47.000Z","dependencies_parsed_at":"2025-03-19T16:33:17.850Z","dependency_job_id":null,"html_url":"https://github.com/controlplaneio/ai-security-architecture","commit_stats":null,"previous_names":["controlplaneio/ai-security-architecture"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/controlplaneio/ai-security-architecture","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/controlplaneio%2Fai-security-architecture","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/controlplaneio%2Fai-security-architecture/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/controlplaneio%2Fai-security-architecture/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/controlplaneio%2Fai-security-architecture/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/controlplaneio","download_url":"https://codeload.github.com/controlplaneio/ai-security-architecture/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/controlplaneio%2Fai-security-architecture/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263848668,"owners_count":23519491,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-08T18:30:29.767Z","updated_at":"2025-07-06T04:30:49.947Z","avatar_url":"https://github.com/controlplaneio.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AI Reference Architecture Demo\n\n## Prerequisites\n\nThe following client tools are needed to run this demo:\n\n- [Docker](https://www.docker.com/)\n- [Kind](https://kind.sigs.k8s.io/)\n- [kubectl](https://kubernetes.io/docs/reference/kubectl/)\n- [Helm](https://helm.sh/)\n\nAn [OpenAI](https://platform.openai.com/) account and API Key are needed, with a Project with some credits.\n\n## Architecture\n\nThe following diagram shows the demo architecture.\n\n```mermaid\ngraph LR;\n client([client])-. port forward .-\u003efw-prompt-svc[Service: fw-prompt];\n\n subgraph cluster\n\n   subgraph \"fw-prompt\"\n     fw-prompt-svc--\u003efw-prompt-deploy[\"Deployment: envoy-proxy(fw-prompt)\"];\n   end\n\n   subgraph \"ctrl-prompt\"\n     fw-prompt-deploy--\u003ectrl-prompt-svc[Service: ctrl-prompt];\n     ctrl-prompt-svc--\u003ectrl-prompt-deploy[\"Deployment: envoy-proxy (ctrl-prompt)\"];\n     ctrl-prompt-deploy--\u003efw-prompt-deploy\n   end\n\n   subgraph \"app-chatbot\"\n     fw-prompt-deploy--\u003eapp-chatbot-svc[Service: app-chatbot];\n     app-chatbot-svc--\u003eapp-chatbot-deploy[\"Deployment: app-chatbot (app-chatbot)\"];\n   end\n\n   subgraph \"app-chatbot-vectordb\"\n     app-chatbot-deploy--\u003eapp-chatbot-vectordb-svc[Service: app-chatbot-vectordb];\n     app-chatbot-vectordb-svc--\u003eapp-chatbot-vectordb-deploy[\"Deployment: app-chatbot-vectordb (app-chatbot)\"];\n     app-chatbot-vectordb-deploy--\u003eapp-chatbot-deploy\n   end\n\n   subgraph \"app-chatbot-confluence\"\n     app-chatbot-deploy--\u003eapp-chatbot-confluence-svc[Service: app-chatbot-confluence];\n     app-chatbot-confluence-svc--\u003eapp-chatbot-confluence-deploy[\"Deployment: app-chatbot-confluence (app-chatbot)\"];\n     app-chatbot-confluence-deploy--\u003eapp-chatbot-deploy\n   end\n\n   subgraph \"fw-model\"\n     app-chatbot-deploy--\u003efw-model-svc[Service: fw-model];\n     fw-model-svc--\u003eenvoy-proxy-fw-model[\"Deployment: envoy-proxy (fw-model)\"];\n   end\n\n   subgraph \"ctrl-model\"\n     envoy-proxy-fw-model--\u003ectrl-model-svc[Service: ctrl-model];\n     ctrl-model-svc--\u003ectrl-model-deploy[\"Deployment: envoy-proxy (ctrl-model)\"];\n     ctrl-model-deploy--\u003eenvoy-proxy-fw-model\n   end\n\n end\n\n  envoy-proxy-fw-model--\u003eexternal-inference[\"External Inference\"];\n\n classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000;\n  classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff;\n  classDef k8s-controls fill:#000,stroke:#fff,stroke-width:4px,color:#fff;\n  classDef k8s-data fill:#964500,stroke:#fff,stroke-width:4px,color:#fff;\n classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5;\n  class ingress,app-chatbot-svc,app-chatbot-deploy,envoy-proxy-fw-model,fw-model-svc,fw-model-deploy,envoy-proxy-fw-prompt,fw-prompt-svc,fw-prompt-deploy k8s;\n  class ctrl-model-deploy,ctrl-model-svc,ctrl-model,ctrl-prompt-deploy,ctrl-prompt-svc,ctrl-prompt k8s-controls;\n  class app-chatbot-confluence-svc,app-chatbot-confluence-deploy,app-chatbot-vectordb-svc,app-chatbot-vectordb-deploy k8s-data;\n class client plain;\n class cluster cluster;\n```\n\n\u003e [!NOTE]\n\u003e This project is built to demo some of the components of the [FINOS AI Governance Framework](https://github.com/finos/ai-readiness).\n\n\n## Demo\n\nIn this demo, placeholder Envoy proxies have been introduced for the prompt firewall and model firewall, which log requests and responses. The role of the AI-enabled application is played by [aichat](https://github.com/sigoden/aichat), which forwards on requests to OpenAI via the model firewall.\n\nSecurity contexts for the proxies and `aichat` have been hardened, and the `fw-prompt`, `app-chatbot` and `fw-model` namespaces have Pod Security Standards enforced at the Restricted level. Cilium is used as the CNI, and network policies have been set up so that inbound traffic to `aichat` must come from the `fw-prompt` namespace, and egress traffic must go to the `fw-model` namespace.\n\n1. Set the `OPENAI_API_KEY` environment variable:\n\n```bash\nexport OPENAI_API_KEY=\u003cPaste Your API Key Here\u003e\n```\n\n2. **Install** and **activate** [mise](https://mise.jdx.dev/) following the instructions for your workstation [here](https://mise.jdx.dev/getting-started.html).\n\n3. Use `mise` to install the **required** CLI tools:\n\n    ```sh\n    mise trust\n    mise install\n    mise run deps\n    ```\n\n4. Spin up the infrastructure:\n\n```bash\nmake all\n```\n\n5. Watch the logs for evidence of startup, for example llm-guard needs to pull models from Hugging Face:\n\nDepending on the first-run conditions, you may need to run `make test-prompt` until the environment is prepared.\n\n```bash\n$ kubectl logs -n ctrl-prompt -l app=llm-guard\n{\"model\": \"Model(path='unitary/unbiased-toxic-roberta', subfolder='', revision='36295dd80b422\ndc49f40052021430dae76241adc', onnx_path='ProtectAI/unbiased-toxic-roberta-onnx', onnx_revision='34480fa958f6657ad835c345808475755b6974a7', onn\nx_subfolder='', onnx_filename='model.onnx', kwargs={}, pipeline_kwargs={'batch_size': 1, 'device': device(type='cpu'), 'padding': 'max_length'\n, 'top_k': None, 'function_to_apply': 'sigmoid', 'return_token_type_ids': False, 'max_length': 256, 'truncation': True}, tokenizer_kwargs={})\"\n, \"device\": \"device(type='cpu')\", \"event\": \"Initialized classification ONNX model\", \"level\": \"debug\", \"timestamp\": \"2024-10-29T07:16:18.009892\nZ\"}\n```\n\n6. After the models are ready:\n\n```bash\nmake netpols-apply\n```\n\n7. Send an example passing request:\n\n```bash\nmake test-prompt\n```\n\n\u003e [!NOTE]\n\u003e This step should complete successfully. Monitor the logs to check for model reconcilliation activity\n\n8. Send an example failing request:\n\n```bash\nmake test-prompt-fail\n```\n\nThis step should fail with something like `{\"is_valid\":false,\"scanners\":{\"BanTopics\":1.0}}`\n\n9. This infrastructure is now ready for red-teaming!\n\n```bash\npython3 -m garak  --model_type rest -G garak-rest.json --probes dan.DanInTheWildMini\n```\n\n## Teardown\n\n```bash\nmake down\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcontrolplaneio%2Fai-security-architecture","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcontrolplaneio%2Fai-security-architecture","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcontrolplaneio%2Fai-security-architecture/lists"}