{"id":26856695,"url":"https://github.com/mihaibc/PrivacyCopilot","last_synced_at":"2025-12-30T23:15:24.054Z","repository":{"id":285283697,"uuid":"957485203","full_name":"mihaibc/PrivacyPilot","owner":"mihaibc","description":"🛡️ PrivacyPilot – A privacy-centric backend API leveraging AI (LLaMA, Mistral, Ollama) to anonymize and moderate sensitive data in real-time. Built with Go, Node.js, and Perl. GDPR-compliant and infrastructure-ready.","archived":false,"fork":false,"pushed_at":"2025-03-30T19:54:32.000Z","size":15,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-30T20:24:42.074Z","etag":null,"topics":["ai","anonymization","azure-ai","backend","devops","gdpr","go","microservices","nodejs","ollama","perl5","privacy","stable-diffusion"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mihaibc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-30T13:49:28.000Z","updated_at":"2025-03-30T20:02:56.000Z","dependencies_parsed_at":"2025-03-30T20:35:15.609Z","dependency_job_id":null,"html_url":"https://github.com/mihaibc/PrivacyPilot","commit_stats":null,"previous_names":["mihaibc/privacypilot"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mihaibc%2FPrivacyPilot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mihaibc%2FPrivacyPilot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mihaibc%2FPrivacyPilot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mihaibc%2FPrivacyPilot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mihaibc","download_url":"https://codeload.github.com/mihaibc/PrivacyPilot/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246397404,"owners_count":20770549,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","anonymization","azure-ai","backend","devops","gdpr","go","microservices","nodejs","ollama","perl5","privacy","stable-diffusion"],"created_at":"2025-03-31T00:23:12.043Z","updated_at":"2025-12-30T23:15:24.047Z","avatar_url":"https://github.com/mihaibc.png","language":"Go","readme":"# 🛡️ Privacy Copilot – Private AI, RAG \u0026 Backend Showcase\n\n\n**Privacy Copilot** is an open-source, privacy-first platform that empowers individuals and organizations with a secure, local or cloud-deployable AI assistant for document Q\u0026A, retrieval-augmented generation (RAG), and private LLM fine-tuning—**without your data ever leaving your control**.\n\n**This project is also a comprehensive showcase**, demonstrating proficiency in:\n*   Hybrid backend \u0026 AI engineering (**Go + Python**) with best practices.\n*   Microservice and distributed systems architecture.\n*   Integration of modern AI (LLMs, RAG, vector databases, fine-tuning).\n*   MLOps, DevOps, cloud-native patterns, and CI/CD pipelines.\n*   End-to-end privacy engineering and compliance-aware design.\n\n---\n\n## 🌟 Key Features\n\n- ✅ **Private Document Q\u0026A \u0026 Retrieval-Augmented Generation**  \n  Upload documents and ask natural language questions—contextual answers are generated using Retrieval-Augmented Generation (RAG), all running locally or in your private cloud.\n\n- ✅ **Per-User Data Isolation \u0026 End-to-End Encryption**  \n  All data, embeddings, and models are encrypted at rest and in transit. Each user’s data is fully siloed for maximum privacy.\n\n- ✅ **Personalized LLM Fine-Tuning**  \n  (Optional) Fine-tune LLMs using your own documents or chat history, with all computation happening privately in your environment. Models are versioned and rollback-supported.\n\n- ✅ **Bring Your Own Model (BYOM)**  \n  Pluggable LLM support—use open-source models (Llama 3, Mistral, GPT-Neo, etc.) or connect your own model endpoints.\n\n- ✅ **Multi-Modal Search**  \n  Securely upload and search both text and images, powered by state-of-the-art embedding models.\n\n- ✅ **Privacy Controls Dashboard**  \n  Manage, export, or delete your data, review audit logs, and control your models—right from the UI.\n\n- ✅ **Modern MLOps \u0026 DevOps**  \n  Automated CI/CD, experiment tracking (MLflow), containerized deployment (Docker Compose, Kubernetes), and monitoring tools for both local and cloud setups.\n\n- ✅ **Compliance \u0026 Security by Design**  \n  Follows privacy-by-design principles (GDPR-aware), audit logging, and secure API access. No telemetry or external calls by default.\n\n---\n\n## 🎯 Showcase Goals\n\nThis project demonstrates advanced engineering in:\n\n*   **Hybrid Go + Python Architecture:**  \n    Go for performant backend API, user management, and privacy enforcement. Python for AI pipelines (LLMs, RAG, embeddings, fine-tuning).\n*   **Modular Microservices:**  \n    Clean separation between backend, AI services, vector DB, and front end—ready for local or cloud scaling.\n*   **Cloud-Native \u0026 On-Prem Deployments:**  \n    Easily run locally (for full privacy) or scale in your own cloud with Kubernetes/Terraform.\n*   **AI \u0026 MLOps Best Practices:**  \n    From RAG pipelines to model versioning, MLflow tracking, and experiment management.\n*   **Privacy \u0026 Security Engineering:**  \n    User isolation, encryption, detailed audit trails, BYOM for ultimate control.\n*   **Enterprise-Ready Patterns:**  \n    Role-based access (planned), OAuth2/OIDC support (planned), and compliance-aware architecture.\n\n---\n\n## 🛠️ Tech Stack\n\n| Category                 | Technologies Used                                                                           |\n| :----------------------- | :------------------------------------------------------------------------------------------ |\n| **Architecture**         | Microservices, REST APIs                                                                    |\n| **Backend Languages**    | Go (API Gateway, user/session management), Python (AI/RAG, LLMs, embeddings)                |\n| **AI/ML**                | HuggingFace, LangChain, FastAPI, MLflow, ChromaDB/FAISS/Qdrant (vector DB)                  |\n| **Frontend**             | React or Streamlit (privacy dashboard \u0026 chat UI)                                            |\n| **Databases**            | PostgreSQL (user data/audit logs), Vector DB (per-user embeddings), Encrypted Storage       |\n| **Containerization**     | Docker, Docker Compose                                                                      |\n| **Orchestration**        | Kubernetes/Helm (Cloud), Terraform (Infra as Code, planned)                                 |\n| **CI/CD**                | GitHub Actions                                                                              |\n| **Observability**        | Prometheus, Grafana, Jaeger                                                                 |\n| **Security**             | End-to-end encryption, OAuth2/OIDC (planned), audit logging                                 |\n\n---\n\n## 🚀 Getting Started (Local Development)\n\n### 📋 Prerequisites\n\n1.  **Git:** [Install Git](https://git-scm.com/downloads).\n2.  **Docker:** [Install Docker Desktop](https://docs.docker.com/get-docker/). Docker Compose required.\n3.  **Python (optional):** For development/debugging AI service outside Docker.\n4.  **Go:** [Install Go](https://go.dev/doc/install) (for backend development).\n5.  **(Optional) Ollama:** For running certain open-source LLMs locally, see [Ollama](https://ollama.com/).\n6.  **(Optional) jq:** JSON CLI tool for testing API responses.\n\n### ⚙️ Installation \u0026 Setup\n\n1.  **Clone the Repository:**\n    ```bash\n    git clone https://github.com/\u003cyour-username\u003e/privacy-copilot.git\n    cd privacy-copilot\n    ```\n\n2.  **Copy \u0026 Edit Environment Variables:**\n    ```bash\n    cp .env.example .env\n    # Edit .env as needed for DB, AI model paths, ports, etc.\n    ```\n\n3.  **Start the Stack:**\n    ```bash\n    docker-compose up --build -d\n    ```\n\n    - This starts the Go API gateway, Python AI service, database(s), and vector DB.\n    - Default UI at: `http://localhost:8080`\n\n4.  **(Optional) Run Ollama and Download a Model:**\n    ```bash\n    ollama pull llama3\n    ```\n\n5.  **Access Logs and Monitor:**\n    ```bash\n    docker-compose logs -f\n    ```\n\n---\n\n## 🧪 Testing the Platform\n\nTry out the REST API (see API docs) or use the web UI:\n\n- **Upload documents**\n- **Ask questions (“What is the main idea of this document?”)**\n- **Export or delete your data from the privacy dashboard**\n- **(Advanced) Launch a model fine-tuning job from the dashboard or via API**\n\n---\n\n## 📚 Project Documentation\n\n- [Contribution Guidelines](CONTRIBUTING.md)\n- [API Reference](docs/api.md)\n- [Architecture \u0026 Security](docs/architecture.md)\n- [Deployment (Local/Cloud)](docs/deployment.md)\n- [BYOM: Bring Your Own Model](docs/models.md)\n\n---\n\n## 🤝 Contributing\n\nWe welcome community contributions! Please review [CONTRIBUTING.md](CONTRIBUTING.md) and link all PRs to relevant issues.\n\n---\n\n## 🏗️ Project Structure Overview\n\n```text\nprivacy-copilot/\n├── backend/              # Go API gateway\n├── ai_service/           # Python FastAPI RAG/LLM service\n├── frontend/             # React or Streamlit UI\n├── infra/                # Helm charts, Terraform scripts\n├── mlops/                # MLflow configs, pipelines, experiment tracking\n├── docs/                 # Documentation \u0026 API specs\n├── docker-compose.yaml   # Local stack orchestration\n└── ...                   # Standard configs (LICENSE, .gitignore, etc.)\n```\n\n⸻\n\n📫 Contact \u0026 Commercial Use\n\n\nPersonal, research, and educational use is free.\nCommercial use requires a separate license—please contact:\nevana.blanche.privacycopilot@gmail.com\n\nFor questions, suggestions, or support:\n  •\tOpen an issue\n\n⸻\n\n⚖️ License\n\n\nThis project is licensed for non-commercial use only.\nCommercial, SaaS, or enterprise deployments require written permission.\nSee LICENSE for full terms.\n\n⸻\n\n🙌 Acknowledgments\n  •\tThanks to the open-source and privacy communities for inspiration and support.\n\n⸻\n\n\nBuilt for privacy, flexibility, and as a modern AI/ML engineering showcase.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmihaibc%2FPrivacyCopilot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmihaibc%2FPrivacyCopilot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmihaibc%2FPrivacyCopilot/lists"}