{"id":29640255,"url":"https://github.com/philodi-dev/rust-production-ai-assistant-app","last_synced_at":"2026-04-09T07:02:26.617Z","repository":{"id":304522325,"uuid":"1019023837","full_name":"philodi-dev/rust-production-ai-assistant-app","owner":"philodi-dev","description":"Resilient AI Agent is a full-stack, production-ready AI Doctor Assistant platform. It features a Rust backend (Axum, MongoDB, Ollama with Llama3 and other models, circuit breaker, retry, logging), a modern Next.js frontend (Tailwind, shadcn/ui), and a Python-based TTS microservice.","archived":false,"fork":false,"pushed_at":"2025-07-13T17:20:31.000Z","size":118,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-13T17:39:41.160Z","etag":null,"topics":["flask","llama3","nestjs","nextjs15","ollama","rust","rust-ai","rust-ai-agent","tts","tts-api"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/philodi-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-13T15:15:35.000Z","updated_at":"2025-07-13T17:20:34.000Z","dependencies_parsed_at":"2025-07-13T17:39:45.789Z","dependency_job_id":"1eece10f-746c-407c-bc57-52d82b4cfb45","html_url":"https://github.com/philodi-dev/rust-production-ai-assistant-app","commit_stats":null,"previous_names":["philodi-dev/rust-production-ai-assistant-app"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/philodi-dev/rust-production-ai-assistant-app","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/philodi-dev%2Frust-production-ai-assistant-app","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/philodi-dev%2Frust-production-ai-assistant-app/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/philodi-dev%2Frust-production-ai-assistant-app/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/philodi-dev%2Frust-production-ai-assistant-app/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/philodi-dev","download_url":"https://codeload.github.com/philodi-dev/rust-production-ai-assistant-app/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/philodi-dev%2Frust-production-ai-assistant-app/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266379169,"owners_count":23920156,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-21T11:47:31.412Z","response_time":64,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["flask","llama3","nestjs","nextjs15","ollama","rust","rust-ai","rust-ai-agent","tts","tts-api"],"created_at":"2025-07-21T21:01:54.385Z","updated_at":"2025-12-30T22:07:00.946Z","avatar_url":"https://github.com/philodi-dev.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Resilient AI Agent\n\nResilient AI Agent is a full-stack, production-ready AI Doctor Assistant platform. It features a Rust backend (Axum, MongoDB, Ollama with Llama3 and other models, circuit breaker, retry, logging), a modern Next.js frontend (Tailwind, shadcn/ui), and a Python-based TTS microservice. The system is robust, modular, and cloud-native, with Docker and Kubernetes support for scalable, secure deployments. It is designed for LAN and cloud, with browser and server TTS fallback, and a clean, accessible chat UI.\n\n## Project Structure\n\n```\nresilient-ai-agent/\n├── backend/      # Rust backend (Axum, MongoDB, Ollama, etc.)\n├── frontend/     # Next.js frontend (React, Tailwind, shadcn/ui)\n├── tts/          # Python TTS microservice (pyttsx3, Flask)\n├── k8s/          # Kubernetes manifests\n├── examples/     # Rust example code\n└── README.md     # Project documentation\n```\n\n## How to Build \u0026 Run\n\n### Backend\n```\ncd backend\ncargo build\ncargo run\n```\n\n### Frontend\n```\ncd frontend\nnpm install\nnpm run dev\n```\n\n### TTS Service\n```\ncd tts\n./start_tts_server.sh\n```\n\n### Kubernetes\nAll manifests are in the `k8s/` folder.\n\n---\n\n## Docker Compose Deployment\n\nYou can run the entire stack (backend, frontend, tts, mongo) using Docker Compose:\n\n```\ncd backend\ndocker-compose up --build\n```\n\n- **backend**: Rust API (Axum, MongoDB, Ollama proxy)\n- **frontend**: Next.js UI\n- **tts**: Python TTS microservice (Flask, pyttsx3)\n- **mongo**: MongoDB database\n\nAll services are networked together. The backend and frontend containers will wait for TTS and MongoDB to be ready.\n\n### Accessing Services\n- Frontend: http://localhost:3000\n- Backend API: http://localhost:3001\n- TTS API: http://localhost:5002\n- MongoDB: localhost:27017 (for development)\n\n---\n\n## Kubernetes Deployment\n\nAll Kubernetes manifests are in the `k8s/` directory. To deploy the stack:\n\n```\nkubectl apply -f k8s/\n```\n\n- **backend-deployment.yaml**: Rust API deployment, service, and secrets\n- **frontend-deployment.yaml**: Next.js UI deployment and service\n- **tts-deployment.yaml**: Python TTS deployment and service\n- **ingress.yaml**: (Optional) Ingress for routing external traffic\n\n### Accessing Services in Kubernetes\n- Use `kubectl port-forward` or configure an ingress controller to access services from outside the cluster.\n- Example:\n  ```\n  kubectl port-forward svc/resilient-ai-backend 3001:3001\n  kubectl port-forward svc/frontend 3000:3000\n  kubectl port-forward svc/tts 5002:5002\n  ```\n\n---\n\n## Environment Variables\n\n- **Backend**:\n  - `MONGODB_URI` (default: `mongodb://mongo:27017`)\n  - `TTS_URL` (default: `http://tts:5002`)\n- **Frontend**:\n  - `NEXT_PUBLIC_BACKEND_URL` (default: `http://backend:3001`)\n  - `NEXT_PUBLIC_TTS_URL` (default: `http://tts:5002`)\n- **TTS**:\n  - No special env vars required (runs on port 5002)\n\n---\n\n## Notes\n- All build artifacts for Rust are kept in `backend/target/`.\n- Each service has its own Dockerfile and can be built/deployed independently.\n- For production, update image tags and ingress as needed.\n- For local development, you can run each service individually or use Docker Compose.\n\n---\n\n## Features\n\n### Backend (Rust)\n- **Async Rust**: Built with `tokio` for high performance and concurrency.\n- **Ollama Integration**: Uses [`ollama-rs`](https://crates.io/crates/ollama-rs) to interact with local Ollama models.\n- **Retry with Exponential Backoff**: Automatically retries failed requests with configurable backoff.\n- **Circuit Breaker**: Prevents repeated failures from overwhelming the system.\n- **Structured Logging**: Uses `env_logger` and `log` for clear, timestamped logs.\n- **Model Availability Check**: Verifies the requested model is available before making requests.\n- **REST API**: `/api/chat` for chat, `/api/health` for health check.\n\n### Frontend (Next.js)\n- **Next.js 14+** with App Router, TypeScript, and Tailwind CSS ([official guide](https://tailwindcss.com/docs/installation/framework-guides/nextjs))\n- **shadcn/ui** for beautiful, accessible UI components ([docs](https://ui.shadcn.com/docs/installation))\n- **Modern chat interface**: Responsive, mobile-friendly, and visually appealing\n- **Live connection status**: Shows if backend/model is available\n- **Avatar, Card, Input, Button, ScrollArea**: All UI built with shadcn/ui\n\n---\n\n## UI/UX Improvements\n\n- The chat input is now always visible and fixed at the bottom of the page, just like a real chat app (no need to scroll to reach it).\n- All footers and branding have been removed for a clean, app-like experience.\n\n---\n\n## Getting Started\n\n### 1. Backend (Rust)\n\n```sh\n# In the project root\ncargo run\n```\n- The API will be available at `http://localhost:3001`\n- Endpoints:\n  - `POST /api/chat` — send chat messages\n  - `GET /api/health` — check model/backend status\n\n**Requirements:**\n- Rust 1.70+\n- [Ollama](https://ollama.com/) running locally\n- The desired model (default: `mistral`) installed:\n  ```sh\n  ollama pull mistral\n  ```\n\n### 2. Frontend (Next.js)\n\n```sh\ncd frontend\nnpm install\nnpm run dev\n```\n- The app will be available at `http://localhost:3000`\n- The frontend will communicate with the backend at `http://localhost:3001`\n\n---\n\n## Example Usage\n\n- Ask health questions in the chat UI\n- The AI doctor will respond using the Ollama model\n- Connection status is shown at the top of the chat\n\n---\n\n## Customization\n- Change the model in `src/lib.rs` (`MODEL` constant) or pass a different prompt/system prompt.\n- Adjust retry/circuit breaker settings in `src/client.rs` or via the respective config structs.\n- Tweak the chat UI in `frontend/src/components/chat/` for your brand/style.\n\n---\n\n## Deployment\n\n### Docker Compose (Recommended for Local/Dev)\n\n1. Build and run both backend and frontend:\n\n```sh\ndocker compose up --build\n```\n\n2. Access the frontend at `http://localhost:3000` and backend at `http://localhost:3001`.\n\n### Docker (Manual)\n\n#### Backend\n```sh\ndocker build -t resilient-ai-backend .\ndocker run -p 3001:3001 resilient-ai-backend\n```\n\n#### Frontend\n```sh\ncd frontend\ndocker build -t resilient-ai-frontend .\ndocker run -p 3000:3000 resilient-ai-frontend\n```\n\n### Kubernetes\n\n1. Apply manifests in the `k8s/` directory:\n```sh\nkubectl apply -f k8s/\n```\n2. Expose services as needed (e.g., via LoadBalancer or Ingress).\n\n---\n\n## Production Environment \u0026 Secrets\n\n### Backend (Rust)\n- Configure secrets (e.g., Ollama API key) via environment variables.\n- In Kubernetes, use a Secret (see k8s/backend-deployment.yaml) and reference it in the deployment.\n- In Docker, use `-e OLLAMA_API_KEY=...` or `--env-file`.\n\n### Frontend (Next.js)\n- Set the backend API URL via `NEXT_PUBLIC_API_URL`.\n- In Kubernetes, set as an env var in the deployment.\n- In Docker, use `-e NEXT_PUBLIC_API_URL=...` or `--env-file`.\n\n### Example: Kubernetes Secret\n```yaml\napiVersion: v1\nkind: Secret\nmetadata:\n  name: ollama-secrets\nstringData:\n  api-key: \"your-ollama-api-key-here\"\n```\n\n---\n\n## API Endpoints\n\n### Backend (Rust)\n\n- `POST /api/chat`\n  - **Request:**\n    ```json\n    { \"message\": \"What is hypertension?\" }\n    ```\n  - **Response:**\n    ```json\n    { \"response\": \"Hypertension is...\", \"success\": true }\n    ```\n\n- `GET /api/chats`\n  - **Response:**\n    ```json\n    [ { \"id\": \"...\", \"content\": \"...\", \"role\": \"user|assistant\", \"timestamp\": \"...\" }, ... ]\n    ```\n\n- `GET /api/health`\n  - **Response:**\n    ```json\n    { \"status\": \"ok\", \"model\": \"mistral\", ... }\n    ```\n\n- `GET /api/tts/voices`\n  - **Response:**\n    ```json\n    [\"Samantha\", \"Alex\", \"Daniel\", ...]\n    ```\n\n- `POST /api/tts`\n  - **Request:**\n    ```json\n    { \"text\": \"Hello, this is a test.\", \"voice\": \"Samantha\" }\n    ```\n  - **Response:**\n    - On success: WAV audio file\n    - On error: `{ \"error\": \"TTS server unavailable\" }` (Content-Type: application/json)\n\n---\n\n## TTS Fallback \u0026 Robustness\n\n- If the backend cannot reach the TTS server, it returns a JSON error. The frontend automatically falls back to browser TTS (using the Web Speech API) so users always hear responses.\n- TTS settings (enable/disable, mute, select voice) are available in the frontend settings page.\n\n---\n\n## Types of Tests\n\n- **Unit Tests:**\n  - Test individual functions, methods, or small modules in isolation.\n  - Fast, do not require external systems (e.g., database, network).\n  - Example: serialization/deserialization tests for types, logic in a single function.\n\n- **Integration Tests:**\n  - Test how multiple parts of the system work together.\n  - May require external systems (e.g., database, TTS server).\n  - Example: API endpoint tests that hit the running backend and check real responses.\n\n- **Functional Tests:**\n  - Test a specific feature or user-facing functionality end-to-end.\n  - May span multiple modules or services, but focus on a user story or workflow.\n  - Example: Simulate a user asking a question and receiving a spoken answer.\n\n- **Smoke Tests:**\n  - Basic tests to check that the most important parts of the system work at all.\n  - Often run after deployment to catch major breakage quickly.\n  - Example: Check that the backend starts and responds to `/api/health`.\n\n---\n\n## Running Tests\n\n- **All tests:**\n  ```sh\n  cd backend\n  cargo test\n  ```\n  This will run all unit, integration, functional, and smoke tests in `backend/tests/`.\n\n- **Run a specific test file:**\n  ```sh\n  cargo test --test \u003ctest_file_name\u003e\n  # Example:\n  cargo test --test types\n  cargo test --test integration_api\n  ```\n\n- **Test file organization:**\n  - All test files are in `backend/tests/`.\n  - Each `.rs` file is a separate test crate (e.g., `types.rs`, `integration_api.rs`).\n  - Add new functional or smoke tests as new files (e.g., `functional_login.rs`, `smoke_health.rs`).\n\n---\n\n## Credits\n- [Ollama](https://ollama.com/)\n- [Next.js](https://nextjs.org/)\n- [Tailwind CSS](https://tailwindcss.com/)\n- [shadcn/ui](https://ui.shadcn.com/)\n\n---\n\n## Kubernetes Production Deployment (AWS/EKS Example)\n\nThe `k8s/` directory now contains manifests for a full production deployment:\n\n- `mongo-statefulset.yaml`: MongoDB StatefulSet with persistent EBS storage and secret\n- `ollama-deployment.yaml`: Ollama model server with persistent EBS storage and secret (preloads and serves Llama3 model automatically)\n- `backend-deployment.yaml`: Rust backend API\n- `frontend-deployment.yaml`: Next.js frontend\n- `tts-deployment.yaml`: Python TTS microservice\n- `ingress.yaml`: Ingress for HTTPS routing (uses cert-manager and nginx)\n\n### Prerequisites\n- An EKS (or other Kubernetes) cluster with storage class `gp2` (EBS)\n- [kubectl](https://kubernetes.io/docs/tasks/tools/) configured for your cluster\n- [cert-manager](https://cert-manager.io/) and [nginx ingress controller](https://kubernetes.github.io/ingress-nginx/) installed (for HTTPS)\n\n### Deploy the Full Stack\n```sh\nkubectl apply -f k8s/mongo-statefulset.yaml\nkubectl apply -f k8s/ollama-deployment.yaml\nkubectl apply -f k8s/backend-deployment.yaml\nkubectl apply -f k8s/tts-deployment.yaml\nkubectl apply -f k8s/frontend-deployment.yaml\nkubectl apply -f k8s/ingress.yaml\n```\n\n- All persistent data (MongoDB, Ollama models) will be stored on EBS volumes.\n- Secrets for database and model API keys are managed as Kubernetes secrets.\n- Ingress provides HTTPS access to the frontend and API (edit `ingress.yaml` for your domain).\n\n### Accessing Services\n- **Frontend:** https://ai.example.com/\n- **Backend API:** https://ai.example.com/api\n- **TTS API:** internal service `tts:5002`\n- **MongoDB:** internal service `mongo:27017`\n- **Ollama:** internal service `ollama:11434`\n\n---\n\n## Roadmap: Security Features\n\nThe following security features are planned or in progress to ensure the safety, privacy, and robustness of the Resilient AI Agent platform:\n\n- **Authentication \u0026 Authorization**\n  - Add user authentication (OAuth2, JWT, or SSO integration)\n  - Role-based access control for sensitive endpoints (e.g., admin, doctor, patient)\n- **API Security**\n  - Rate limiting and abuse prevention on all public endpoints\n  - Input validation and sanitization to prevent injection attacks\n  - CORS policy hardening for production\n- **Data Protection**\n  - Encrypt sensitive data at rest (MongoDB encryption)\n  - Encrypted secrets management (Kubernetes Secrets, Docker secrets)\n  - Secure handling of environment variables and API keys\n- **Transport Security**\n  - Enforce HTTPS everywhere (Ingress, frontend, backend)\n  - Automatic TLS certificate management (cert-manager)\n- **Audit \u0026 Monitoring**\n  - Structured, tamper-resistant logging for all services\n  - Audit trails for user actions and admin operations\n  - Integration with monitoring/alerting (Prometheus, Grafana, Sentry)\n- **Vulnerability Management**\n  - Automated dependency scanning (GitHub Dependabot, cargo audit, npm audit)\n  - Regular security updates for all base images and dependencies\n- **Privacy \u0026 Compliance**\n  - Data retention and deletion policies (user chat history, logs)\n  - GDPR/CCPA compliance roadmap\n- **AI Safety**\n  - Prompt injection mitigation and output filtering\n  - Guardrails to prevent unsafe or non-medical advice\n\n*Contributions and suggestions for additional security features are welcome!*\n\n---\n\n## Data Protection\n\n- **MongoDB Encryption at Rest**: For production, enable MongoDB's encryption at rest (see [MongoDB docs](https://www.mongodb.com/docs/manual/core/security-encryption-at-rest/)). The provided `mongo-statefulset.yaml` is ready for use with encrypted storage classes and can be extended to enable encryption options.\n- **Kubernetes Secrets**: All sensitive credentials (MongoDB root password, Ollama API key, etc.) are managed via Kubernetes Secrets and never hardcoded. See `k8s/mongo-statefulset.yaml` and `k8s/backend-deployment.yaml` for examples.\n- **Environment Variables**: The backend loads all secrets and sensitive config from environment variables, never from code or public files. This includes database URIs and API keys.\n- **Docker Secrets**: For Docker Compose or Swarm, use Docker secrets for production deployments to avoid exposing secrets in environment variables.\n\n**Best Practices:**\n- Always rotate secrets regularly and never commit them to version control.\n- Use encrypted storage (EBS, etc.) for all persistent data in production.\n- Review and restrict access to secrets in your cloud provider and Kubernetes cluster.\n\n---\n\n## License\n\nMIT ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphilodi-dev%2Frust-production-ai-assistant-app","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fphilodi-dev%2Frust-production-ai-assistant-app","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphilodi-dev%2Frust-production-ai-assistant-app/lists"}