{"id":43091844,"url":"https://github.com/mheadd/llm-persistent-instructions","last_synced_at":"2026-01-31T16:16:09.528Z","repository":{"id":306944314,"uuid":"1027772488","full_name":"mheadd/llm-persistent-instructions","owner":"mheadd","description":"𝌭 A prototype application demonstrating how to create different AI personas using instruction layering with open-source LLMs","archived":false,"fork":false,"pushed_at":"2025-10-26T11:46:34.000Z","size":4477,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-26T13:24:02.835Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mheadd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-07-28T14:07:26.000Z","updated_at":"2025-10-26T11:46:37.000Z","dependencies_parsed_at":"2025-07-28T16:26:18.881Z","dependency_job_id":"3550b9e6-4ada-45c1-a88b-500aba1a7625","html_url":"https://github.com/mheadd/llm-persistent-instructions","commit_stats":null,"previous_names":["mheadd/llm-persistent-instructions"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mheadd/llm-persistent-instructions","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mheadd%2Fllm-persistent-instructions","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mheadd%2Fllm-persistent-instructions/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mheadd%2Fllm-persistent-instructions/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mheadd%2Fllm-persistent-instructions/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mheadd","download_url":"https://codeload.github.com/mheadd/llm-persistent-instructions/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mheadd%2Fllm-persistent-instructions/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28947573,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-31T14:26:55.697Z","status":"ssl_error","status_checked_at":"2026-01-31T14:26:52.545Z","response_time":128,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-31T16:16:03.894Z","updated_at":"2026-01-31T16:16:09.522Z","avatar_url":"https://github.com/mheadd.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Government AI Prototype - Persistent Instruction Layering Demo\n\n[![CI - Tests \u0026 Security](https://github.com/mheadd/llm-persistent-instructions/actions/workflows/ci.yml/badge.svg)](https://github.com/mheadd/llm-persistent-instructions/actions/workflows/ci.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\nA containerized application demonstrating how to create different AI personas using instruction layering with **multiple LLM providers**. This prototype shows how the same instruction layering approach works seamlessly across local models (Ollama) and cloud APIs (OpenAI, Anthropic, etc.). This is an idea that I first wrote about in [this LinkedIn post](https://www.linkedin.com/pulse/open-source-ai-government-two-paths-better-digital-services-headd-2iine/).\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"llm-instructions-layering-example.gif\" alt=\"LLM Instructions Layering Demo\" style=\"border: 2px solid #ddd; border-radius: 8px; box-shadow: 0 4px 8px rgba(0,0,0,0.1); max-width: 100%;\"\u003e\n\u003c/div\u003e\n\n## 📋 Table of Contents\n\n- [What This Demonstrates](#-what-this-demonstrates)\n- [Government Service Personas](#️-government-service-personas)\n- [Architecture](#️-architecture)\n- [Provider Configuration](#-provider-configuration)\n- [Quick Start](#-quick-start)\n- [API Endpoints](#-api-endpoints)\n- [Technical Stack](#️-technical-stack)\n- [Project Structure](#-project-structure)\n- [Configuration](#-configuration)\n- [How Instruction Layering Works](#-how-instruction-layering-works)\n- [Testing Different Personas](#-testing-different-personas)\n- [Automated Testing](#-automated-testing)\n- [Development Workflow](#-development-workflow)\n- [Continuous Integration](#-continuous-integration)\n- [Troubleshooting](#-troubleshooting)\n- [Use Cases \u0026 Applications](#-use-cases--applications)\n- [Contributing](#-contributing)\n- [License](#-license)\n\n## 🎯 What This Demonstrates\n\nThis project showcases **persistent instruction layering** - a technique where different system prompts and contexts are dynamically applied to any compatible AI model to create specialized assistants. Instead of training separate models, we use strategic prompt engineering to create distinct AI personas that work consistently across **multiple LLM backends**.\n\n### 🔄 **Multi-Provider Architecture**\n\nThe application now supports:\n- **🏠 Local Models**: Ollama with Phi-3, Llama, etc. (free, private)\n- **☁️ Cloud APIs**: OpenAI GPT-3.5/GPT-4, Anthropic Claude (paid, powerful)\n- **🔀 Easy Switching**: Change providers via environment variables\n- **📊 Monitoring**: Real-time provider status and health checks\n\n## 🏛️ Government Service Personas\n\nThe demo includes three government service assistants:\n\n1. **🏢 Unemployment Benefits Assistant**\n   - Eligibility requirements and application guidance\n   - Benefit calculations and status checks\n   - Appeals process information\n\n2. **🌳 Parks and Recreation Guide**\n   - Park information and facility details\n   - Activity recommendations and schedules\n   - Reservation systems and accessibility info\n\n3. **📋 Business Licensing Assistant**\n   - Permit requirements and application processes\n   - Regulatory compliance guidance\n   - Fee structures and renewal procedures\n\nFor more information on personas, and instructions on how to add new ones - see the [PERSONAS-GUIDE.md](PERSONAS-GUIDE.md)\n\n## 🏗️ Architecture\n\n```\n┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐\n│   Client App    │    │   Node.js API    │    │  LLM Providers  │\n│                 │────│   (Port 3000)    │────│                 │\n│   Web/Mobile    │    │  Provider        │    │ • Ollama+Phi-3  │\n│                 │    │  Abstraction     │    │ • OpenAI GPT    │\n└─────────────────┘    └──────────────────┘    │ • Anthropic     │\n                                               │ • Azure OpenAI  │\n                                               └─────────────────┘\n```\n\n### 🔄 **Provider Abstraction Layer**\nThe application features a flexible provider abstraction that allows seamless switching between:\n- **Local Models**: Ollama with various models (Phi-3, Llama2, CodeLlama, etc.)\n- **OpenAI**: GPT-3.5-turbo, GPT-4, GPT-4-turbo\n- **Future Support**: Anthropic Claude, Azure OpenAI, Google PaLM, and others\n\n### Data Flow\n1. Client sends request to specific persona endpoint\n2. API loads corresponding instruction template from YAML config\n3. **Provider abstraction** applies user message with persona-specific system prompt\n4. **Configurable LLM backend** generates response using instruction layering\n5. AI response returned with provider metadata and persona context maintained\n\n## 🔧 Provider Configuration\n\nThe application supports multiple LLM providers through a flexible configuration system using environment variables and `.env` files.\n\n### 🎛️ **Supported Providers**\n\n| Provider | Status | Type | Models | Cost | Setup Complexity |\n|----------|--------|------|---------|------|------------------|\n| **Ollama** | ✅ Ready | Local | Phi-3, Llama2, CodeLlama, etc. | Free | Medium (Docker required) |\n| **OpenAI** | ✅ Ready | Cloud API | GPT-3.5-turbo, GPT-4, GPT-4-turbo | Paid | Easy (API key only) |\n| **Anthropic** | 🚧 Planned | Cloud API | Claude-3-haiku, Claude-3-sonnet | Paid | Easy (API key only) |\n| **Azure OpenAI** | 🚧 Planned | Cloud API | GPT models via Azure | Paid | Medium (Azure setup) |\n\n### 📄 **Configuration File Setup**\n\n#### 1. **Copy Environment Template**\n```bash\ncd api\ncp .env.example .env\n```\n\n#### 2. **Configure Your Preferred Provider**\n\n**Option A: Local Ollama (Default)**\n```bash\n# Edit .env file\nLLM_PROVIDER=ollama\nOLLAMA_URL=http://ollama:11434\nOLLAMA_MODEL=phi3:mini\n```\n\n**Option B: OpenAI Cloud API**\n```bash\n# Edit .env file  \nLLM_PROVIDER=openai\nOPENAI_API_KEY=sk-your-actual-api-key-here\nOPENAI_MODEL=gpt-3.5-turbo\n```\n\n**Option C: OpenAI GPT-4 (More Capable)**\n```bash\n# Edit .env file\nLLM_PROVIDER=openai-gpt4\nOPENAI_API_KEY=sk-your-actual-api-key-here\nOPENAI_MODEL=gpt-4\n```\n\n### 🔄 **Easy Provider Switching**\n\n#### **Method 1: Edit .env File**\n```bash\n# Switch to OpenAI\nsed -i 's/LLM_PROVIDER=.*/LLM_PROVIDER=openai/' .env\n\n# Switch back to Ollama\nsed -i 's/LLM_PROVIDER=.*/LLM_PROVIDER=ollama/' .env\n\n# Restart the application\ndocker-compose restart api\n```\n\n#### **Method 2: Environment Variables (No Restart Required)**\n```bash\n# Run with OpenAI\nLLM_PROVIDER=openai OPENAI_API_KEY=your-key docker-compose up\n\n# Run with Ollama  \nLLM_PROVIDER=ollama docker-compose up\n```\n\n### 🔍 **Provider Status Monitoring**\n\nCheck current provider status and health:\n```bash\n# Current provider info\ncurl http://localhost:3000/api/provider/status\n\n# List all available providers\ncurl http://localhost:3000/api/providers\n\n# Test a specific provider\ncurl -X POST http://localhost:3000/api/provider/test \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"provider\": \"openai\"}'\n```\n\n### 🚀 **Provider-Specific Setup Instructions**\n\n#### **🏠 Ollama (Local) Setup**\n```bash\n# Default setup - no API keys required\nLLM_PROVIDER=ollama\nOLLAMA_URL=http://ollama:11434\nOLLAMA_MODEL=phi3:mini\n\n# Alternative models\nOLLAMA_MODEL=llama2           # Meta's Llama 2\nOLLAMA_MODEL=codellama        # Code-focused model  \nOLLAMA_MODEL=mistral          # Mistral 7B model\n```\n\n#### **☁️ OpenAI Setup**\n1. Get API key from [OpenAI Platform](https://platform.openai.com/api-keys)\n2. Configure environment:\n```bash\nLLM_PROVIDER=openai\nOPENAI_API_KEY=sk-your-actual-api-key-here\nOPENAI_MODEL=gpt-3.5-turbo    # Fast, cost-effective\n# OPENAI_MODEL=gpt-4          # More capable, slower, expensive\n```\n\n#### **🧠 Anthropic Claude Setup** (Future Support)\n```bash\nLLM_PROVIDER=anthropic\nANTHROPIC_API_KEY=your-anthropic-key\nANTHROPIC_MODEL=claude-3-haiku-20240307\n```\n\n### ⚡ **Performance \u0026 Cost Comparison**\n\n| Provider | Speed | Quality | Cost (1K tokens) | Setup Time |\n|----------|-------|---------|------------------|------------|\n| Ollama (Phi-3) | Fast | Good | Free | 5-10 min |\n| OpenAI GPT-3.5 | Very Fast | Excellent | $0.002 | 30 seconds |\n| OpenAI GPT-4 | Moderate | Outstanding | $0.03 | 30 seconds |\n\n### 🎯 **Use Case Recommendations**\n\n- **🏠 Development/Testing**: Use Ollama (free, private)\n- **💼 Production (Budget)**: Use OpenAI GPT-3.5-turbo  \n- **🚀 Production (Quality)**: Use OpenAI GPT-4\n- **🔒 High Security**: Use local Ollama (no data leaves your infrastructure)\n\n## 🚀 Quick Start\n\n### Prerequisites\n- Docker and Docker Compose\n- 8GB+ RAM (for Ollama) OR Cloud API key (for OpenAI/Anthropic)\n- Git\n\n### ⚡ **Quick Setup (Cloud API - Recommended for First-Time Users)**\n\n```bash\n# Clone the repository\ngit clone \u003crepository-url\u003e\ncd llm-persistent-instructions/api\n\n# Setup OpenAI (fastest way to get started)\ncp .env.example .env\n\n# Edit .env file and add your OpenAI API key:\n# LLM_PROVIDER=openai\n# OPENAI_API_KEY=sk-your-actual-api-key-here\n\n# Start the API only (no local model download required)\ndocker-compose up api\n\n# Test immediately (no waiting for model downloads)\ncurl -X POST http://localhost:3000/api/chat/unemployment-benefits \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"message\": \"How do I apply for unemployment benefits?\"}'\n```\n\n### 🏠 **Local Setup (Ollama - Free but Slower Initial Setup)**\n\n```bash\n# Clone the repository\ngit clone \u003crepository-url\u003e\ncd llm-persistent-instructions\n\n# Use default Ollama configuration\n# (No API keys required)\n\n# Start all services\ndocker-compose up -d\n\n# Pull the Phi-3 model (required on first run)\ndocker exec ollama-service ollama pull phi3:mini\n\n# Wait for model download (2.2GB - may take several minutes)\n# Test the API (first request may take 30-90 seconds)\ncurl -X POST http://localhost:3000/api/chat/unemployment-benefits \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"message\": \"How do I apply for unemployment benefits?\"}'\n```\n\n### ⏱️ **Setup Time Comparison**\n\n| Method | Initial Setup | First Response | Ongoing Cost |\n|--------|---------------|----------------|--------------|\n| **OpenAI** | 30 seconds | 2-5 seconds | ~$0.002/request |\n| **Ollama** | 5-10 minutes | 30-90 seconds | Free |\n\n### 🔄 **Switching Between Providers**\n\nAfter initial setup, you can easily switch providers:\n\n```bash\n# Switch to OpenAI (if you have an API key)\necho \"LLM_PROVIDER=openai\" \u003e api/.env\necho \"OPENAI_API_KEY=your-key-here\" \u003e\u003e api/.env\ndocker-compose restart api\n\n# Switch back to Ollama\necho \"LLM_PROVIDER=ollama\" \u003e api/.env  \ndocker-compose restart api\n\n# Check current provider\ncurl http://localhost:3000/api/provider/status\n```\n\n## 📡 API Endpoints\n\n### 🎭 **Persona Endpoints** (Work with any provider)\n| Endpoint | Purpose | Example Use Case |\n|----------|---------|------------------|\n| `POST /api/chat/unemployment-benefits` | Unemployment assistance | \"Am I eligible for benefits?\" |\n| `POST /api/chat/parks-recreation` | Parks \u0026 recreation info | \"What activities are available at Central Park?\" |\n| `POST /api/chat/business-licensing` | Business permit guidance | \"What permits do I need for a food truck?\" |\n| `POST /api/chat/default` | General assistant | \"Hello, how can you help me?\" |\n\n### 🔧 **Provider Management Endpoints** (New!)\n| Endpoint | Purpose | Example Use Case |\n|----------|---------|------------------|\n| `GET /api/provider/status` | Current provider health | Check if current provider is working |\n| `GET /api/providers` | List all available providers | See configuration options |\n| `POST /api/provider/test` | Test a specific provider | Validate API keys before switching |\n\n### Request Format\n```json\n{\n  \"message\": \"Your question here\"\n}\n```\n\n### Response Format (Enhanced with Provider Info)\n```json\n{\n  \"response\": \"AI-generated response with persona-specific guidance\",\n  \"persona\": \"unemployment-benefits\",\n  \"provider\": \"openai\",\n  \"model\": \"gpt-3.5-turbo\", \n  \"usage\": {\n    \"prompt_tokens\": 45,\n    \"completion_tokens\": 120,\n    \"total_tokens\": 165\n  },\n  \"timestamp\": \"2025-08-03T15:30:00Z\"\n}\n```\n\n### Provider Status Response\n```json\n{\n  \"provider\": \"OpenAI (gpt-3.5-turbo)\",\n  \"healthy\": true,\n  \"config\": {\n    \"type\": \"openai\",\n    \"model\": \"gpt-3.5-turbo\",\n    \"hasApiKey\": true\n  },\n  \"timestamp\": \"2025-08-03T15:30:00Z\"\n}\n```\n\n## 🛠️ Technical Stack\n\n### 🧠 **AI Models (Configurable)**\n- **Local**: Phi-3 Mini (3.8B), Llama2, CodeLlama, Mistral via Ollama\n- **Cloud**: OpenAI GPT-3.5-turbo, GPT-4, GPT-4-turbo\n- **Future**: Anthropic Claude, Azure OpenAI, Google PaLM\n\n### 🏗️ **Backend Architecture**\n- **API**: Node.js with Express.js\n- **Provider Abstraction**: Factory pattern with base provider interface\n- **Configuration**: YAML files for personas + .env for provider settings\n- **Environment Management**: dotenv with validation and fallbacks\n- **Containerization**: Docker Compose with service profiles\n\n### 📊 **Monitoring \u0026 Management**\n- **Health Checks**: Real-time provider status monitoring\n- **Usage Tracking**: Token/request counting and performance metrics\n- **Configuration Validation**: Startup environment validation\n- **Error Handling**: Provider-specific error handling and fallbacks\n\n## 📁 Project Structure\n\n```\nllm-persistent-instructions/\n├── docker-compose.yml           # Multi-container orchestration\n├── api/                        # Node.js API service\n│   ├── Dockerfile             \n│   ├── package.json           \n│   ├── server.js              # Express.js server with provider abstraction\n│   ├── .env.example           # Environment configuration template\n│   ├── .env                   # Your actual configuration (git-ignored)\n│   ├── demo-provider-switching.sh  # Interactive provider demo\n│   ├── providers/             # LLM provider abstraction layer\n│   │   ├── base-provider.js   # Base provider interface\n│   │   ├── ollama-provider.js # Ollama implementation\n│   │   ├── openai-provider.js # OpenAI implementation\n│   │   └── provider-factory.js # Provider factory pattern\n│   └── config/                # Configuration management\n│       ├── llm-config.yaml    # Provider configurations\n│       ├── llm-config-manager.js # Configuration loader\n│       ├── environment-config.js # Environment validation\n│       ├── unemployment-benefits.yaml # Persona configs\n│       ├── parks-recreation.yaml\n│       └── business-licensing.yaml\n├── README.md                   # This file\n├── TODO.md                    # Implementation roadmap\n├── IMPLEMENTATION-COMPLETE.md # Implementation summary\n└── ENV-CONFIGURATION.md       # Environment setup guide\n```\n\n## 🔧 Configuration\n\nEach persona is defined by a YAML configuration file in `api/config/`:\n\n```yaml\npersona: \"unemployment-benefits\"\nsystem_prompt: |\n  You are a helpful assistant specializing in unemployment insurance benefits.\n  Provide accurate, empathetic guidance on eligibility, applications, and processes.\n  Always suggest contacting local unemployment offices for official determinations.\nexamples:\n  - user: \"Am I eligible for unemployment?\"\n    assistant: \"To determine eligibility, I'll need to understand your situation...\"\n```\n\n## 🔍 How Instruction Layering Works\n\n1. **Base Model**: Phi-3 Mini provides general language understanding\n2. **System Prompts**: Each endpoint loads persona-specific instructions\n3. **Context Injection**: User messages are prefixed with system prompts\n4. **Response Generation**: Model responds within the established persona context\n5. **Consistency**: Conversation maintains persona throughout the session\n\n## 🧪 Testing Different Personas\n\nThe instruction layering works identically across all providers. Try these examples:\n\n### 🎭 **Persona Testing (Any Provider)**\n```bash\n# Unemployment Benefits\ncurl -X POST http://localhost:3000/api/chat/unemployment-benefits \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"message\": \"I was laid off last week. What should I do?\"}'\n\n# Parks \u0026 Recreation  \ncurl -X POST http://localhost:3000/api/chat/parks-recreation \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"message\": \"I want to plan a family picnic. Any suggestions?\"}'\n\n# Business Licensing\ncurl -X POST http://localhost:3000/api/chat/business-licensing \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"message\": \"I want to start a consulting business. What permits do I need?\"}'\n```\n\n### 🔍 **Provider Comparison Testing**\n```bash\n# Test with Ollama (local, free)\necho \"LLM_PROVIDER=ollama\" \u003e api/.env\ndocker-compose restart api\ncurl -X POST http://localhost:3000/api/chat/business-licensing \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"message\": \"What permits do I need for a food truck?\"}' | jq '.provider'\n\n# Test with OpenAI (cloud, paid) \necho \"LLM_PROVIDER=openai\" \u003e api/.env\necho \"OPENAI_API_KEY=your-key\" \u003e\u003e api/.env\ndocker-compose restart api\ncurl -X POST http://localhost:3000/api/chat/business-licensing \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"message\": \"What permits do I need for a food truck?\"}' | jq '.provider'\n```\n\n### 📊 **Provider Status Monitoring**\n```bash\n# Check current provider\ncurl http://localhost:3000/api/provider/status | jq\n\n# List all available providers  \ncurl http://localhost:3000/api/providers | jq\n\n# Test specific provider connectivity\ncurl -X POST http://localhost:3000/api/provider/test \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"provider\": \"openai\"}' | jq\n```\n\n### 🎯 **Demonstration Script**\nRun the included demo to see provider switching in action:\n```bash\ncd api\n./demo-provider-switching.sh\n```\n\n## ✅ Automated Testing\n\nThis project includes a comprehensive test suite to ensure reliability and catch regressions during development.\n\n### Test Categories\n\n- **🔬 Unit Tests**: Configuration loading, data validation\n- **🔗 Integration Tests**: API endpoints, request/response handling\n- **⚡ Performance Tests**: Response times, concurrent request handling\n- **🎯 End-to-End Tests**: Complete user workflows, cross-persona consistency\n\n### Running Tests\n\n#### Quick Test (Recommended)\n```bash\n# Run the complete test suite\n./test.sh\n```\n\n#### Specific Test Categories\n```bash\n# Unit tests only (fast, no Docker required)\n./test.sh unit\n\n# Integration tests (requires Docker containers)\n./test.sh integration\n\n# Performance tests\n./test.sh performance\n\n# End-to-end tests\n./test.sh e2e\n\n# Generate coverage report\n./test.sh coverage\n```\n\n#### Manual Testing (Advanced)\n```bash\n# Install test dependencies\ncd api\nnpm install\n\n# Run specific test files\nnpm test -- config.test.js\nnpm test -- api.test.js\nnpm test -- performance.test.js\nnpm test -- e2e.test.js\n\n# Watch mode for development\nnpm run test:watch\n\n# Coverage report\nnpm run test:coverage\n```\n\n### Test Environment Setup\n\nThe test suite automatically:\n1. ✅ Checks Docker availability\n2. ✅ Starts containers if needed\n3. ✅ Waits for services to be ready\n4. ✅ Installs dependencies\n5. ✅ Runs tests with mocked AI responses\n6. ✅ Generates reports\n\n### Test Features\n\n- **Mocked AI Responses**: Tests run quickly without hitting the actual LLM\n- **Realistic Scenarios**: Tests cover common government service inquiries\n- **Error Handling**: Validates graceful handling of service failures\n- **Performance Monitoring**: Ensures response times meet expectations\n- **Cross-Browser Support**: API tests work with any HTTP client\n\n### Continuous Integration\n\nFor CI/CD pipelines, use:\n```bash\n# Non-interactive test run with coverage\n./test.sh coverage\n\n# Results are saved to api/coverage/ for CI reporting\n```\n\n### Test Results\n\nAfter running tests, you'll see:\n- ✅ **Pass/Fail Status**: Clear indication of test results\n- 📊 **Coverage Report**: Code coverage metrics\n- ⏱️ **Performance Metrics**: Response time measurements\n- 🐛 **Error Details**: Specific failure information when tests fail\n\nExample output:\n```\n🧪 Government AI Prototype - Test Suite\n========================================\n✅ Docker containers are running\n📦 Installing test dependencies...\n🔬 Running unit tests...\n   ✓ Configuration loading (4 tests)\n🔗 Running integration tests...\n   ✓ API endpoints (12 tests)\n⚡ Running performance tests...\n   ✓ Response times (6 tests)\n🎯 Running end-to-end tests...\n   ✓ User workflows (8 tests)\n📊 Coverage: 95% statements, 92% branches\n✅ Test execution completed!\n```\n\n## 🔄 Development Workflow\n\n1. **Modify Personas**: Edit YAML files in `api/config/`\n2. **Test Changes**: Run tests with `./test.sh unit` or `./test.sh integration`\n3. **Add New Personas**: Create new YAML config + add route in `server.js`\n4. **Monitor Logs**: `docker-compose logs -f api ollama`\n5. **Commit Changes**: Push to main branch triggers automated CI/CD\n\n## 🚀 Continuous Integration\n\nThis project includes automated GitHub Actions workflows that run on every commit to the main branch:\n\n### CI/CD Pipeline\n- **🔬 Unit Tests**: Configuration loading and validation tests\n- **🔗 Integration Tests**: API endpoint and request handling tests  \n- **🛡️ Security Scanning**: npm audit for dependency vulnerabilities\n- **📊 Coverage Reports**: Automated test coverage generation\n- **📤 Artifacts**: Test results and coverage reports saved for 7 days\n\n### Workflow Triggers\n- **Push to main**: Full test suite + security scan\n- **Pull Requests**: Tests + dependency review\n- **Manual**: Can be triggered manually from GitHub Actions tab\n\n### Viewing Results\n1. Go to the **Actions** tab in your GitHub repository\n2. Click on the latest workflow run\n3. View test results, coverage reports, and security scan results\n4. Download artifacts for detailed analysis\n\n### Local vs CI Testing\n```bash\n# Local development testing\n./test.sh unit integration    # Quick feedback during development\n\n# CI pipeline runs automatically\ngit push origin main          # Triggers full CI pipeline\n```\n\n## 🚨 Troubleshooting\n\n### Common Issues and Solutions\n\n#### 1. **Docker Build Fails - NPM Dependencies**\n**Problem**: `npm ci --only=production` fails during Docker build\n```\n=\u003e ERROR [api 4/8] RUN npm ci --only=production\n```\n\n**Solution**: Ensure `package-lock.json` exists by running locally:\n```bash\ncd api\nnpm install  # This creates package-lock.json\ncd ..\ndocker-compose build api\n```\n\n#### 2. **Ollama Container Fails to Start**\n**Problem**: `dependency failed to start: container ollama-service exited (1)`\n\n**Solution**: Check if Docker has enough resources and restart:\n```bash\n# Check container logs\ndocker-compose logs ollama\n\n# Clean restart\ndocker-compose down\ndocker-compose up -d\n```\n\n#### 3. **API Timeout Errors**\n**Problem**: `timeout of 30000ms exceeded` or similar timeout errors\n\n**Symptoms**:\n- First requests take very long\n- Timeout errors on initial model loading\n\n**Solution**: This is expected behavior on first runs:\n```bash\n# Check if containers are running\ndocker-compose ps\n\n# Wait for model to fully load (can take 2-3 minutes)\n# Monitor Ollama logs\ndocker-compose logs -f ollama\n\n# The first request to each persona may timeout - this is normal\n# Subsequent requests will be much faster\n```\n\n#### 4. **API Returns Health Check Failures**\n**Problem**: API container shows as `unhealthy` in `docker-compose ps`\n\n**Solution**: The health check may fail initially while services start:\n```bash\n# Check API logs\ndocker-compose logs api\n\n# Test health endpoint manually\ncurl http://localhost:3000/health\n\n# If still failing, rebuild the API:\ndocker-compose build api\ndocker-compose up -d api\n```\n\n#### 5. **Model Not Found Errors**\n**Problem**: `model 'phi3:mini' not found` in API responses\n\n**Solution**: Manually pull the model:\n```bash\n# Pull the model explicitly\ndocker exec ollama-service ollama pull phi3:mini\n\n# Verify it's available\ndocker exec ollama-service ollama list\n```\n\n#### 6. **Port Already in Use**\n**Problem**: `port is already allocated` errors\n\n**Solution**: Stop conflicting services or change ports:\n```bash\n# Find what's using the ports\nlsof -i :3000\nlsof -i :11434\n\n# Stop conflicting services or modify docker-compose.yml ports\n# For example, change \"3000:3000\" to \"3001:3000\"\n```\n\n#### 7. **Out of Memory Issues**\n**Problem**: Containers crash or become unresponsive\n\n**Symptoms**:\n- Docker containers randomly stopping\n- Very slow response times\n- System becomes unresponsive\n\n**Solution**: Ensure adequate system resources:\n```bash\n# Check Docker resource allocation\ndocker stats\n\n# Minimum requirements:\n# - 8GB RAM available\n# - 4GB free disk space\n# - Docker allocated sufficient memory (Docker Desktop settings)\n```\n\n### Performance Optimization Tips\n\n1. **Keep containers running**: Avoid frequent restarts to prevent model reloading\n2. **Monitor resource usage**: Use `docker stats` to monitor container resource consumption\n3. **Sequential testing**: Test one persona at a time initially to avoid overloading\n4. **Patience on first run**: Initial setup and first requests require patience\n\n### Getting Help\n\nIf you encounter issues not covered here:\n\n1. **Check container logs**: `docker-compose logs [service-name]`\n2. **Verify container status**: `docker-compose ps`\n3. **Test individual components**: \n   - API health: `curl http://localhost:3000/health`\n   - Ollama service: `curl http://localhost:11434/api/tags`\n4. **Clean restart**: `docker-compose down \u0026\u0026 docker-compose up -d`\n\n## 💡 Use Cases \u0026 Applications\n\n### 🎯 **Instruction Layering Pattern**\nThis approach is valuable for creating specialized AI assistants in:\n- **Government Services**: Different department assistants\n- **Customer Support**: Product-specific help agents  \n- **Education**: Subject-matter tutoring bots\n- **Healthcare**: Specialized medical information assistants\n- **E-commerce**: Category-specific shopping advisors\n\n### 🔄 **Multi-Provider Benefits**\nThe provider abstraction enables:\n- **💰 Cost Optimization**: Free local models for development, paid APIs for production\n- **🔒 Security Options**: Local models for sensitive data, cloud APIs for general use\n- **⚡ Performance Tuning**: Fast cloud APIs for production, local models for testing\n- **🌍 Global Deployment**: Choose providers based on regional availability\n- **🎛️ A/B Testing**: Compare model performance across different providers\n\n### 🚀 **Deployment Scenarios**\n\n#### **Development Environment**\n```bash\nLLM_PROVIDER=ollama          # Free, local development\n```\n\n#### **Production (Budget-Conscious)**\n```bash\nLLM_PROVIDER=openai          # Cost-effective cloud API\nOPENAI_MODEL=gpt-3.5-turbo\n```\n\n#### **Production (Quality-First)**\n```bash\nLLM_PROVIDER=openai          # High-quality responses\nOPENAI_MODEL=gpt-4\n```\n\n#### **High-Security Environment**\n```bash\nLLM_PROVIDER=ollama          # No data leaves your infrastructure\nOLLAMA_MODEL=phi3:mini\n```\n\n### ✨ **Key Architectural Achievement**\n\nThe same persona configurations and instruction layering approach work **identically** across all providers, demonstrating true architectural flexibility and provider independence! 🎉\n\n## 🤝 Contributing\n\nThis is a learning prototype. Feel free to:\n- Add new government service personas\n- Improve instruction templates\n- Enhance the API with conversation memory\n- Add a web frontend for easier testing\n- Experiment with different LLM models\n\n**Note**: This is a prototype for demonstration purposes. For production government services, ensure proper security, accessibility, and compliance with relevant regulations.\n\n## 📄 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n### 🎯 What this means:\n- ✅ **Free to use** for any purpose (personal, commercial, government)\n- ✅ **Free to modify** and distribute\n- ✅ **No warranty** - use at your own risk\n- ✅ **Attribution required** - just keep the license notice\n\n**MIT License** | Copyright (c) 2025 Government AI Prototype Contributors\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmheadd%2Fllm-persistent-instructions","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmheadd%2Fllm-persistent-instructions","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmheadd%2Fllm-persistent-instructions/lists"}