{"id":30756432,"url":"https://github.com/liteobject/pihole-llm-analytics","last_synced_at":"2026-04-15T13:32:45.625Z","repository":{"id":311862892,"uuid":"1045358952","full_name":"LiteObject/pihole-llm-analytics","owner":"LiteObject","description":"AI-powered DNS log analyzer for Pi-hole - detect suspicious domains, traffic patterns, and security threats using gpt-oss.","archived":false,"fork":false,"pushed_at":"2025-08-27T03:48:19.000Z","size":9,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-27T11:59:37.450Z","etag":null,"topics":["dns","gpt-oss","network-security","ollama","pi-hole","python","security"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LiteObject.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-27T03:30:17.000Z","updated_at":"2025-08-27T03:48:23.000Z","dependencies_parsed_at":"2025-08-27T11:59:45.824Z","dependency_job_id":"a7fa7df6-2ba6-4b5d-a639-f1280b530461","html_url":"https://github.com/LiteObject/pihole-llm-analytics","commit_stats":null,"previous_names":["liteobject/pihole-llm-analytics"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/LiteObject/pihole-llm-analytics","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiteObject%2Fpihole-llm-analytics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiteObject%2Fpihole-llm-analytics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiteObject%2Fpihole-llm-analytics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiteObject%2Fpihole-llm-analytics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LiteObject","download_url":"https://codeload.github.com/LiteObject/pihole-llm-analytics/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiteObject%2Fpihole-llm-analytics/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273589298,"owners_count":25132860,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-04T02:00:08.968Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dns","gpt-oss","network-security","ollama","pi-hole","python","security"],"created_at":"2025-09-04T10:02:31.236Z","updated_at":"2026-04-15T13:32:45.594Z","avatar_url":"https://github.com/LiteObject.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pi-hole LLM Analytics\n\nA comprehensive Python application that analyzes Pi-hole DNS logs using Large Language Models (LLMs) to provide intelligent insights, threat detection, and automated anomaly analysis.\n\n## Table of Contents\n- [Features](#features)\n- [Quick Start](#quick-start)\n- [Installation](#installation)\n- [Configuration](#configuration)\n- [Usage](#usage)\n- [Architecture](#architecture)\n- [Examples](#examples)\n- [Troubleshooting](#troubleshooting)\n- [Development](#development)\n- [References](#references)\n- [License](#license)\n\n## Features\n\n### **Intelligent DNS Analysis**\n- AI-powered categorization of DNS queries and domains\n- Automated anomaly detection using machine learning patterns\n- Natural language search over DNS logs\n- Comprehensive reporting with actionable insights\n\n### **Advanced Security Monitoring**\n- Real-time threat detection and alerting\n- Domain reputation analysis with threat intelligence\n- DNS tunneling and beaconing detection\n- Suspicious client activity monitoring\n- Pattern-based attack recognition\n\n### **Analytics \u0026 Reporting**\n- Daily, weekly, and security-focused reports\n- Client-specific activity analysis\n- Domain categorization and trend analysis\n- Customizable alert thresholds and notifications\n- Export capabilities (JSON, text, structured reports)\n\n### **Modular Architecture**\n- Clean separation of concerns with dedicated modules\n- Type-safe data models with comprehensive validation\n- Configurable components for different environments\n- Extensive logging and error handling\n- RESTful API design patterns\n\n## Common Use Cases\n\n### **Home Network Monitoring**\nMonitor and protect your home network from threats, track family internet usage, and identify potential security risks in real-time.\n\n### **Small Business Security**\nTrack employee browsing patterns, detect security risks, generate compliance reports, and monitor for data exfiltration attempts.\n\n### **Research \u0026 Analysis**\nAnalyze DNS patterns for research purposes, study internet usage trends, and investigate network behavior patterns.\n\n### **Security Operations**\nReal-time threat detection, automated incident response, DNS-based attack prevention, and comprehensive security reporting.\n\n## Quick Start\n\n### System Requirements\n\n- **Operating System**: Windows 10+, macOS 10.15+, Linux (Ubuntu 20.04+)\n- **Python**: 3.11 or higher (required for modern type hints and performance)\n- **RAM**: Minimum 2GB, Recommended 4GB+\n- **Disk Space**: 500MB for application and logs\n- **Network**: Stable connection to Pi-hole server\n\n### Prerequisites\n\n- **Pi-hole v6+** with API access enabled\n- **LLM Provider** (choose one):\n  - **Ollama** with any compatible model (e.g., gpt-oss:latest, llama3.2:latest, gemma3:4b)\n  - **OpenAI API** with valid API key\n- Network access between the analytics server and Pi-hole\n\n### Installation\n\n1. **Clone the repository:**\n   ```bash\n   git clone https://github.com/LiteObject/pihole-llm-analytics.git\n   cd pihole-llm-analytics\n   ```\n\n2. **Set up Python virtual environment:**\n   ```bash\n   python -m venv .venv\n   \n   # On Windows:\n   .venv\\Scripts\\activate\n   \n   # On macOS/Linux:\n   source .venv/bin/activate\n   ```\n\n3. **Install dependencies:**\n   ```bash\n   pip install -r requirements.txt\n   ```\n\n4. **Configure environment variables:**\n   ```bash\n   # Copy the example configuration\n   cp .env.example .env\n   \n   # Edit .env with your settings\n   nano .env\n   ```\n\n### Configuration\n\nCreate a `.env` file in the project root with your settings:\n\n```bash\n# Pi-hole Configuration\nPIHOLE_HOST=192.168.7.99        # Pi-hole server IP/hostname\nPIHOLE_PORT=8080                # Pi-hole admin interface port (default: 80)\nPIHOLE_PASSWORD=your_admin_password  # Pi-hole admin password (NOT API token)\n\n# LLM Configuration (Ollama - default)\nOLLAMA_URL=http://localhost:11434\nOLLAMA_MODEL=gpt-oss:latest\nOLLAMA_TIMEOUT=120\nOLLAMA_TEMPERATURE=0.2\nOLLAMA_MAX_TOKENS=512\n\n# LLM Configuration (OpenAI - alternative)\n# LLM_PROVIDER=openai\n# LLM_API_KEY=your_openai_api_key_here\n# LLM_MODEL=gpt-3.5-turbo\n# LLM_API_BASE_URL=https://api.openai.com/v1\n\n# General LLM Configuration\nLLM_TIMEOUT=120\nLLM_TEMPERATURE=0.2\nLLM_MAX_TOKENS=512\nMAX_PROMPT_CHARS=18000\n\n# Analytics Configuration\nANALYTICS_LOG_COUNT=1000\nANALYTICS_ENABLE_CACHING=true\nANALYTICS_ANOMALY_THRESHOLD=0.1\n\n# Security Configuration\nSECURITY_ENABLE_THREAT_DETECTION=true\nSECURITY_ALERT_THRESHOLD=medium\nSECURITY_ENABLE_REPUTATION_CHECK=true\n\n# Application Configuration\nAPP_LOG_LEVEL=INFO\nAPP_LOG_FORMAT=json\n```\n\n**Important Notes**:\n- Use `PIHOLE_PASSWORD` for your Pi-hole admin password, not the API token\n- Include `PIHOLE_HOST` and `PIHOLE_PORT` separately for better control\n- **Always test connections first**: `python integrated_analysis.py --test-connection`\n- **Pi-hole data fetching is reliable** - even if LLM analysis fails, you'll get DNS query data\n- Use `pihole_diagnostics.py` for comprehensive Pi-hole diagnostics\n\n## Usage\n\n### Using the Integrated Analysis Script\n```bash\n# Test connections first (recommended)\npython integrated_analysis.py --test-connection\n\n# Basic analysis (standalone functionality)\npython integrated_analysis.py\n\n# Analyze more queries with JSON output\npython integrated_analysis.py --count 500 --output json\n\n# Use different model\npython integrated_analysis.py --model llama3.2:latest --verbose\n\n# Test with verbose logging for troubleshooting\npython integrated_analysis.py --count 50 --verbose\n```\n\n### Command Line Interface\n\nThe application provides a comprehensive CLI for all operations:\n\n```bash\n# Run comprehensive DNS analysis\npython -m pihole_analytics analyze --count 1000\n\n# Generate security report for the last 7 days\npython -m pihole_analytics report --type security --days 7\n\n# Search DNS logs with natural language\npython -m pihole_analytics search \"suspicious DNS queries from last hour\"\n\n# Check system status and component health\npython -m pihole_analytics status\n\n# Analyze specific client activity\npython -m pihole_analytics client-analysis 192.168.1.100 --hours 24\n\n# Check domain reputation\npython -m pihole_analytics reputation suspicious-domain.com\n\n# Categorize domains using AI\npython -m pihole_analytics categorize google.com facebook.com doubleclick.net\n```\n\n### Python API\n\nUse the modular API for integration into other applications:\n\n```python\nfrom pihole_analytics.main import PiholeAnalytics\n\n# Initialize with default configuration\nanalytics = PiholeAnalytics()\n\n# Run comprehensive analysis\nresults = analytics.run_analysis(count=1000)\n\n# Generate security report\nsecurity_report = analytics.generate_report(\"security\", days=7)\n\n# Search logs with natural language\nsearch_results = analytics.search_logs(\"blocked advertising domains\")\n\n# Analyze specific client\nclient_analysis = analytics.get_client_analysis(\"192.168.1.100\", hours=24)\n\n# Check system health\nstatus = analytics.get_system_status()\n```\n\n### Direct LLM Integration (Factory Pattern)\n\nFor advanced users who want direct access to the new LLM analyzer with factory pattern:\n\n```python\nfrom pihole_analytics.core.pihole_client import PiholeClient\nfrom pihole_analytics.analytics.llm_analyzer import LLMAnalyzer\nfrom pihole_analytics.analytics.llm_providers.factory import LLMProvider\nfrom pihole_analytics.utils.config import PiholeConfig\n\n# Configure Pi-hole client\npihole_config = PiholeConfig(host=\"192.168.7.99\", port=8080, password=\"your_password\")\npihole_client = PiholeClient(pihole_config)\n\n# Initialize LLM analyzer using factory pattern\nllm_analyzer = LLMAnalyzer.create_with_provider(LLMProvider.OLLAMA)\n\n# Test connections\nif not llm_analyzer.test_connection():\n    print(\"Warning: LLM service not available\")\n\n# Fetch and analyze\nwith pihole_client:\n    queries = pihole_client.fetch_queries(100)\n    analysis = llm_analyzer.analyze_queries(queries)\n    print(f\"Risk level: {analysis.threat_summary.get('risk_level')}\")\n    \n    # Custom analysis with specific instructions\n    custom_analysis = llm_analyzer.analyze_queries(queries, \"\"\"\n    Focus on:\n    1. Gaming traffic patterns\n    2. Streaming service usage  \n    3. Potential security threats\n    Return JSON with detailed breakdowns.\n    \"\"\")\n\n# Alternative: Legacy configuration method (still supported)\nfrom pihole_analytics.analytics.llm_analyzer import LLMConfig\nllm_config = LLMConfig.from_env()\nlegacy_analyzer = LLMAnalyzer(llm_config)\n```\n\n## Architecture\n\n### New Integrated Architecture\n\nThis project has been redesigned from a standalone script to a comprehensive, modular architecture with factory pattern for LLM providers:\n\n### **Key Improvements**\n- **Modular Design**: Separated concerns into dedicated modules\n- **Factory Pattern**: Pluggable LLM providers (Ollama, OpenAI, extensible)\n- **Error Handling**: Comprehensive error handling with graceful fallbacks\n- **Multiple Interfaces**: CLI, Python API, and standalone scripts\n- **Configuration Management**: Environment variables and programmatic config\n- **Robust Authentication**: Multiple Pi-hole authentication methods\n- **LLM Integration**: Structured LLM analysis with connection testing\n\n### **Factory Pattern Benefits**\nThe new factory pattern implementation provides:\n- **Provider Abstraction**: Easy switching between LLM providers\n- **Extensibility**: Simple to add new providers (Azure OpenAI, Anthropic, etc.)\n- **Fallback Support**: Automatic fallback when primary provider fails\n- **Type Safety**: Fully typed provider interfaces\n- **Configuration**: Environment-based or programmatic provider selection\n\n### **Supported LLM Providers**\n- **Ollama**: Local LLM hosting (default, requires Ollama service)\n- **OpenAI**: OpenAI API integration (requires API key)\n- **Extensible**: Easy to add Azure OpenAI, Anthropic Claude, etc.\n\n### **Architecture Benefits**\n| **Feature** | **Implementation** | **Benefit** |\n|-------------|-------------------|-------------|\n| **Modular Structure** | Separated packages | Easy maintenance \u0026 testing |\n| **Factory Pattern** | Pluggable LLM providers | Easy provider switching |\n| **Error Handling** | Comprehensive logging \u0026 fallbacks | Reliable operation |\n| **Configuration** | Environment + programmatic | Flexible deployment |\n| **Authentication** | Multiple Pi-hole auth methods | Broad compatibility |\n| **Output Formats** | JSON, text, structured | Integration flexibility |\n| **Connection Testing** | Built-in diagnostics | Quick troubleshooting |\n| **Extensibility** | Plugin architecture | Easy customization |\n\n### Project Structure\n\n```\npihole-llm-analytics/\n├── pihole_analytics/          # Main package\n│   ├── core/                  # Core functionality\n│   │   ├── pihole_client.py   # Pi-hole API client\n│   │   └── llm_client.py      # Legacy LLM integration\n│   ├── analytics/             # Analytics engine\n│   │   ├── analyzer.py        # Traditional DNS analysis\n│   │   ├── llm_analyzer.py    # AI-powered analysis with factory pattern\n│   │   └── llm_providers/     # LLM provider factory pattern\n│   │       ├── __init__.py    # Package initialization\n│   │       ├── base.py        # Abstract base provider\n│   │       ├── factory.py     # Provider factory and enum\n│   │       ├── ollama_provider.py  # Ollama implementation\n│   │       └── openai_provider.py  # OpenAI implementation\n│   ├── security/              # Security monitoring\n│   │   └── threat_detector.py # Threat detection and alerting\n│   ├── utils/                 # Shared utilities\n│   │   ├── models.py          # Data models and types\n│   │   ├── config.py          # Configuration management\n│   │   └── logging.py         # Logging utilities\n│   ├── main.py                # Main application interface\n│   ├── cli.py                 # Command-line interface\n│   └── __main__.py            # Module entry point\n├── integrated_analysis.py     # Standalone analysis script\n├── requirements.txt           # Python dependencies\n└── README.md                  # This documentation\n```\n\n### Key Components\n\n#### **Core Clients**\n- **PiholeClient**: Robust Pi-hole API integration with authentication, retry logic, and error handling\n- **LLMClient**: Legacy Ollama integration (maintained for compatibility)\n- **LLMAnalyzer**: AI-powered analysis with factory pattern for multiple LLM providers\n\n#### **Analytics Engine**\n- **DNSAnalyzer**: Traditional DNS log analysis with anomaly detection, categorization, and reporting  \n- **LLMAnalyzer**: AI-powered analysis with factory pattern, custom instructions and structured output\n- **ThreatDetector**: Advanced security monitoring with pattern recognition and threat intelligence\n- **LLM Providers**: Factory pattern implementation supporting Ollama, OpenAI, and extensible architecture\n\n#### **Data Models**\n- Type-safe dataclasses for all data structures\n- Comprehensive enums for status codes and categories\n- Validation and serialization support\n- Enhanced AnalysisResult model with LLM insights\n\n#### **Configuration System**\n- Environment variable-based configuration\n- Dataclass-based config objects with validation\n- Support for multiple configuration sources\n- Factory pattern configuration for LLM providers\n\n### Security Features\n\n#### **Threat Detection**\n- **Domain Reputation Analysis**: Real-time checking against threat intelligence feeds\n- **Pattern Recognition**: Detection of DNS tunneling, beaconing, and other suspicious patterns\n- **Anomaly Detection**: Statistical analysis to identify unusual query patterns\n- **Alert Generation**: Configurable alerts with severity levels and automated notifications\n\n#### **Monitoring Capabilities**\n- **Client Behavior Analysis**: Track individual client query patterns and anomalies\n- **Temporal Analysis**: Identify time-based attack patterns and suspicious activity windows\n- **Domain Classification**: AI-powered categorization of domains and threat assessment\n- **Trend Analysis**: Long-term pattern recognition for emerging threats\n\n## Configuration Options\n\n### Pi-hole Settings\n- `PIHOLE_HOST`: Pi-hole server IP address or hostname\n- `PIHOLE_PORT`: Pi-hole admin interface port (default: 80)\n- `PIHOLE_PASSWORD`: Pi-hole admin password (required for query logs access)\n- `PIHOLE_TIMEOUT`: Request timeout in seconds (default: 30)\n\n### LLM Settings\n- `LLM_URL`: Ollama server URL (default: http://localhost:11434)\n- `LLM_MODEL`: Model name (gpt-oss:latest recommended)\n- `LLM_TIMEOUT`: Request timeout in seconds (default: 30)\n\n### Analytics Settings\n- `ANALYTICS_LOG_COUNT`: Number of recent queries to analyze (default: 1000)\n- `ANALYTICS_ENABLE_CACHING`: Enable response caching (default: true)\n- `ANALYTICS_ANOMALY_THRESHOLD`: Anomaly detection sensitivity (default: 0.1)\n\n### LLM Configuration\n\n#### Advanced LLM Settings\n```python\nfrom pihole_analytics.analytics.llm_analyzer import LLMAnalyzer\nfrom pihole_analytics.analytics.llm_providers.factory import LLMProvider\n\n# Factory pattern (recommended)\nanalyzer = LLMAnalyzer.create_with_provider(LLMProvider.OLLAMA)\n\n# Legacy configuration (still supported)\nfrom pihole_analytics.analytics.llm_analyzer import LLMConfig\nllm_config = LLMConfig(\n    url=\"http://localhost:11434\",      # Ollama server URL\n    model=\"gpt-oss:latest\",            # Model name\n    timeout=120,                       # Request timeout\n    max_prompt_chars=18000,            # Prompt size limit\n    temperature=0.2,                   # Response creativity (0.0-1.0)\n    max_tokens=512                     # Response length limit\n)\nlegacy_analyzer = LLMAnalyzer(llm_config)\n\n# Test connection and get available models\nif analyzer.test_connection():\n    models = analyzer.get_available_models()\n    print(f\"Available models: {models}\")\n```\n\n#### Environment Variables for LLM\n```bash\n# LLM Configuration\nOLLAMA_URL=http://localhost:11434\nOLLAMA_MODEL=gpt-oss:latest\nOLLAMA_TIMEOUT=120\nOLLAMA_TEMPERATURE=0.2\nOLLAMA_MAX_TOKENS=512\nMAX_PROMPT_CHARS=18000\n```\n\n### Security Settings\n- `SECURITY_ENABLE_THREAT_DETECTION`: Enable threat detection (default: true)\n- `SECURITY_ALERT_THRESHOLD`: Minimum alert severity (low/medium/high/critical)\n- `SECURITY_ENABLE_REPUTATION_CHECK`: Enable domain reputation checking (default: true)\n\n## Examples\n\n### Quick Start Examples\n\n#### Basic Usage\n```bash\n# Simple analysis\npython integrated_analysis.py\n\n# Comprehensive analysis with options\npython integrated_analysis.py --count 500 --output json --verbose\n\n# Full CLI interface\npython -m pihole_analytics analyze --count 500\n```\n\n### Basic Analysis\n```python\nfrom pihole_analytics.main import PiholeAnalytics\n\nanalytics = PiholeAnalytics()\nresults = analytics.run_analysis(count=500)\n\nprint(f\"Analyzed {results['summary']['total_queries']} queries\")\nprint(f\"Block rate: {results['summary']['block_rate']*100:.1f}%\")\nprint(f\"Anomalies detected: {results['summary']['anomalies_detected']}\")\n```\n\n### Security Monitoring\n```python\n# Generate security report\nsecurity_report = analytics.generate_report(\"security\", days=7)\n\n# Check for active threats\nif security_report.get(\"threat_analysis\"):\n    threats = security_report[\"threat_analysis\"][\"threats_detected\"]\n    print(f\"Active threats: {threats}\")\n\n# Review security alerts\nfor alert in security_report.get(\"active_alerts\", []):\n    print(f\"Alert: {alert['title']} - {alert['severity']}\")\n```\n\n### Advanced LLM Analysis\n```python\nfrom pihole_analytics.analytics.llm_analyzer import LLMAnalyzer\nfrom pihole_analytics.analytics.llm_providers.factory import LLMProvider\n\n# Initialize LLM analyzer with factory pattern\nanalyzer = LLMAnalyzer.create_with_provider(LLMProvider.OLLAMA)\n\n# Test connection first\nif not analyzer.test_connection():\n    print(\"LLM service unavailable\")\n    exit(1)\n\n# Custom analysis with specific focus\ncustom_instructions = \"\"\"\nAnalyze the DNS logs focusing on:\n1. Gaming traffic patterns (Steam, Epic, etc.)\n2. Streaming service usage (Netflix, YouTube, etc.)\n3. Social media activity (Facebook, Twitter, etc.)\n4. Potential security threats or suspicious domains\n5. Bandwidth-heavy applications\n\nProvide detailed breakdown with percentages and specific recommendations.\nReturn analysis in the standard JSON format.\n\"\"\"\n\nanalysis = analyzer.analyze_queries(queries, custom_instructions)\n\n# Extract specific insights\nrisk_level = analysis.threat_summary.get('risk_level', 'unknown')\nprint(f\"Risk Assessment: {risk_level}\")\n\nif analysis.anomalies:\n    print(f\"Security Anomalies: {len(analysis.anomalies)}\")\n    for anomaly in analysis.anomalies:\n        print(f\"  • {anomaly.description} (Confidence: {anomaly.confidence:.1%})\")\n```\n\n### Integration Testing\n```python\n# Complete integration test\nfrom pihole_analytics.core.pihole_client import PiholeClient\nfrom pihole_analytics.analytics.llm_analyzer import LLMAnalyzer\nfrom pihole_analytics.analytics.llm_providers.factory import LLMProvider\nfrom pihole_analytics.utils.config import PiholeConfig\n\ndef test_integration():\n    \"\"\"Test the complete integration.\"\"\"\n    # Load from environment or configure directly\n    pihole_config = PiholeConfig.from_env()  # Or manual config\n    \n    # Test connections\n    pihole_client = PiholeClient(pihole_config)\n    llm_analyzer = LLMAnalyzer.create_with_provider(LLMProvider.OLLAMA)\n    \n    print(\"Testing Pi-hole connection...\")\n    with pihole_client:\n        queries = pihole_client.fetch_queries(10)\n        print(f\"Fetched {len(queries)} queries\")\n    \n    print(\"Testing LLM connection...\")\n    if llm_analyzer.test_connection():\n        print(\"LLM service available\")\n        models = llm_analyzer.get_available_models()\n        print(f\"Available models: {models[:3]}...\")\n    \n    return True\n\n# Run the test\nif test_integration():\n    print(\"Integration test successful!\")\n```\n\n### Client Analysis\n```python\n# Analyze specific client activity\nclient_ip = \"192.168.1.100\"\nanalysis = analytics.get_client_analysis(client_ip, hours=24)\n\nprint(f\"Client {client_ip} analysis:\")\nprint(f\"  Total queries: {analysis['summary']['total_queries']}\")\nprint(f\"  Unique domains: {analysis['summary']['unique_domains']}\")\nprint(f\"  Block rate: {analysis['summary']['block_rate']*100:.1f}%\")\n```\n\n## Testing \u0026 Validation\n\n### Connection Testing\n```bash\n# Test all connections without running analysis\npython integrated_analysis.py --test-connection\n\n# Verbose testing with detailed output\npython integrated_analysis.py --test-connection --verbose\n```\n\n### Performance Testing\n```bash\n# Test with different query counts\npython integrated_analysis.py --count 50 --output text    # Small test\npython integrated_analysis.py --count 500 --output json   # Medium test\npython integrated_analysis.py --count 1000 --verbose      # Large test\n\n# Test different models\npython integrated_analysis.py --model llama3.2:latest --count 100\npython integrated_analysis.py --model gemma3:4b --count 100\n```\n\n## Troubleshooting\n\n### Common Issues\n\n#### **Pi-hole Authentication Errors (401 Unauthorized)**\n\n**Problem**: Application authenticates successfully but fails when fetching queries with 401 Unauthorized error.\n\n**Symptoms**:\n```\n2025-08-27 13:19:39,471 - pihole_analytics.piholeclient - INFO - Successfully authenticated to Pi-hole\n2025-08-27 13:19:39,532 - pihole_analytics.piholeclient - ERROR - Error occurred: 401 Client Error: Unauthorized for url: http://192.168.7.99:8080/api/queries?sid=...\n```\n\n**Fixed in v1.1**: The application now automatically tries multiple authentication methods for Pi-hole v6.0+ compatibility.\n\n**Integrated Architecture Benefits**:\n- **Multiple Auth Methods**: Automatically tries session-based, password-based, and legacy token methods\n- **Graceful Fallbacks**: Continues with available data when some endpoints fail\n- **Enhanced Logging**: Detailed error messages with troubleshooting hints\n- **Connection Testing**: Built-in testing via `--test-connection` flag\n\n**How it works**:\n1. **Session-based authentication** (Pi-hole v5.x, backward compatibility)\n2. **Password-based per-request authentication** (Pi-hole v6.0+)\n3. **Legacy API token method** (older custom installations)\n\n**Quick Test**:\n```bash\n# Test with integrated script\npython integrated_analysis.py --test-connection\n\n# Test with full CLI\npython -m pihole_analytics status\n\n# Test specific analysis (will gracefully handle API restrictions)\npython integrated_analysis.py --count 10 --verbose\n```\n\n**If still having issues**:\n\n1. **Run integrated diagnostics**:\n   ```bash\n   # Test both Pi-hole and LLM connections\n   python integrated_analysis.py --test-connection --verbose\n   \n   # Test Pi-hole authentication and endpoints\n   python pihole_diagnostics.py\n   ```\n\n2. **Verify Pi-hole admin password**:\n   ```bash\n   # Test login via Pi-hole web interface first\n   # Use the exact same password in PIHOLE_PASSWORD\n   ```\n\n3. **Check Pi-hole API settings**:\n   - Navigate to Pi-hole Settings → API\n   - Ensure API access is enabled\n   - Check Query log display permissions\n\n4. **Test integrated authentication**:\n   ```python\n   from pihole_analytics.core.pihole_client import PiholeClient\n   from pihole_analytics.utils.config import PiholeConfig\n   \n   config = PiholeConfig(host=\"your_ip\", port=8080, password=\"your_password\")\n   with PiholeClient(config) as client:\n       queries = client.fetch_queries(5)  # Uses multiple auth methods automatically\n       print(f\"Success! Got {len(queries)} queries\")\n   ```\n\n**Pi-hole Version Notes**:\n- **v5.x and earlier**: Uses session-based authentication\n- **v6.0+**: Requires password-based auth for queries endpoint  \n- **Custom installations**: May require legacy API token format\n\nThe application automatically detects your Pi-hole version and uses the appropriate authentication method.\n\n#### **Pi-hole Connection Errors**\n- Verify Pi-hole URL and API key in `.env`\n- Ensure Pi-hole API is enabled in admin settings\n- Check network connectivity and firewall rules\n\n#### **LLM Service Errors**\n- **Connection Testing**: Use `python integrated_analysis.py --test-connection` to verify LLM connectivity\n- **Model Availability**: Check available models with the integrated analyzer\n- **Graceful Fallback**: The application continues with basic analysis if LLM fails\n\n```bash\n# Test LLM connectivity\npython integrated_analysis.py --test-connection\n\n# List available models\npython -c \"\nfrom pihole_analytics.analytics.llm_analyzer import LLMAnalyzer\nfrom pihole_analytics.analytics.llm_providers.factory import LLMProvider\nanalyzer = LLMAnalyzer.create_with_provider(LLMProvider.OLLAMA)\nprint('Available models:', analyzer.get_available_models())\n\"\n\n# Test with different model\npython integrated_analysis.py --model llama3.2:latest --test-connection\n```\n\n**Common LLM Parsing Error**: If you see `unhashable type: 'dict'` error:\n```bash\n# This indicates the LLM service is responding but with unexpected format\n# Pi-hole data collection still works - only LLM analysis fails\n\n# Try with a smaller query count first\npython integrated_analysis.py --count 10 --verbose\n\n# Test only the connection without analysis\npython integrated_analysis.py --test-connection\n\n# Check if Ollama service is running properly\nollama ps  # Should show running models\nollama list  # Should show available models\n```\n\n**What works regardless of LLM issues:**\n- Pi-hole authentication and connection\n- DNS query data fetching (100+ queries)\n- Basic DNS log parsing and validation\n- Only the AI-powered analysis may fail\n\nThe Pi-hole data fetching will still work even if LLM analysis fails.\n\n#### **Analysis Errors**\n- Start with smaller query counts for testing\n- Check Pi-hole logs for any query retrieval issues\n- Verify sufficient system memory for large datasets\n\n### Debugging\n\nEnable verbose logging for detailed diagnostic information:\n\n```bash\n# Set environment variable\nexport APP_LOG_LEVEL=DEBUG\n\n# Or use CLI flag\npython -m pihole_analytics --verbose analyze\n```\n\n#### **Using the Diagnostic Tools**\n\nThe project includes comprehensive diagnostic capabilities:\n\n```bash\n# Test both Pi-hole and LLM connections\npython integrated_analysis.py --test-connection\n\n# Test Pi-hole authentication and all endpoints\npython pihole_diagnostics.py\n\n# This will test:\n# - Authentication to Pi-hole\n# - All available API endpoints\n# - LLM service connectivity\n# - Data retrieval and parsing\n# - Provide detailed error information\n```\n\n**Example diagnostic output**:\n```\nGetting session ID...\nAuthentication successful: BOf8VGaTLcX62ezS0Zw64g=\n\nTesting: http://192.168.7.99:8080/api/summary?sid=...\n  Keys: ['domains_being_blocked', 'dns_queries_today', 'ads_blocked_today', ...]\n\nTesting: http://192.168.7.99:8080/api/queries?sid=...\n  Failed: 401 Client Error: Unauthorized for url: ...\n```\n\nThis helps identify exactly which endpoints are working and which are failing.\n\n### Performance Optimization\n\n- **Query Count**: Start with 500-1000 queries for initial testing\n- **Caching**: Enable analytics caching for repeated analyses\n- **LLM Timeout**: Adjust timeout based on model size and hardware\n- **Memory Usage**: Monitor system memory with large query datasets\n\n## Development\n\n### Setting up Development Environment\n\n1. **Install development dependencies:**\n   ```bash\n   pip install pytest pylint black mypy\n   ```\n\n2. **Run code quality checks:**\n   ```bash\n   # Linting\n   pylint pihole_analytics/\n   \n   # Type checking\n   mypy pihole_analytics/\n   \n   # Code formatting\n   black pihole_analytics/\n   ```\n\n3. **Run tests:**\n   ```bash\n   pytest tests/\n   ```\n\n### Contributing\n\n1. Fork the repository\n2. Create a feature branch: `git checkout -b feature-name`\n3. Make your changes with proper testing\n4. Ensure code quality: `pylint`, `mypy`, `black`\n5. Submit a pull request with detailed description\n\n### API Design\n\nThe application follows clean architecture principles:\n\n- **Separation of Concerns**: Each module has a single responsibility\n- **Dependency Injection**: Configuration and clients are injected\n- **Type Safety**: Comprehensive type hints and validation\n- **Error Handling**: Structured error handling with proper logging\n- **Extensibility**: Easy to add new analyzers and detectors\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Acknowledgments\n\n- **Pi-hole Project**: For providing excellent DNS filtering capabilities\n- **Ollama**: For local LLM hosting and management\n- **Python Community**: For robust libraries and development tools\n\n## References\n\n### Pi-hole API Documentation\n\n- **Pi-hole FTL API Specification**: [Official OpenAPI specification](https://github.com/pi-hole/FTL/blob/bc185680fc2af2f7e21bd120f56749051207914f/src/api/docs/content/specs/main.yaml#L4) - Complete reference for Pi-hole v6+ API endpoints and authentication methods\n\n### Related Documentation\n\n- **Pi-hole Documentation**: [docs.pi-hole.net](https://docs.pi-hole.net/)\n- **DNS Security Best Practices**: Industry standards for DNS monitoring and threat detection\n- **LLM Integration Patterns**: Architectural patterns for AI-powered network analysis\n\n## Support\n\nFor questions, issues, or contributions:\n\n1. **Issues**: Use GitHub Issues for bug reports and feature requests\n2. **Documentation**: Check the comprehensive examples and code comments\n3. **Community**: Join discussions in the repository discussions section\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fliteobject%2Fpihole-llm-analytics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fliteobject%2Fpihole-llm-analytics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fliteobject%2Fpihole-llm-analytics/lists"}