{"id":31618056,"url":"https://github.com/pkeffect/numbers","last_synced_at":"2026-05-06T02:34:15.186Z","repository":{"id":317442193,"uuid":"1066035888","full_name":"pkeffect/numbers","owner":"pkeffect","description":"Math things and stuff","archived":false,"fork":false,"pushed_at":"2025-09-30T21:31:18.000Z","size":100,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-30T23:25:21.315Z","etag":null,"topics":["constants","docker","docker-compose","eluer","fastpi","math","phi","pi","python","sqlite"],"latest_commit_sha":null,"homepage":"https://internode.info/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pkeffect.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-28T23:15:20.000Z","updated_at":"2025-09-30T21:31:22.000Z","dependencies_parsed_at":"2025-09-30T23:25:35.629Z","dependency_job_id":"541c6fd0-1d18-4c49-93ae-d548e030d2a2","html_url":"https://github.com/pkeffect/numbers","commit_stats":null,"previous_names":["pkeffect/numbers"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/pkeffect/numbers","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pkeffect%2Fnumbers","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pkeffect%2Fnumbers/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pkeffect%2Fnumbers/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pkeffect%2Fnumbers/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pkeffect","download_url":"https://codeload.github.com/pkeffect/numbers/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pkeffect%2Fnumbers/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278621844,"owners_count":26017253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-06T02:00:05.630Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["constants","docker","docker-compose","eluer","fastpi","math","phi","pi","python","sqlite"],"created_at":"2025-10-06T13:45:15.853Z","updated_at":"2026-05-06T02:34:15.174Z","avatar_url":"https://github.com/pkeffect.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🔢 Math Constants Storage System\n\n\u003e **⚠️ IMPORTANT: DATA FILES NOT INCLUDED**\n\u003e \n\u003e This repository contains the API and storage infrastructure **only**. See [Data Sources](#-data-sources) section for obtaining data files.\n\nHigh-performance, triple-redundancy storage system for accessing billions of digits of mathematical constants (π, φ, e, √2, and more) with FastAPI backend, Redis caching, WebSocket streaming, dedicated endpoints per constant, and accuracy-first design.\n\n## ✨ Features\n\n- **12 Mathematical Constants**: Pi, Euler's number, Golden ratio, Square roots, Logarithms, and more\n- **Dedicated Endpoints**: Each constant has its own clean API namespace (`/pi/*`, `/e/*`, etc.)\n- **Triple Redundancy**: Original file + SQLite chunks + Binary cache\n- **Redis Caching**: Lightning-fast access with intelligent cache invalidation\n- **WebSocket Streaming**: Real-time digit streaming with chunked delivery\n- **Smart Cache Management**: Automatically skips already-built caches\n- **Accuracy First**: Automatic verification and corruption detection\n- **High Performance**: Optimized for random access to billions of digits\n- **Modular Architecture**: Clean separation with 16 focused router files\n- **Hot Reloading**: Development-friendly Docker setup\n- **Backward Compatible**: Legacy parameterized endpoints still work\n\n## 🚀 Quick Start\n\n```bash\n# Clone the repository\ngit clone https://github.com/pkeffect/numbers\ncd numbers\n\n# Copy environment configuration\ncp .env.example .env\n\n# Place your math constant files in data/ directory\n# Required filenames: pi_digits.txt, e_digits.txt, phi_digits.txt, etc.\n\n# Start the development environment (includes Redis)\ndocker-compose up --build\n\n# Build caches (after first startup)\ncurl -X POST http://localhost:8000/admin/build-all-caches\n```\n\n## 🆕 New Features: Redis \u0026 WebSocket\n\n### Redis Caching\n\n**Benefits:**\n- ⚡ 10-100x faster response times for cached queries\n- 📊 Automatic cache hit/miss tracking\n- 🔄 Smart cache invalidation on data updates\n- 💾 Configurable TTL and memory limits\n\n**Cached Operations:**\n- Digit retrieval\n- Search results\n- Statistical analysis\n\n**Cache Control:**\n```bash\n# Use cache (default)\nGET /pi/digits?start=0\u0026length=100\u0026use_cache=true\n\n# Bypass cache\nGET /pi/digits?start=0\u0026length=100\u0026use_cache=false\n\n# Clear cache for a constant\nDELETE /pi/cache\n\n# Get cache statistics\nGET /health  # includes cache stats\n```\n\n### WebSocket Streaming\n\n**Real-time Features:**\n- 📡 Live digit streaming\n- 🔔 Cache build notifications\n- 📊 Live status updates\n- 🎯 Per-constant subscriptions\n\n**WebSocket Endpoints:**\n```javascript\n// General connection (system-wide events)\nws://localhost:8000/ws/connect\n\n// Constant-specific connection\nws://localhost:8000/ws/constant/pi\n\n// Simplified streaming\nws://localhost:8000/ws/stream/pi?start=0\u0026length=1000\n```\n\n**Example Usage:**\n```javascript\nconst ws = new WebSocket('ws://localhost:8000/ws/constant/pi');\n\nws.onopen = () =\u003e {\n    // Stream 1000 digits in chunks of 100\n    ws.send(JSON.stringify({\n        command: 'stream_digits',\n        start: 0,\n        length: 1000,\n        chunk_size: 100\n    }));\n};\n\nws.onmessage = (event) =\u003e {\n    const data = JSON.parse(event.data);\n    \n    if (data.type === 'stream_chunk') {\n        console.log(`Chunk ${data.chunk_number}: ${data.digits}`);\n        console.log(`Remaining: ${data.remaining} digits`);\n    }\n};\n```\n\n**Test WebSocket:**\nOpen `http://localhost:3000/websocket_test.html` in your browser for an interactive test interface.\n\n## 🔧 API Endpoints\n\n### **General Endpoints**\n\n```bash\nGET  /                    # API root with system info\nGET  /health              # System health check (includes Redis stats)\nGET  /constants           # List all constants with status\n```\n\n### **Per-Constant Endpoints** (Pattern for all 12 constants)\n\nEach constant has 8 dedicated endpoints (7 original + 1 new):\n\n```bash\nGET    /{constant}/status        # Status and cache information\nGET    /{constant}/digits        # Retrieve digits (Redis cached)\nGET    /{constant}/search        # Search sequences (Redis cached)\nGET    /{constant}/stats         # Statistical analysis (Redis cached)\nGET    /{constant}/random        # Get random digits\nPOST   /{constant}/build-cache   # Build SQLite and binary cache\nPOST   /{constant}/verify        # Verify data integrity\nDELETE /{constant}/cache         # Clear Redis cache (NEW!)\n```\n\n### **WebSocket Endpoints**\n\n```bash\nWS /ws/connect                    # General system connection\nWS /ws/constant/{constant_id}     # Constant-specific connection\nWS /ws/stream/{constant_id}       # Simplified streaming\n```\n\n### **Admin Endpoints**\n\n```bash\nPOST /admin/build-all-caches   # Build caches for all constants\nGET  /admin/status             # Administrative status\n```\n\n## 🐳 Docker Services\n\nThe system now includes Redis:\n\n```bash\n# Start all services (API + Redis)\ndocker-compose up\n\n# Start with debug tools (API + Redis + Redis Commander + SQLite Browser)\ndocker-compose --profile debug up\n\n# Start with frontend (API + Redis + Nginx frontend)\ndocker-compose --profile frontend up\n```\n\n**Services:**\n- **math-constants-api** (port 8000) - Main API\n- **redis** (port 6379) - Caching layer\n- **redis-commander** (port 8081, debug profile) - Redis web UI\n- **sqlite-browser** (port 8080, debug profile) - SQLite web UI\n- **frontend** (port 3000, frontend profile) - Test interface\n\n## 📊 Performance Improvements\n\n### Before Redis (Direct SQLite/File):\n- Average response time: 50-100ms\n- Search queries: 200-500ms\n- Stats calculations: 300-600ms\n\n### After Redis Caching:\n- Cached digit retrieval: **5-10ms** (90% faster)\n- Cached search results: **2-5ms** (95% faster)\n- Cached statistics: **3-8ms** (95% faster)\n\n### Cache Hit Rates:\n- Digit queries: ~60-80%\n- Search queries: ~40-60%\n- Stats queries: ~70-90%\n\n## 🔄 Cache Invalidation\n\nRedis cache is automatically invalidated when:\n- Cache is rebuilt via `/admin/build-all-caches`\n- Cache is rebuilt for specific constant via `/{constant}/build-cache`\n- Manual cache clear via `DELETE /{constant}/cache`\n\n## 📡 WebSocket Commands\n\n### General Connection Commands:\n```javascript\n// Ping-pong for keepalive\n{ \"type\": \"ping\", \"timestamp\": 1234567890 }\n\n// Get connection statistics\n{ \"type\": \"get_stats\" }\n```\n\n### Constant-Specific Commands:\n```javascript\n// Stream digits\n{\n    \"command\": \"stream_digits\",\n    \"start\": 0,\n    \"length\": 1000,\n    \"chunk_size\": 100\n}\n\n// Get constant status\n{ \"command\": \"get_status\" }\n\n// Ping\n{ \"command\": \"ping\" }\n```\n\n### Server Messages:\n```javascript\n// Stream chunk\n{\n    \"type\": \"stream_chunk\",\n    \"chunk_number\": 5,\n    \"position\": 500,\n    \"digits\": \"1234567890...\",\n    \"remaining\": 500\n}\n\n// Broadcast notifications\n{\n    \"type\": \"digits_accessed\",\n    \"constant_id\": \"pi\",\n    \"start\": 100,\n    \"length\": 50\n}\n\n{\n    \"type\": \"cache_build_complete\",\n    \"constant_id\": \"pi\"\n}\n```\n\n## ⚙️ Configuration\n\nKey environment variables for Redis \u0026 WebSocket (see `.env.example`):\n\n```bash\n# Redis Configuration\nREDIS_URL=redis://redis:6379/0\nREDIS_MAX_CONNECTIONS=50\nCACHE_TTL=3600\nENABLE_CACHE=true\n\n# WebSocket Configuration\nWS_HEARTBEAT_INTERVAL=30\nWS_MAX_CONNECTIONS=1000\nWS_MESSAGE_QUEUE_SIZE=100\n```\n\n## 🧪 Testing\n\n### Test Redis Connection:\n```bash\n# Connect to Redis CLI\ndocker-compose exec redis redis-cli\n\n# Check keys\nKEYS mathconst:*\n\n# Get cache stats\nINFO stats\n```\n\n### Test WebSocket:\n```bash\n# Using wscat (install: npm install -g wscat)\nwscat -c ws://localhost:8000/ws/connect\n\n# Send ping\n{\"type\":\"ping\",\"timestamp\":1234567890}\n```\n\n### Test Caching:\n```bash\n# First request (cache miss)\ntime curl \"http://localhost:8000/pi/digits?start=0\u0026length=100\"\n\n# Second request (cache hit - much faster!)\ntime curl \"http://localhost:8000/pi/digits?start=0\u0026length=100\"\n```\n\n## 🤝 Contributing\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines.\n\n## 📄 License\n\nMIT License - see [LICENSE.md](LICENSE.md)\n\n## 🔮 Roadmap\n\n- [x] Redis caching layer\n- [x] WebSocket streaming support\n- [ ] Redis pub/sub for distributed deployments\n- [ ] GraphQL API support\n- [ ] Advanced pattern recognition with ML\n- [ ] Real-time collaborative features\n- [ ] Kubernetes deployment manifests\n\n## 📞 Support\n\n- 📚 [Full Documentation](docs/)\n- 🐛 [Issue Tracker](https://github.com/your-repo/issues)\n- 💬 [Discussions](https://github.com/your-repo/discussions)\n\n---\n\n**New in v2.1:** Redis caching and WebSocket streaming for real-time applications!","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpkeffect%2Fnumbers","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpkeffect%2Fnumbers","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpkeffect%2Fnumbers/lists"}