{"id":21166860,"url":"https://github.com/agora-lab-ai/neocore","last_synced_at":"2025-07-09T17:33:05.042Z","repository":{"id":262643100,"uuid":"887905146","full_name":"Agora-Lab-AI/NeoCore","owner":"Agora-Lab-AI","description":"NeoCore™ - Next Generation CPU-Native Transformer. ","archived":false,"fork":false,"pushed_at":"2024-11-13T13:44:52.000Z","size":0,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-11-13T14:34:49.332Z","etag":null,"topics":["ai","cpu-native","gpt1000","gpt5","gpts","ml","noam-gpt","transformer","transformers"],"latest_commit_sha":null,"homepage":"https://swarms.xyz/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Agora-Lab-AI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["kyegomez"]}},"created_at":"2024-11-13T13:40:47.000Z","updated_at":"2024-11-13T13:53:50.000Z","dependencies_parsed_at":"2024-11-14T18:00:33.603Z","dependency_job_id":null,"html_url":"https://github.com/Agora-Lab-AI/NeoCore","commit_stats":null,"previous_names":["agora-lab-ai/neocore"],"tags_count":0,"template":false,"template_full_name":"The-Swarm-Corporation/Swarms-Example-1-Click-Template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Agora-Lab-AI%2FNeoCore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Agora-Lab-AI%2FNeoCore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Agora-Lab-AI%2FNeoCore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Agora-Lab-AI%2FNeoCore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Agora-Lab-AI","download_url":"https://codeload.github.com/Agora-Lab-AI/NeoCore/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225579018,"owners_count":17491280,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","cpu-native","gpt1000","gpt5","gpts","ml","noam-gpt","transformer","transformers"],"created_at":"2024-11-20T14:53:25.541Z","updated_at":"2024-11-20T14:53:26.143Z","avatar_url":"https://github.com/Agora-Lab-AI.png","language":"Python","funding_links":["https://github.com/sponsors/kyegomez"],"categories":[],"sub_categories":[],"readme":"# NeoCore™ - Next Generation CPU-Native Transformer\n\n\n[![Join our Discord](https://img.shields.io/badge/Discord-Join%20our%20server-5865F2?style=for-the-badge\u0026logo=discord\u0026logoColor=white)](https://discord.gg/agora-999382051935506503) [![Subscribe on YouTube](https://img.shields.io/badge/YouTube-Subscribe-red?style=for-the-badge\u0026logo=youtube\u0026logoColor=white)](https://www.youtube.com/@kyegomez3242) [![Connect on LinkedIn](https://img.shields.io/badge/LinkedIn-Connect-blue?style=for-the-badge\u0026logo=linkedin\u0026logoColor=white)](https://www.linkedin.com/in/kye-g-38759a207/) [![Follow on X.com](https://img.shields.io/badge/X.com-Follow-1DA1F2?style=for-the-badge\u0026logo=x\u0026logoColor=white)](https://x.com/kyegomezb)\n\n\n[![GitHub stars](https://img.shields.io/github/stars/The-Swarm-Corporation/Legal-Swarm-Template?style=social)](https://github.com/The-Swarm-Corporation/Legal-Swarm-Template)\n[![Swarms Framework](https://img.shields.io/badge/Built%20with-Swarms-blue)](https://github.com/kyegomez/swarms)\n\n\n[![PyPI version](https://badge.fury.io/py/neocore.svg)](https://badge.fury.io/py/neocore)\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)\n\n## 🚀 Overview\n\nNeoCore is a state-of-the-art, CPU-optimized transformer architecture designed for edge computing and enterprise deployment. By leveraging advanced CPU-specific optimizations and modern architectural improvements, NeoCore achieves exceptional performance without requiring GPU acceleration.\n\n### Key Features\n\n- 🔋 **CPU-Native Design**: Optimized from the ground up for modern CPU architectures\n- 🎯 **Memory Efficient**: Advanced caching and chunking strategies for optimal memory usage\n- 🛠 **Enterprise Ready**: Production-grade implementation with comprehensive logging and monitoring\n- 🔄 **Modern Architecture**: Incorporates Multi-Query Attention, RMSNorm, and Rotary Embeddings\n- 📊 **Extensive Benchmarking**: Built-in performance profiling and optimization tools\n\n## 🔧 Installation\n\n```bash\npip install neocore\n```\n\n## 🏗 Architecture\n\nNeoCore introduces several architectural innovations:\n\n### Core Components\n\n1. **Multi-Query Attention (MQA)**\n```python\nQ: [Batch, Seq, Heads, Head_Dim]  # Multiple query heads\nK,V: [Batch, 1, Head_Dim]         # Single key/value\n```\n\n2. **RMSNorm for Stabilization**\n```python\nRMSNorm(x) = x * scale / sqrt(mean(x²) + ε)\n```\n\n3. **Block-wise Computation**\n```\nInput -\u003e Chunked Processing -\u003e Cache-Friendly Operations -\u003e Output\n```\n\n### Performance Optimizations\n\n#### Memory Access Pattern\n```\n┌──────────────────┐\n│ Input Embedding  │\n└────────┬─────────┘\n         │\n    ┌────▼────┐\n    │ Chunk 1 │──┐\n    └─────────┘  │\n    ┌─────────┐  │\n    │ Chunk 2 │──┼─► Parallel Processing\n    └─────────┘  │\n    ┌─────────┐  │\n    │ Chunk N │──┘\n    └─────────┘\n```\n\n## 💫 Key Innovations\n\n### 1. Cache-Optimized Linear Operations\n- Custom blocked matrix multiplication\n- Adaptive chunk sizing\n- Operation result caching\n\n### 2. Efficient Attention Mechanism\n```python\n# Traditional vs NeoCore MQA\nTraditional: O(N * H * D) memory\nNeoCore:     O(N * D) memory\n```\n\n### 3. Advanced Position Encoding\n- Rotary embeddings for enhanced position awareness\n- Cache-friendly implementation\n- Optimized for CPU SIMD operations\n\n## 📊 Performance Metrics\n\n| Batch Size | Sequence Length | Processing Time (ms) | Tokens/Second |\n|------------|----------------|---------------------|---------------|\n| 1          | 32             | 31.17              | 1,026        |\n| 4          | 64             | 43.51              | 5,883        |\n| 16         | 128            | 161.28             | 12,700       |\n\n## 🚀 Quick Start\n\n```python\nfrom neocore import NoamConfig, CPUOptimizedNoamTransformer\n\n# Initialize configuration\nconfig = NoamConfig(\n    d_model=512,\n    n_heads=8,\n    n_layers=6,\n    warmup_steps=4000,\n    chunk_size=32\n)\n\n# Create model\nmodel = CPUOptimizedNoamTransformer(config)\n\n# Process input\noutput = model(input_ids)\n```\n\n\n## 🎯 Use Cases\n\n- **Edge Computing**: Optimal for deployment on CPU-only edge devices\n- **Enterprise Systems**: Reliable performance on standard server hardware\n- **CI/CD Pipelines**: Efficient inference in production pipelines\n- **Privacy-First Applications**: On-device processing without GPU requirements\n\n## 🔬 Technical Details\n\n### Memory Management\n- Intelligent cache management system\n- Adaptive chunk sizing based on input\n- Memory-efficient attention patterns\n\n### Threading Model\n```python\nNumber of Threads = min(CPU_COUNT, MAX_EFFICIENT_THREADS)\nThread Pool Size = Adaptive based on workload\n```\n\n### Optimization Levels\n1. **Level 1**: Basic CPU optimizations\n2. **Level 2**: Cache-aware operations\n3. **Level 3**: Advanced parallelization\n4. **Level 4**: Full SIMD utilization\n\n## 📈 Benchmarking\n\nRun comprehensive benchmarks:\n```bash\npython -m neocore.benchmark --config benchmark_config.yaml\n```\n\n## 🤝 Contributing\n\nWe welcome contributions! Please see our [Contributing Guidelines](CONTRIBUTING.md) for details.\n\n## 📜 License\n\nApache License 2.0. See [LICENSE](LICENSE) for details.\n\n## 🌟 Acknowledgments\n\nBuilt on modern transformer innovations with specific optimizations for CPU architectures. Special thanks to the research community for their groundbreaking work in efficient transformer designs.\n\n---\n\n## Citation\n\n```bibtex\n@software{neocore2024,\n  title={NeoCore: CPU-Optimized Transformer Architecture},\n  author={Kye Gomez},\n  year={2024},\n  publisher={GitHub},\n  url={https://github.com/neocore/neocore}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagora-lab-ai%2Fneocore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fagora-lab-ai%2Fneocore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagora-lab-ai%2Fneocore/lists"}