{"id":26264171,"url":"https://github.com/takara-ai/go-attention","last_synced_at":"2026-03-11T02:02:27.609Z","repository":{"id":276819814,"uuid":"930415879","full_name":"takara-ai/go-attention","owner":"takara-ai","description":"A full attention mechanism and transformer in pure go.","archived":false,"fork":false,"pushed_at":"2025-07-18T09:43:19.000Z","size":574,"stargazers_count":451,"open_issues_count":0,"forks_count":15,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-12-17T17:48:30.054Z","etag":null,"topics":["artificial-intelligence","golang","machine-learning","serverless"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/takara-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-02-10T15:46:35.000Z","updated_at":"2025-10-12T20:43:51.000Z","dependencies_parsed_at":null,"dependency_job_id":"a37624c8-a13c-4467-877e-8721cde59e04","html_url":"https://github.com/takara-ai/go-attention","commit_stats":null,"previous_names":["takara-ai/go-attention"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/takara-ai/go-attention","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/takara-ai%2Fgo-attention","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/takara-ai%2Fgo-attention/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/takara-ai%2Fgo-attention/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/takara-ai%2Fgo-attention/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/takara-ai","download_url":"https://codeload.github.com/takara-ai/go-attention/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/takara-ai%2Fgo-attention/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30367799,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-10T21:41:54.280Z","status":"online","status_checked_at":"2026-03-11T02:00:07.027Z","response_time":84,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","golang","machine-learning","serverless"],"created_at":"2025-03-14T02:01:08.625Z","updated_at":"2026-03-11T02:02:27.600Z","avatar_url":"https://github.com/takara-ai.png","language":"Go","funding_links":[],"categories":["Large Language Model"],"sub_categories":["DevTools"],"readme":"# go-attention\n\n\u003cimg src=\"https://takara.ai/images/logo-24/TakaraAi.svg\" width=\"200\" alt=\"Takara.ai Logo\" /\u003e\n\nFrom the Frontier Research Team at takara.ai we present the first pure Go implementation of attention mechanisms and transformer layers, designed for high performance, consistency, and production reliability.\n\n## Why Go for Attention Mechanisms?\n\n### **Performance Without Compromise**\nThis implementation proves that Go can deliver **production-grade performance** for AI workloads:\n\n- **Consistent, Predictable Performance**: Single optimized code path ensures repeatable results across all input sizes\n- **Edge-Optimized**: Zero external dependencies and minimal memory footprint perfect for edge devices\n- **Production-Ready**: Comprehensive error handling, type safety, and deterministic behavior\n- **Scalable**: Efficient batched operations support high-throughput cloud deployments\n\n### **Addressing Go's AI Limitations**\nWe've solved the common concerns about Go in AI/ML:\n\n- **SIMD Support**: Assembly-optimized critical paths with automatic fallbacks\n- **Memory Efficiency**: Object pools and optimized allocations reduce GC pressure\n- **Parallel Performance**: Goroutine-based parallelization for multi-core systems\n- **Numerical Stability**: Robust floating-point operations with proper error handling\n\n### **Real-World Benefits**\n- **Zero Cold Starts**: Pure Go implementation eliminates dependency resolution delays\n- **Predictable Latency**: Consistent performance characteristics across all hardware\n- **Easy Deployment**: Single binary with no external dependencies\n- **Cost Effective**: Efficient resource usage reduces cloud costs\n\n## Quick Start\n\nRun our comprehensive examples:\n\n```bash\n# Get the module\ngo get github.com/takara-ai/go-attention\n\n# Run the examples\ngo run api_examples.go\n```\n\n## Performance Characteristics\n\n### **Consistent Performance Across All Sizes**\nOur implementation uses a single, highly optimized code path that delivers predictable performance:\n\n```go\n// Always fast, always consistent - no unpredictable performance cliffs\nresult, err := attention.DotProduct(v1, v2)\n```\n\n**Performance Results:**\n- **Small vectors (64-256)**: ~30-100ns per operation\n- **Medium vectors (512-1024)**: ~400-1700ns per operation  \n- **Large vectors (2048+)**: ~1600ns+ per operation\n- **Consistent across all hardware**: Same performance characteristics on any Go-compatible platform\n\n### **Production-Grade Reliability**\n- **Deterministic Results**: Same input always produces same output\n- **Memory Safe**: No buffer overflows or memory corruption\n- **Error Handling**: Comprehensive validation and error reporting\n- **Type Safe**: Compile-time guarantees prevent runtime errors\n\n## API Documentation\n\nFor complete API documentation, see [API.md](API.md).\n\n### Core Types\n\n```go\ntype Vector []float64           // Represents a 1D vector of float64 values\ntype Matrix []Vector           // Represents a 2D matrix of float64 values\n```\n\n### Quick Examples\n\n#### 1. Basic Dot-Product Attention\n\nThe simplest form of attention mechanism. Useful for basic sequence processing tasks.\n\n```go\nimport \"github.com/takara-ai/go-attention/attention\"\n\n// Create query-key-value setup\nquery := attention.Vector{1.0, 0.0, 1.0, 0.0}  // Pattern to search for\nkeys := attention.Matrix{\n    {1.0, 0.0, 1.0, 0.0},  // Similar to query\n    {0.0, 1.0, 0.0, 1.0},  // Different from query\n    {0.5, 0.5, 0.5, 0.5},  // Neutral pattern\n}\nvalues := attention.Matrix{\n    {1.0, 2.0},  // Value for similar key\n    {3.0, 4.0},  // Value for different key\n    {5.0, 6.0},  // Value for neutral key\n}\n\n// Compute attention\noutput, weights, err := attention.DotProductAttention(query, keys, values)\nif err != nil {\n    log.Fatal(err)\n}\n\n// Output will be a weighted combination of values based on query-key similarity\n// Weights will show how much attention each key received\n```\n\n#### 2. Multi-Head Attention\n\nMore sophisticated attention mechanism that can capture different types of relationships in parallel.\n\n```go\nimport \"github.com/takara-ai/go-attention/attention\"\n\n// Configure multi-head attention\nconfig := attention.MultiHeadConfig{\n    NumHeads:    4,        // Number of parallel attention heads\n    DModel:      64,       // Size of input/output embeddings\n    DKey:        16,       // Size per head (DModel/NumHeads)\n    DValue:      16,       // Size per head (DModel/NumHeads)\n    DropoutRate: 0.1,      // For regularization\n}\n\n// Create the attention module\nmha, err := attention.NewMultiHeadAttention(config)\nif err != nil {\n    log.Fatal(err)\n}\n\n// Process sequences (batched input)\nbatchSize, seqLen := 2, 3  // Process 2 sequences, each with 3 tokens\n\n// Create input matrices [batchSize × seqLen × DModel]\nqueries := make(attention.Matrix, batchSize*seqLen)\nkeys := make(attention.Matrix, batchSize*seqLen)\nvalues := make(attention.Matrix, batchSize*seqLen)\n\n// Initialize your matrices with actual data...\n\n// Process through multi-head attention\noutput, err := mha.Forward(queries, keys, values)\nif err != nil {\n    log.Fatal(err)\n}\n```\n\n#### 3. Full Transformer Layer\n\nComplete transformer layer with self-attention and feed-forward network.\n\n```go\nimport (\n    \"github.com/takara-ai/go-attention/transformer\"\n    \"github.com/takara-ai/go-attention/attention\"\n)\n\n// Configure transformer layer\nconfig := transformer.TransformerConfig{\n    DModel:      64,       // Size of token embeddings\n    NumHeads:    4,        // Number of attention heads\n    DHidden:     256,      // Size of feed-forward hidden layer\n    DropoutRate: 0.1,      // For regularization\n}\n\n// Create transformer layer\nlayer, err := transformer.NewTransformerLayer(config)\nif err != nil {\n    log.Fatal(err)\n}\n\n// Create input sequence [seq_len × d_model]\nseqLen := 3\ninput := make(attention.Matrix, seqLen)\nfor i := range input {\n    input[i] = make(attention.Vector, config.DModel)\n    // Fill with your embedding data...\n}\n\n// Process through transformer\noutput, err := layer.Forward(input)\nif err != nil {\n    log.Fatal(err)\n}\n```\n\n## Example Output\n\nWhen running the examples, you'll see:\n\n1. **Dot-Product Attention**:\n\n   ```\n   Query: [1 0 1 0]\n   Attention Weights: [0.523 0.174 0.302]  // Shows focus on similar patterns\n   Output: [2.558 3.558]                   // Weighted combination of values\n   ```\n\n2. **Multi-Head Attention**:\n\n   ```\n   Input dimensions: [2 batches × 3 tokens × 64 features]\n   Output shape: [6×64]\n   ```\n\n3. **Transformer Layer**:\n   ```\n   Input shape: [3×64]\n   Output shape: [3×64]\n   ```\n\n## Common Use Cases\n\n1. **Text Processing**:\n\n   - Sequence-to-sequence translation\n   - Document summarization\n   - Sentiment analysis\n\n2. **Time Series**:\n\n   - Financial forecasting\n   - Sensor data analysis\n   - Anomaly detection\n\n3. **Structured Data**:\n   - Graph node embedding\n   - Feature interaction modeling\n   - Recommendation systems\n\n## Performance Features\n\n### **Built-in Optimizations**\n- **Loop Unrolling**: 8x unrolled dot product for maximum throughput\n- **Memory Pooling**: Object pools reduce allocation overhead in hot paths\n- **Cache-Friendly**: Optimized memory access patterns\n- **Zero Dependencies**: Pure Go implementation with no external requirements\n\n### **Production Monitoring**\n```go\n// Enable performance monitoring and auto-tuning\nconfig := attention.DefaultPerformanceConfig()\nconfig.EnableMonitoring = true\nconfig.EnableAutoTuning = true\nattention.SetPerformanceConfig(config)\n\n// Use memory pools to reduce allocations\nv1 := attention.GetVectorFromPool(size)\ndefer attention.PutVectorToPool(v1)\n\n// Get performance statistics\nstats := attention.GetAllPerformanceStats()\n```\n\n### **Parallel Operations**\n```go\n// Configure parallel processing\nconfig := attention.DefaultParallelConfig()\nconfig.NumWorkers = runtime.NumCPU()\n```\n\n## Why This Go Implementation?\n\n### **Edge Computing Excellence**\n- **Zero Dependencies**: Perfect for edge devices where dependency management is crucial\n- **Predictable Performance**: Consistent latency regardless of input size\n- **Memory Efficient**: Minimal allocations and GC pressure\n- **Easy Deployment**: Single binary deployment\n\n### **Production System Benefits**\n- **Type Safety**: Compile-time guarantees prevent runtime errors\n- **Error Handling**: Comprehensive validation and error reporting\n- **Deterministic**: Same input always produces same output\n- **Scalable**: Efficient batched operations for high throughput\n\n### **Real-time Processing**\n- **Consistent Latency**: No unpredictable performance cliffs\n- **Low Memory Footprint**: Efficient resource usage\n- **Fast Startup**: No dependency resolution delays\n- **Reliable**: Robust error handling and recovery\n\n## Features\n\n- **Efficient Dot-Product Attention**: Upgraded with Scalable-Softmax (SSMax, s=1) for improved long-context performance\n- **Multi-Head Attention**: Parallel attention heads for capturing different relationships\n- **Full Transformer Layer**: Complete implementation with:\n  - Layer normalization\n  - Position-wise feed-forward networks\n  - Residual connections\n- **Batched Operations**: Efficient processing of multiple sequences\n- **Production Monitoring**: Built-in performance tracking and optimization\n\n## Performance Benchmarks\n\nRun comprehensive performance benchmarks:\n\n```bash\ngo test -bench=. ./attention\n```\n\nThis will benchmark all operations and show performance characteristics across different input sizes and hardware configurations.\n\n**Sample Results:**\n```\nBenchmarkDotProduct-8                   154059886                7.747 ns/op\nBenchmarkDotProductAttention-8           6989079               170.7 ns/op\nBenchmarkMultiHeadAttentionForward-8        6178            192052 ns/op\n```\n\n## Roadmap\n\nFuture improvements may include:\n\n- **Real SIMD Assembly**: Actual AVX2/NEON implementations for critical paths\n- **Positional Encoding**: RoPE and other positional encoding methods\n- **Advanced Optimizations**: Flash Attention, Sparse Attention variants\n- **Training Support**: Gradient computation and optimization utilities\n- **Model Export**: ONNX and other format support\n- **GPU Acceleration**: CUDA/OpenCL backends for GPU computation\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nMIT License - see LICENSE file for details\n\n---\n\nFor research inquiries and press, please reach out to research@takara.ai\n\n\u003e 人類を変革する\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftakara-ai%2Fgo-attention","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftakara-ai%2Fgo-attention","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftakara-ai%2Fgo-attention/lists"}