{"id":42552869,"url":"https://github.com/thinmanj/python-optimizer","last_synced_at":"2026-01-28T19:22:07.984Z","repository":{"id":317634795,"uuid":"1068205129","full_name":"thinmanj/python-optimizer","owner":"thinmanj","description":null,"archived":false,"fork":false,"pushed_at":"2025-11-10T12:11:26.000Z","size":437,"stargazers_count":0,"open_issues_count":3,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-10T07:43:08.450Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thinmanj.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-02T02:34:05.000Z","updated_at":"2025-11-05T23:29:21.000Z","dependencies_parsed_at":"2025-10-07T16:32:48.194Z","dependency_job_id":null,"html_url":"https://github.com/thinmanj/python-optimizer","commit_stats":null,"previous_names":["thinmanj/python-optimizer"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/thinmanj/python-optimizer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thinmanj%2Fpython-optimizer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thinmanj%2Fpython-optimizer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thinmanj%2Fpython-optimizer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thinmanj%2Fpython-optimizer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thinmanj","download_url":"https://codeload.github.com/thinmanj/python-optimizer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thinmanj%2Fpython-optimizer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28849835,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-28T15:15:36.453Z","status":"ssl_error","status_checked_at":"2026-01-28T15:15:13.020Z","response_time":57,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-28T19:22:07.919Z","updated_at":"2026-01-28T19:22:07.978Z","avatar_url":"https://github.com/thinmanj.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Python Optimizer 🚀\n\nA high-performance Python optimization toolkit that provides JIT compilation, advanced variable specialization, intelligent caching, and runtime optimizations to accelerate Python code execution without changing language syntax.\n\n[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)\n[![Numba](https://img.shields.io/badge/numba-JIT%20compilation-orange.svg)](http://numba.pydata.org/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Performance](https://img.shields.io/badge/performance-up%20to%20500x-brightgreen.svg)]()\n[![Thread Safe](https://img.shields.io/badge/thread-safe-blue.svg)]()\n\n## 🎯 Goal\n\nAccelerate Python program execution by **10-500x** through:\n- **Advanced JIT compilation** with Numba and custom optimizations\n- **GPU acceleration** with automatic CPU/GPU dispatching (CUDA/CuPy)\n- **Intelligent variable specialization** with type-aware caching  \n- **Adaptive optimization** based on runtime patterns\n- **Specialization caching** with smart memory management\n- **Zero syntax changes** - works with existing Python code\n\n## ⚡ Performance Results\n\nReal-world performance improvements achieved:\n\n| Function Type | Original Time | Optimized Time | Speedup | Cache Hit Rate |\n|---------------|---------------|----------------|---------|----------------|\n| Numerical Computation | 2.06ms | 0.04ms | **51x** | 95% |\n| Financial Metrics | 100ms | 2ms | **50x** | 88% |\n| Trading Simulation | 500ms | 5ms | **100x** | 92% |\n| Genetic Algorithm | 30s | 0.14s | **214x** | - |\n| Specialized Functions | 1.2ms | 0.003ms | **400x** | 97% |\n| Array Operations | 50ms | 0.1ms | **500x** | 91% |\n\n**Throughput:** Up to **36,456 evaluations/second**\n**Cache Efficiency:** 90%+ hit rates with intelligent eviction\n\n## 🛠 Installation\n\n```bash\n# Clone the repository\ngit clone https://github.com/thinmanj/python-optimizer.git\ncd python-optimizer\n\n# Install with pip\npip install -e .\n\n# Or install from PyPI (coming soon)\npip install python-optimizer\n```\n\n## 🚀 Quick Start\n\n### 1. Basic JIT Optimization\n\n```python\nfrom python_optimizer import optimize\n\n@optimize(jit=True)\ndef fibonacci(n):\n    if n \u003c= 1:\n        return n\n    return fibonacci(n-1) + fibonacci(n-2)\n\n# First call compiles, subsequent calls are blazing fast\nresult = fibonacci(35)  # ~100x faster after compilation\n```\n\n### 2. Variable Specialization\n\n```python\nfrom python_optimizer import optimize\n\n@optimize(specialize=True, jit=False)\ndef adaptive_function(data):\n    if isinstance(data, list):\n        return sum(data)\n    elif hasattr(data, '__len__'):\n        return len(data)\n    return data\n\n# Automatically creates specialized versions for different types\nresult1 = adaptive_function([1, 2, 3, 4])      # List specialization\nresult2 = adaptive_function(\"hello world\")      # String specialization  \nresult3 = adaptive_function(range(100))         # Range specialization\n# Each type gets its own optimized version cached for future use\n```\n\n### 3. Financial Computing Example\n\n```python\nimport numpy as np\nfrom python_optimizer.jit import calculate_sharpe_ratio_jit\n\n# JIT-compiled financial metrics\nreturns = np.random.normal(0.001, 0.02, 252)  # Daily returns\nsharpe = calculate_sharpe_ratio_jit(returns)   # ~50x faster\n```\n\n### 4. Trading Strategy Optimization\n\n```python\nfrom python_optimizer.jit import JITBacktestFitnessEvaluator\nfrom python_optimizer.genetic import Individual\n\n# Ultra-fast backtesting with JIT compilation\nevaluator = JITBacktestFitnessEvaluator(initial_cash=10000)\nindividual = Individual(genes={'ma_short': 10, 'ma_long': 30})\n\n# Evaluate strategy performance\nmetrics = evaluator.evaluate(individual, market_data)\n# Achieves 36,000+ evaluations per second\n```\n\n### 5. GPU Acceleration (NEW! 🎉)\n\n```python\nfrom python_optimizer import optimize, is_gpu_available\nimport numpy as np\n\n# Check GPU availability\nif is_gpu_available():\n    print(\"GPU acceleration available!\")\n\n# Enable GPU acceleration with automatic CPU/GPU dispatching\n@optimize(gpu=True, gpu_min_size=10_000, jit=False)\ndef gpu_compute(data):\n    return data ** 2 + data * 3\n\n# Automatically uses GPU for large arrays, CPU for small ones\nsmall_array = np.random.randn(1000)       # Uses CPU\nlarge_array = np.random.randn(1_000_000)  # Uses GPU\n\nresult1 = gpu_compute(small_array)\nresult2 = gpu_compute(large_array)  # 5-20x faster on GPU!\n```\n\n### 6. Advanced Caching \u0026 Monitoring\n\n```python\nfrom python_optimizer import (\n    get_specialization_stats, \n    clear_specialization_cache,\n    configure_specialization,\n    get_gpu_info,\n    get_gpu_memory_info\n)\n\n# Configure specialization behavior\nconfigure_specialization(\n    min_calls_for_specialization=3,\n    enable_adaptive_learning=True,\n    max_cache_size=1000\n)\n\n# Monitor performance\nstats = get_specialization_stats()\nprint(f\"Cache hit rate: {stats.get('cache_hit_rate', 0):.1%}\")\nprint(f\"Specializations created: {stats.get('specializations_created', 0)}\")\n\n# Check GPU status\nif is_gpu_available():\n    gpu_info = get_gpu_info()\n    mem_info = get_gpu_memory_info()\n    print(f\"GPU: {gpu_info['devices'][0]['name']}\")\n    print(f\"Memory: {mem_info.used_gb:.1f}/{mem_info.total_gb:.1f} GB\")\n```\n\n## 📦 Features\n\n### Advanced JIT Compilation Engine\n- **Numba-powered** JIT compilation for numerical code\n- **Automatic type inference** and optimization\n- **GIL-free execution** for parallel processing\n- **Intelligent caching** system for compiled functions\n- **Custom optimization passes** for domain-specific code\n\n### Intelligent Variable Specialization\n- **Type-aware specialization** with automatic detection\n- **Adaptive learning** from runtime patterns\n- **Memory-efficient** specialized code paths\n- **Multi-level caching** with eviction policies\n- **Thread-safe** specialization cache\n- **Performance monitoring** and analytics\n\n### GPU Acceleration (Phase 1 - NEW! 🚀)\n- **Automatic GPU detection** with graceful CPU fallback\n- **Smart CPU/GPU dispatching** based on data size\n- **GPU memory management** with pooling and caching\n- **CuPy/CUDA integration** for array operations\n- **Configurable thresholds** for GPU usage\n- **Multi-GPU support** for device selection\n- **Zero code changes** - same API works on CPU or GPU\n- **Performance monitoring** with GPU statistics\n\n### Advanced Caching System\n- **Specialization cache** with multiple eviction policies (LRU, LFU, Adaptive)\n- **Memory-bounded** cache with configurable limits\n- **Weak references** to prevent memory leaks\n- **TTL-based expiration** for temporal optimization\n- **Thread-safe** concurrent access\n- **Real-time statistics** and monitoring\n\n### Performance Profiling \u0026 Analytics\n- **Runtime profiling** with minimal overhead\n- **Hot path detection** and prioritization\n- **Performance analytics** and reporting\n- **Specialization effectiveness** tracking\n- **Cache performance** monitoring\n- **Adaptive optimization** recommendations\n\n### Financial Computing \u0026 Trading\n- **JIT-optimized** financial metrics (Sharpe ratio, drawdown, etc.)\n- **Ultra-fast backtesting** engine for trading strategies\n- **Genetic algorithm** optimization for parameter tuning\n- **High-frequency trading** optimizations\n- **Portfolio optimization** with risk management\n\n## 📖 Documentation\n\n### Core Decorator\n\nThe `@optimize` decorator is the main entry point:\n\n```python\nfrom python_optimizer import optimize\n\n@optimize(\n    jit=True,                    # Enable JIT compilation\n    specialize=True,             # Enable variable specialization\n    profile=True,                # Enable performance profiling\n    aggressiveness=2,            # Optimization level (0-3)\n    cache=True,                  # Enable specialization caching\n    gpu=True,                    # Enable GPU acceleration (NEW!)\n    gpu_min_size=10_000,         # Minimum size to use GPU (NEW!)\n    adaptive_learning=True,      # Enable adaptive optimization\n    memory_limit_mb=100,         # Cache memory limit\n    min_calls_for_spec=3         # Minimum calls before specialization\n)\ndef your_function(x, y):\n    # Your code here - automatically optimized based on usage patterns\n    # GPU acceleration for large arrays, JIT for numerical code\n    return x * y + compute_heavy_operation()\n```\n\n### New Specialization Functions\n\n```python\nfrom python_optimizer import (\n    get_specialization_stats,\n    clear_specialization_cache,\n    configure_specialization,\n    get_cache_stats\n)\n\n# Configure global specialization behavior\nconfigure_specialization(\n    min_calls_for_specialization=3,\n    min_performance_gain=0.1,\n    enable_adaptive_learning=True,\n    max_cache_size=1000,\n    max_memory_mb=100\n)\n\n# Get performance statistics\nstats = get_specialization_stats('function_name')\nprint(f\"Specializations created: {stats.get('specializations_created')}\")\nprint(f\"Cache hit rate: {stats.get('cache_hit_rate'):.2%}\")\nprint(f\"Performance gain: {stats.get('avg_performance_gain'):.2f}x\")\n\n# Global cache statistics\ncache_stats = get_cache_stats()\nprint(f\"Total cache entries: {cache_stats['total_entries']}\")\nprint(f\"Memory usage: {cache_stats['memory_usage_estimate']:.2f} MB\")\n\n# Clear cache when needed\nclear_specialization_cache()  # Clear all\nclear_specialization_cache('specific_function')  # Clear specific function\n```\n\n### JIT Functions\n\nPre-built JIT-optimized functions:\n\n```python\nfrom python_optimizer.jit import (\n    calculate_returns_jit,\n    calculate_sharpe_ratio_jit,\n    calculate_max_drawdown_jit,\n    simulate_strategy_jit\n)\n```\n\n### Genetic Algorithm Optimization\n\n```python\nfrom python_optimizer.genetic import GeneticOptimizer, ParameterRange\n\n# Define optimization parameters\nparam_ranges = [\n    ParameterRange('learning_rate', 0.001, 0.1, 'float'),\n    ParameterRange('hidden_layers', 1, 5, 'int'),\n]\n\n# Run optimization\noptimizer = GeneticOptimizer(param_ranges, population_size=100)\nbest_params = optimizer.optimize(fitness_function, generations=50)\n```\n\n## 🧪 Examples\n\nCheck out the `examples/` directory for:\n\n- **Financial modeling** with JIT optimization\n- **Machine learning** parameter optimization\n- **Numerical computing** acceleration\n- **Trading strategy** backtesting\n\n## 📊 Benchmarks\n\nRun benchmarks to see performance on your system:\n\n```bash\npython benchmarks/jit_performance_test.py\npython benchmarks/genetic_algorithm_benchmark.py\npython benchmarks/financial_metrics_benchmark.py\n```\n\n## 🔧 Configuration\n\n### Environment Variables\n\n```bash\nexport PYTHON_OPTIMIZER_JIT_CACHE=1     # Enable JIT cache\nexport PYTHON_OPTIMIZER_PROFILE=1       # Enable profiling\nexport PYTHON_OPTIMIZER_PARALLEL=1      # Enable parallel execution\n```\n\n### Configuration File\n\nCreate `python_optimizer.toml`:\n\n```toml\n[jit]\ncache_dir = \"~/.python_optimizer/cache\"\ncompile_timeout = 30\n\n[profiling]\nenabled = true\noutput_dir = \"./profiles\"\n\n[specialization]\nmax_variants = 5\nthreshold = 100\n```\n\n## 🤝 Contributing\n\nWe welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.\n\n### Development Setup\n\n```bash\ngit clone https://github.com/thinmanj/python-optimizer.git\ncd python-optimizer\n\n# Create virtual environment\npython -m venv venv\nsource venv/bin/activate  # On Windows: venv\\Scripts\\activate\n\n# Install development dependencies\npip install -e \".[dev]\"\n\n# Run tests\npytest tests/\n\n# Run linting\nblack python_optimizer/\nisort python_optimizer/\nflake8 python_optimizer/\n```\n\n## 📈 Roadmap\n\n- [x] **JIT Compilation Engine** - Numba-based optimization\n- [x] **Advanced Variable Specialization** - Type-aware optimization with caching\n- [x] **Intelligent Caching System** - Multi-policy cache with memory management\n- [x] **Performance Monitoring** - Real-time analytics and adaptive learning\n- [x] **Financial Computing Module** - Trading strategy optimization\n- [x] **Genetic Algorithm** - Parameter optimization\n- [x] **Thread-Safe Operations** - Concurrent optimization support\n- [x] **GPU Acceleration** - CUDA/CuPy support with automatic CPU/GPU dispatching ✨\n- [x] **ML Model Optimization** - PyTorch integration with training/inference optimization ✨\n- [x] **TensorFlow Integration** - TensorFlow/Keras model optimization with XLA ✨\n- [x] **Distributed Computing** - Multi-node optimization with Ray/Dask/multiprocessing ✨\n- [ ] **Advanced Profiling** - Visual performance analysis tools\n- [ ] **Web Interface** - Browser-based optimization dashboard\n\n## 📄 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## 🙏 Acknowledgments\n\n- **Numba team** for excellent JIT compilation framework\n- **NumPy community** for foundational numerical computing\n- **Trading algorithm researchers** for inspiration and validation\n\n## 📞 Support\n\n- **Issues:** [GitHub Issues](https://github.com/thinmanj/python-optimizer/issues)\n- **Discussions:** [GitHub Discussions](https://github.com/thinmanj/python-optimizer/discussions)\n- **Email:** thinmanj@gmail.com\n\n---\n\n⭐ **Star this repository** if Python Optimizer helps accelerate your code!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthinmanj%2Fpython-optimizer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthinmanj%2Fpython-optimizer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthinmanj%2Fpython-optimizer/lists"}