https://github.com/cortexlabs/cortex

Production infrastructure for machine learning at scale
https://github.com/cortexlabs/cortex

infrastructure machine-learning

Last synced: 7 days ago
JSON representation

Production infrastructure for machine learning at scale

• Host: GitHub
• URL: https://github.com/cortexlabs/cortex
• Owner: cortexlabs
• License: apache-2.0
• Created: 2019-01-24T04:43:14.000Z (almost 6 years ago)
• Default Branch: master
• Last Pushed: 2024-06-12T19:34:23.000Z (5 months ago)
• Last Synced: 2024-10-14T15:24:25.047Z (22 days ago)
• Topics: infrastructure, machine-learning
• Language: Go
• Homepage: https://cortexlabs.com/
• Size: 11 MB
• Stars: 8,019
• Watchers: 145
• Forks: 606
• Open Issues: 129
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE

Awesome Lists containing this project

• awesome - cortexlabs/cortex - Production infrastructure for machine learning at scale (Go)
• awesome-repositories - cortexlabs/cortex - Production infrastructure for machine learning at scale (Go)
• awesome-starts - cortexlabs/cortex - Scale compute-intensive serverless workloads (Go)
• awesome-list - cortex - Production infrastructure for machine learning at scale. (Deep Learning Framework / Deployment & Distribution)
• awesome-production-machine-learning - Cortex - Cortex is an open source platform for deploying machine learning models—trained with any framework—as production web services. No DevOps required. (Deployment and Serving)
• awesome-generative-models - cortex: Deploy machine learning models in production possibly without docker and kubernetes
• awesome-python-machine-learning-resources - GitHub - 10% open · ⏱️ 23.04.2022): (模型序列化和转换)
• awesome-production-machine-learning - Cortex - Cortex is an open source platform for deploying machine learning models—trained with nearly any framework—as production web services. (Model Deployment and Orchestration Frameworks)
• Awesome-AIML-Data-Ops - Cortex - Cortex is an open source platform for deploying machine learning models—trained with any framework—as production web services. No DevOps required. (Model Serving and Monitoring)

README

        
**[Docs](https://docs.cortexlabs.com)** • **[Slack](https://community.cortexlabs.com)**









Note: This project is no longer actively maintained by its original authors.

# Production infrastructure for machine learning at scale

Deploy, manage, and scale machine learning models in production.




## Serverless workloads

**Realtime** - respond to requests in real-time and autoscale based on in-flight request volumes.

**Async** - process requests asynchronously and autoscale based on request queue length.

**Batch** - run distributed and fault-tolerant batch processing jobs on-demand.




## Automated cluster management

**Autoscaling** - elastically scale clusters with CPU and GPU instances.

**Spot instances** - run workloads on spot instances with automated on-demand backups.

**Environments** - create multiple clusters with different configurations.




## CI/CD and observability integrations

**Provisioning** - provision clusters with declarative configuration or a Terraform provider.

**Metrics** - send metrics to any monitoring tool or use pre-built Grafana dashboards.

**Logs** - stream logs to any log management tool or use the pre-built CloudWatch integration.




## Built for AWS

**EKS** - Cortex runs on top of EKS to scale workloads reliably and cost-effectively.

**VPC** - deploy clusters into a VPC on your AWS account to keep your data private.

**IAM** - integrate with IAM for authentication and authorization workflows.