https://github.com/beam-cloud/beta9
Run serverless GPU workloads with fast cold starts on bare-metal servers, anywhere in the world
https://github.com/beam-cloud/beta9
autoscaler cloudrun cuda developer-productivity distributed-computing faas fine-tuning functions-as-a-service generative-ai gpu large-language-models llm llm-inference ml-platform paas self-hosted serverless serverless-containers
Last synced: about 4 hours ago
JSON representation
Run serverless GPU workloads with fast cold starts on bare-metal servers, anywhere in the world
- Host: GitHub
- URL: https://github.com/beam-cloud/beta9
- Owner: beam-cloud
- License: agpl-3.0
- Created: 2023-11-15T00:53:21.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2025-04-14T18:06:30.000Z (11 months ago)
- Last Synced: 2025-04-14T18:16:42.379Z (11 months ago)
- Topics: autoscaler, cloudrun, cuda, developer-productivity, distributed-computing, faas, fine-tuning, functions-as-a-service, generative-ai, gpu, large-language-models, llm, llm-inference, ml-platform, paas, self-hosted, serverless, serverless-containers
- Language: Go
- Homepage: https://docs.beam.cloud
- Size: 11.1 MB
- Stars: 751
- Watchers: 4
- Forks: 43
- Open Issues: 12
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
- Awesome-LLMOps - beta9 - cloud/beta9.svg?style=flat&color=green)   (Inference / Inference Platform)
README
## Run AI Workloads at Scale
**[Beam](https://beam.cloud?utm_source=github_readme)** is a fast, open-source runtime for serverless AI workloads. It gives you a Pythonic interface to deploy and scale AI applications with zero infrastructure overhead.
## ✨ Features
- **Fast Image Builds**: Launch containers in under a second using a custom container runtime
- **Parallelization and Concurrency**: Fan out workloads to 100s of containers
- **First-Class Developer Experience**: Hot-reloading, webhooks, and scheduled jobs
- **Scale-to-Zero**: Workloads are serverless by default
- **Volume Storage**: Mount distributed storage volumes
- **GPU Support**: Run on our cloud (4090s, H100s, and more) or bring your own GPUs
## 📦 Installation
```shell
pip install beam-client
```
## ⚡️ Quickstart
1. Create an account [here](https://beam.cloud?utm_source=github_readme)
2. Follow our [Getting Started Guide](https://platform.beam.cloud/onboarding?utm_source=github_readme)
## Creating a sandbox
Spin up isolated containers to run LLM-generated code:
```python
from beam import Image, Sandbox
sandbox = Sandbox(image=Image()).create()
response = sandbox.process.run_code("print('I am running remotely')")
print(response.result)
```
## Deploy a serverless inference endpoint
Create an autoscaling endpoint for your custom model:
```python
from beam import Image, endpoint
from beam import QueueDepthAutoscaler
@endpoint(
image=Image(python_version="python3.11"),
gpu="A10G",
cpu=2,
memory="16Gi",
autoscaler=QueueDepthAutoscaler(max_containers=5, tasks_per_container=30)
)
def handler():
return {"label": "cat", "confidence": 0.97}
```
## Run background tasks
Schedule resilient background tasks (or replace your Celery queue) by adding a simple decorator:
```python
from beam import Image, TaskPolicy, schema, task_queue
class Input(schema.Schema):
image_url = schema.String()
@task_queue(
name="image-processor",
image=Image(python_version="python3.11"),
cpu=1,
memory=1024,
inputs=Input,
task_policy=TaskPolicy(max_retries=3),
)
def my_background_task(input: Input, *, context):
image_url = input.image_url
print(f"Processing image: {image_url}")
return {"image_url": image_url}
if __name__ == "__main__":
# Invoke a background task from your app (without deploying it)
my_background_task.put(image_url="https://example.com/image.jpg")
# You can also deploy this behind a versioned endpoint with:
# beam deploy app.py:my_background_task --name image-processor
```
> ## Self-Hosting vs Cloud
>
> Beta9 is the open-source engine powering [Beam](https://beam.cloud), our fully-managed cloud platform. You can self-host Beta9 for free or choose managed cloud hosting through Beam.
## 👋 Contributing
We welcome contributions big or small. These are the most helpful things for us:
- Submit a [feature request](https://github.com/beam-cloud/beta9/issues/new?assignees=&labels=&projects=&template=feature-request.md&title=) or [bug report](https://github.com/beam-cloud/beta9/issues/new?assignees=&labels=&projects=&template=bug-report.md&title=)
- Open a PR with a new feature or improvement
## ❤️ Thanks to Our Contributors
