Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/beam-cloud/beta9
Run serverless GPU workloads with fast cold starts on bare-metal servers, anywhere in the world
https://github.com/beam-cloud/beta9
autoscaler cloudrun cuda developer-productivity distributed-computing faas fine-tuning functions-as-a-service generative-ai gpu lambda large-language-models llm llm-inference ml-platform paas self-hosted serverless serverless-containers
Last synced: 2 days ago
JSON representation
Run serverless GPU workloads with fast cold starts on bare-metal servers, anywhere in the world
- Host: GitHub
- URL: https://github.com/beam-cloud/beta9
- Owner: beam-cloud
- License: agpl-3.0
- Created: 2023-11-15T00:53:21.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-12-15T19:43:43.000Z (about 1 month ago)
- Last Synced: 2024-12-16T05:29:56.979Z (about 1 month ago)
- Topics: autoscaler, cloudrun, cuda, developer-productivity, distributed-computing, faas, fine-tuning, functions-as-a-service, generative-ai, gpu, lambda, large-language-models, llm, llm-inference, ml-platform, paas, self-hosted, serverless, serverless-containers
- Language: Go
- Homepage: https://docs.beam.cloud
- Size: 8.97 MB
- Stars: 600
- Watchers: 4
- Forks: 34
- Open Issues: 12
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
---
### AI Infrastructure for Developers
Run AI workloads anywhere with zero complexity. One line of Python, global GPUs, full control.
---
[English](https://github.com/beam-cloud/beta9/blob/master/README.md) | [简体中文](https://github.com/beam-cloud/beta9/blob/master/docs/zh/zh_cn/README.md) | [繁體中文](https://github.com/beam-cloud/beta9/blob/master/docs/zh/zh_cw/README.md) | [Türkçe](https://github.com/beam-cloud/beta9/blob/master/docs/tr/README.md) | [हिंदी](https://github.com/beam-cloud/beta9/blob/master/docs/in/README.md) | [Português (Brasil)](https://github.com/beam-cloud/beta9/blob/master/docs/pt/README.md) | [Italiano](https://github.com/beam-cloud/beta9/blob/master/docs/it/README.md) | [Español](https://github.com/beam-cloud/beta9/blob/master/docs/es/README.md) | [한국어](https://github.com/beam-cloud/beta9/blob/master/docs/kr/README.md) | [日本語](https://github.com/beam-cloud/beta9/blob/master/docs/jp/README.md)
---
## Features
- Run serverless workloads using a friendly Python interface
- Autoscaling and automatic scale-to-zero
- Read large files at the edge using distributed, cross-region storage
- Connect bare-metal nodes to your cluster with a single cURL command
- Manage your fleet of servers using a Tailscale-powered service mesh
- Securely run workloads with end-to-end encryption through WireGuard
## How Does It Work?
### Run Serverless AI Workloads
Add an `endpoint` decorator to your code, and you'll get a load-balanced HTTP endpoint (with auth!) to invoke your code.
You can also run long-running functions with `@function`, deploy task queues using `@task_queue`, and schedule jobs with `@schedule`:
```python
from beta9 import endpoint
# This will run on a remote A100-40 in your cluster
@endpoint(cpu=1, memory=128, gpu="A100-40")
def square(i: int):
return i**2
```
Deploy with a single command:
```
$ beta9 deploy app.py:square --name inference
=> Building image
=> Using cached image
=> Deployed 🎉
curl -X POST 'https://inference.beam.cloud/v1' \
-H 'Authorization: Bearer [YOUR_AUTH_TOKEN]' \
-H 'Content-Type: application/json' \
-d '{}'
```
### Run on Bare-Metal Servers Around the World
Connect any GPU to your cluster with one CLI command and a cURL.
```sh
$ beta9 machine create --pool lambda-a100-40
=> Created machine with ID: '9541cbd2'. Use the following command to set up the node:
#!/bin/bash
sudo curl -L -o agent https://release.beam.cloud/agent/agent && \
sudo chmod +x agent && \
sudo ./agent --token "AUTH_TOKEN" \
--machine-id "9541cbd2" \
--tailscale-url "" \
--tailscale-auth "AUTH_TOKEN" \
--pool-name "lambda-a100-40" \
--provider-name "lambda"
```
You can run this install script on your VM to connect it to your cluster.
### Manage Your CPU or GPU Fleet
Manage your distributed cross-region cluster using a centralized control plane.
```sh
$ beta9 machine list
| ID | CPU | Memory | GPU | Status | Pool |
|----------|---------|------------|---------|------------|-------------|
| edc9c2d2 | 30,000m | 222.16 GiB | A10G | registered | lambda-a10g |
| d87ad026 | 30,000m | 216.25 GiB | A100-40 | registered | gcp-a100-40 |
```
# Local Installation
You can run Beta9 locally, or in an existing Kubernetes cluster using our [Helm chart](https://github.com/beam-cloud/beta9/tree/main/deploy/charts/beta9).
### Setting Up the Server
k3d is used for local development. You'll need Docker to get started.
To use our fully automated setup, run the `setup` make target.
```bash
make setup
```
#### Local DNS
This is required to use an external file service for mulitpart uploads and range downloads. Its optional for using the subdomain middlware (host-based URLs).
```shell
brew install dnsmasq
echo 'address=/cluster.local/127.0.0.1' >> /opt/homebrew/etc/dnsmasq.conf
sudo bash -c 'mkdir -p /etc/resolver'
sudo bash -c 'echo "nameserver 127.0.0.1" > /etc/resolver/cluster.local'
sudo brew services start dnsmasq
```
To use subdomain or host-based URLs, add this to the config and rebuild the Beta9 gateway.
```yaml
gateway:
invokeURLType: host
```
You should now be able to access your local k3s instance via a domain.
```shell
curl http://beta9-gateway.beta9.svc.cluster.local:1994/api/v1/health
```
### Setting Up the SDK
The SDK is written in Python. You'll need Python 3.8 or higher. Use the `setup-sdk` make target to get started.
```bash
make setup-sdk
```
### Using the SDK
After you've setup the server and SDK, check out the SDK readme [here](sdk/README.md).
## Contributing
We welcome contributions big or small. These are the most helpful things for us:
- Submit a [feature request](https://github.com/beam-cloud/beta9/issues/new?assignees=&labels=&projects=&template=feature-request.md&title=) or [bug report](https://github.com/beam-cloud/beta9/issues/new?assignees=&labels=&projects=&template=bug-report.md&title=)
- Open a PR with a new feature or improvement
## Community & Support
If you need support, you can reach out through any of these channels:
- [Slack](https://join.slack.com/t/beam-cloud/shared_invite/zt-2f16bwiiq-oP8weCLWNrf_9lJZIDf0Fg) \(Chat live with maintainers and community members\)
- [GitHub issues](https://github.com/beam-cloud/issues) \(Bug reports, feature requests, and anything roadmap related)
- [Twitter](https://twitter.com/beam_cloud) \(Updates on releases and more)
## Thanks to Our Contributors
