https://github.com/commaai/miniray
Minimal library for distributed python work. Can efficiently run CPU and GPU tasks across 100s of machines.
https://github.com/commaai/miniray
Last synced: about 1 month ago
JSON representation
Minimal library for distributed python work. Can efficiently run CPU and GPU tasks across 100s of machines.
- Host: GitHub
- URL: https://github.com/commaai/miniray
- Owner: commaai
- License: mit
- Created: 2026-01-29T19:46:18.000Z (6 months ago)
- Default Branch: master
- Last Pushed: 2026-06-19T17:47:11.000Z (about 2 months ago)
- Last Synced: 2026-06-19T19:26:13.504Z (about 2 months ago)
- Language: Python
- Size: 499 KB
- Stars: 131
- Watchers: 0
- Forks: 4
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# miniray
Miniray is a library for distributed compute across a datacenter. Miniray is designed to dispatch tasks of arbitrary python code through redis. Miniray uses python's *concurrent.futures* API.
### example
```
import miniray
def is_even(n):
return n % 2 == 0
x = np.arange(100)
results_loop = [is_even(n) for n in x]
with miniray.Executor(job_name='miniray_example_map') as executor:
results_map = executor.map(is_even, np.arange(100))
with miniray.Executor(job_name='miniray_example_submit') as executor:
futures = [executor.submit(is_even, n) for n in x]
results_submit = [future.result() for future in as_completed(futures)]
for a, b, c in zip(results_loop, results_map, results_submit):
assert a == b == c
```
### want to use?
If you have tasks that you rant to parallelize across multiple machines, miniray might be right for you! Contact harald@comma.ai if miniray is missing something you would like.