{"id":43931334,"url":"https://github.com/runpod/flash","last_synced_at":"2026-04-16T01:01:16.036Z","repository":{"id":285417837,"uuid":"954961206","full_name":"runpod/flash","owner":"runpod","description":"Application framework for Multimodal Distributed inference \u0026 Orchestration. ","archived":false,"fork":false,"pushed_at":"2026-04-10T21:30:18.000Z","size":5093,"stargazers_count":100,"open_issues_count":7,"forks_count":10,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-04-10T22:12:27.826Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/runpod.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-03-25T22:16:46.000Z","updated_at":"2026-04-10T08:00:46.000Z","dependencies_parsed_at":null,"dependency_job_id":"481059a7-695e-45e6-b738-90a390857b2a","html_url":"https://github.com/runpod/flash","commit_stats":null,"previous_names":["runpod/tetra-rp","runpod/flash"],"tags_count":62,"template":false,"template_full_name":null,"purl":"pkg:github/runpod/flash","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runpod%2Fflash","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runpod%2Fflash/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runpod%2Fflash/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runpod%2Fflash/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/runpod","download_url":"https://codeload.github.com/runpod/flash/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runpod%2Fflash/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31866357,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-15T15:24:51.572Z","status":"ssl_error","status_checked_at":"2026-04-15T15:24:39.138Z","response_time":63,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-07T00:04:50.381Z","updated_at":"2026-04-16T01:01:16.026Z","avatar_url":"https://github.com/runpod.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Flash\n\nFlash is a Python SDK for developing cloud-native AI apps where you define everything—hardware, remote functions, and dependencies—using local code.\n\n```python\nimport asyncio\nfrom runpod_flash import Endpoint, GpuType\n\n# Mark the function below for remote execution\n@Endpoint(name=\"hello-gpu\", gpu=GpuType.NVIDIA_GEFORCE_RTX_4090, dependencies=[\"torch\"]) \nasync def hello(): # This function runs on Runpod\n    import torch\n    gpu_name = torch.cuda.get_device_name(0)\n    print(f\"Hello from your GPU! ({gpu_name})\")\n    return {\"gpu\": gpu_name}\n\nasyncio.run(hello())\nprint(\"Done!\") # This runs locally\n```\n\nWrite `@Endpoint` decorated Python functions on your local machine. Run them, and Flash automatically handles GPU/CPU provisioning and worker scaling on [Runpod Serverless](https://docs.runpod.io/serverless/overview).\n\n## Setup\n\n### Install Flash\n\nInstall Flash using `pip` or `uv`:\n\n```bash\n# Install with pip\npip install runpod-flash\n\n# Or uv\nuv add runpod-flash\n```\n\nFlash requires [Python 3.10+](https://www.python.org/downloads/), and is currently available for macOS and Linux. Windows support is in development.\n\n### Authentication\n\nBefore you can use Flash, you need to authenticate with your Runpod account:\n\n```bash\nflash login\n```\n\nThis saves your API key securely and allows you to use the Flash CLI and run `@Endpoint` functions.\n\n### Coding agent integration (optional)\n\nInstall the Flash skill package for AI coding agents like Claude Code, Cline, and Cursor:\n\n```bash\nnpx skills add runpod/skills\n```\n\nYou can review the `SKILL.md` file in the [runpod/skills repository](https://github.com/runpod/skills/blob/main/flash/SKILL.md).\n\n## Quickstart\n\nCreate `gpu_demo.py`:\n\n```python\nimport asyncio\nfrom runpod_flash import Endpoint, GpuType\n\n@Endpoint(\n    name=\"flash-quickstart\",\n    gpu=GpuType.NVIDIA_GEFORCE_RTX_4090,\n    workers=3,\n    dependencies=[\"numpy\", \"torch\"]\n)\ndef gpu_matrix_multiply(size):\n    # IMPORTANT: Import packages INSIDE the function\n    import numpy as np\n    import torch\n\n    # Get GPU name\n    device_name = torch.cuda.get_device_name(0)\n\n    # Create random matrices\n    A = np.random.rand(size, size)\n    B = np.random.rand(size, size)\n\n    # Multiply matrices\n    C = np.dot(A, B)\n\n    return {\n        \"matrix_size\": size,\n        \"result_mean\": float(np.mean(C)),\n        \"gpu\": device_name\n    }\n\n# Call the function\nasync def main():\n    print(\"Running matrix multiplication on Runpod GPU...\")\n    result = await gpu_matrix_multiply(1000)\n\n    print(f\"\\n✓ Matrix size: {result['matrix_size']}x{result['matrix_size']}\")\n    print(f\"✓ Result mean: {result['result_mean']:.4f}\")\n    print(f\"✓ GPU used: {result['gpu']}\")\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n```\n\nRun it:\n\n```bash\npython gpu_demo.py\n```\n\nFirst run takes 30-60 seconds (provisioning). Subsequent runs take 2-3 seconds.\n\n## What Flash does\n\n- **Remote execution**: `@Endpoint` functions run on Runpod Serverless GPUs/CPUs\n- **Auto-scaling**: Workers scale from 0 to N based on demand\n- **Dependency management**: Packages install automatically on remote workers\n- **Two patterns**: Queue-based (`@Endpoint`) for batch work, load-balanced (`Endpoint()` + routes) for REST APIs\n- **Concurrency control**: `max_concurrency` lets each worker process multiple jobs simultaneously\n\n## Documentation\n\nFull documentation: **[docs.runpod.io/flash](https://docs.runpod.io/flash)**\n\n- [Quickstart](https://docs.runpod.io/flash/quickstart) - First GPU workload in 5 minutes\n- [Create endpoints](https://docs.runpod.io/flash/endpoint-functions) - Queue-based, load-balancing, and custom Docker endpoints\n- [CLI reference](https://docs.runpod.io/flash/cli/overview) - `flash run`, `flash deploy`, `flash build`\n- [Configuration](https://docs.runpod.io/flash/configuration/parameters) - All endpoint parameters\n\n## Flash apps\n\nWhen you're ready to move beyond scripts and build a production-ready API, you can create a [Flash app](https://docs.runpod.io/flash/apps/overview) (a collection of interconnected endpoints with diverse hardware configurations) and deploy it to Runpod.\n\n[Follow this tutorial to build your first Flash app](https://docs.runpod.io/flash/apps/build-app).\n\n## Flash CLI\n\nThe Flash CLI provides a set of commands for managing your Flash apps and endpoints.\n\n```bash\nflash --help\n```\n\n[Learn more about the Flash CLI](https://docs.runpod.io/flash/cli/overview).\n\n\n## Examples\n\nBrowse working examples: **[github.com/runpod/flash-examples](https://github.com/runpod/flash-examples)**\n\n## Requirements\n\n- Python 3.12\n- macOS or Linux (Windows support in development)\n- A [Runpod account](https://runpod.io/console) (email must be verified) with an API key\n\n## Contributing\n\nWe welcome contributions! See [RELEASE_SYSTEM.md](RELEASE_SYSTEM.md) for development workflow.\n\n```bash\n# Clone and install\ngit clone https://github.com/runpod/flash.git\ncd flash\npip install -e \".[dev]\"\n\n# Use conventional commits\ngit commit -m \"feat: add new feature\"\ngit commit -m \"fix: resolve issue\"\n```\n\n## Support\n\n- [Discord](https://discord.gg/cUpRmau42V) - Community support\n- [GitHub Issues](https://github.com/runpod/flash/issues) - Bug reports\n\n## License\n\nMIT License - see [LICENSE](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frunpod%2Fflash","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frunpod%2Fflash","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frunpod%2Fflash/lists"}