{"id":32177560,"url":"https://github.com/proxystore/proxystore","last_synced_at":"2026-04-02T01:44:30.886Z","repository":{"id":37080391,"uuid":"357984234","full_name":"proxystore/proxystore","owner":"proxystore","description":"Advanced data flow management for distributed Python applications","archived":false,"fork":false,"pushed_at":"2026-02-11T01:44:52.000Z","size":56575,"stargazers_count":37,"open_issues_count":2,"forks_count":17,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-11T05:58:33.716Z","etag":null,"topics":["distributed-computing","object-proxying","python","workflow-management"],"latest_commit_sha":null,"homepage":"https://docs.proxystore.dev/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/proxystore.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":".github/SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2021-04-14T17:14:13.000Z","updated_at":"2026-02-11T01:44:09.000Z","dependencies_parsed_at":"2023-10-26T22:23:20.575Z","dependency_job_id":"9c9b6dcb-1105-4504-806b-90b79fdace23","html_url":"https://github.com/proxystore/proxystore","commit_stats":null,"previous_names":["gpauloski/proxystore"],"tags_count":27,"template":false,"template_full_name":null,"purl":"pkg:github/proxystore/proxystore","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/proxystore%2Fproxystore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/proxystore%2Fproxystore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/proxystore%2Fproxystore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/proxystore%2Fproxystore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/proxystore","download_url":"https://codeload.github.com/proxystore/proxystore/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/proxystore%2Fproxystore/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29630829,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-19T18:02:07.722Z","status":"ssl_error","status_checked_at":"2026-02-19T18:01:46.144Z","response_time":117,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-computing","object-proxying","python","workflow-management"],"created_at":"2025-10-21T20:12:40.639Z","updated_at":"2026-04-02T01:44:30.879Z","avatar_url":"https://github.com/proxystore.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ProxyStore\n\n![PyPI - Version](https://img.shields.io/pypi/v/proxystore)\n![PyPI - Python Version](https://img.shields.io/pypi/pyversions/proxystore)\n![GitHub License](https://img.shields.io/github/license/proxystore/proxystore)\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.8077899.svg)](https://doi.org/10.5281/zenodo.8077899)\n\n[![docs](https://github.com/proxystore/proxystore/actions/workflows/docs.yml/badge.svg)](https://github.com/proxystore/proxystore/actions/workflows/docs.yml)\n[![tests](https://github.com/proxystore/proxystore/actions/workflows/tests.yml/badge.svg?label=tests)](https://github.com/proxystore/proxystore/actions)\n[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/proxystore/proxystore/main.svg)](https://results.pre-commit.ci/latest/github/proxystore/proxystore/main)\n\nProxyStore facilitates efficient data flow management in distributed Python applications, such as dynamic task-based workflows or serverless and edge applications.\n\nThe [*transparent object proxy*](https://docs.proxystore.dev/latest/concepts/proxy/), the core building block within ProxyStore, acts like a wide-area reference that can be cheaply communicated.\nUnlike traditional references that are only valid within the virtual address space of a single process, the proxy references an object in remote storage and can be implicitly dereferenced in arbitrary processes—even on remote machines.\nThe proxy is transparent in that it implicitly dereferences its target object when used—referred to a *just-in-time resolution*—and afterwards forwards all operations on itself to the cached target object.\n\nThis paradigm results in the best of both pass-by-reference and pass-by-value semantics, improves performance and portability by reducing transfer overheads through intermediaries, and abstracts low-level communication methods which reduces code complexity.\nA proxy contains within itself all the information and logic necessary to resolve the target object.\nThis self-contained nature means a proxy consumer need not be aware of the low-level communication mechanisms used by the proxy; rather, this is unilaterally determined by the producer of the proxy.\n\nProxyStore supports a diverse set of programming patterns built on the proxy paradigm:\n\n* [Task-based Workflows](https://arxiv.org/abs/2303.08803)\n* [Function-as-a-Service/Serverless Applications](https://docs.proxystore.dev/latest/guides/globus-compute/)\n* [Distributed Futures](https://docs.proxystore.dev/latest/guides/proxy-futures/)\n* [Bulk Data Streaming](https://docs.proxystore.dev/latest/guides/streaming/)\n* and more!\n\nProxyStore can leverage many popular mediated data transfer and storage systems:\n[DAOS](https://docs.daos.io/v2.4/),\n[Globus Transfer](https://www.globus.org/data-transfer),\n[Kafka](https://kafka.apache.org/),\n[KeyDB](https://docs.keydb.dev/), and\n[Redis](https://redis.io/).\nCustom communication methods built on\n[Mochi](https://mochi.readthedocs.io/en/latest/margo.html),\n[UCX](https://openucx.org/),\n[WebRTC](https://webrtc.org/), and\n[ZeroMQ](https://zeromq.org/)\nare provided for high-performance and peer-to-peer applications.\n\nRead more about ProxyStore's concepts [here](https://docs.proxystore.dev/latest/concepts/).\nComplete documentation for ProxyStore is available at\n[docs.proxystore.dev](https://docs.proxystore.dev).\n\n## Installation\n\nThe base ProxyStore package can be installed with [`pip`](https://pip.pypa.io/en/stable/).\n```bash\npip install proxystore\n```\n\nLeveraging third-party libraries may require dependencies not installed by default but can be enabled via extras installation options (e.g., `endpoints`, `kafka`, or `redis`).\n*All* additional dependencies can be installed with:\n```bash\npip install proxystore[all]\n```\nThis will also install the [`proxystore-ex`](https://extensions.proxystore.dev/)\npackage which contains extensions and experimental features.\nThe extensions package can also be installed with `pip` using\n`proxystore[extensions]` or `proxystore-ex`.\n\nSee the [Installation](https://docs.proxystore.dev/latest/installation) guide\nfor more information about the available extras installation options.\nSee the [Contributing](https://docs.proxystore.dev/latest/contributing) guide\nto get started for local development.\n\n## Example\n\nCreating proxies of objects only requires a few lines of code.\n\n```python\nfrom proxystore.connectors.local import LocalConnector\nfrom proxystore.proxy import Proxy\nfrom proxystore.store import Store\n\ndef process(x: dict[str, str]) -\u003e None:\n    # x is transparently resolved from the store when first\n    # used by the function. After which the proxy behaves as\n    # target object (the dict) for the rest of its existence.\n    assert isinstance(x, dict)\n    assert x['hello'] == 'world'\n\nwith Store('example', connector=LocalConnector(), register=True) as store:\n    # Store the object via a connector interface (here, a thread local\n    # store). The returned proxy acts like a reference to the object.\n    x = {'hello': 'world'}\n    proxy = store.proxy(x)\n    assert isinstance(proxy, Proxy)\n\n    # Invoking a function with proxy works without function changes.\n    process(proxy)\n```\n\nCheck out the [Get Started](https://docs.proxystore.dev/latest/get-started)\nguide to learn more!\n\n## Citation\n\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.8077899.svg)](https://doi.org/10.5281/zenodo.8077899)\n\nIf you use ProxyStore or any of this code in your work, please cite our ProxyStore ([SC '23](https://dl.acm.org/doi/10.1145/3581784.3607047)) and Proxy Patterns ([TPDS](https://ieeexplore.ieee.org/document/10776778)) papers.\n```bib\n@inproceedings{pauloski2023proxystore,\n    title = {Accelerating {C}ommunications in {F}ederated {A}pplications with {T}ransparent {O}bject {P}roxies},\n    author = {Pauloski, J. Gregory and Hayot-Sasson, Valerie and Ward, Logan and Hudson, Nathaniel and Sabino, Charlie and Baughman, Matt and Chard, Kyle and Foster, Ian},\n    address = {New York, NY, USA},\n    articleno = {59},\n    booktitle = {Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis},\n    doi = {10.1145/3581784.3607047},\n    isbn = {9798400701092},\n    location = {Denver, CO, USA},\n    numpages = {15},\n    publisher = {Association for Computing Machinery},\n    series = {SC '23},\n    url = {https://doi.org/10.1145/3581784.3607047},\n    year = {2023}\n}\n\n@article{pauloski2024proxystore,\n    title = {Object {P}roxy {P}atterns for {A}ccelerating {D}istributed {A}pplications},\n    author = {Pauloski, J. Gregory and Hayot-Sasson, Valerie and Ward, Logan and Brace, Alexander and Bauer, André and Chard, Kyle and Foster, Ian},\n    doi = {10.1109/TPDS.2024.3511347},\n    journal = {IEEE Transactions on Parallel and Distributed Systems},\n    number = {},\n    pages = {1-13},\n    volume = {},\n    year = {2024}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fproxystore%2Fproxystore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fproxystore%2Fproxystore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fproxystore%2Fproxystore/lists"}