{"id":17059270,"url":"https://github.com/danijar/teleport","last_synced_at":"2025-04-12T17:51:53.678Z","repository":{"id":249913781,"uuid":"832938106","full_name":"danijar/teleport","owner":"danijar","description":"Efficiently send large arrays across machines","archived":false,"fork":false,"pushed_at":"2024-07-24T03:28:42.000Z","size":18,"stargazers_count":15,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-26T12:11:50.223Z","etag":null,"topics":["array","distributed","multiprocessing","numpy","python","server","socket"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/danijar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-24T03:24:38.000Z","updated_at":"2024-10-26T14:08:31.000Z","dependencies_parsed_at":"2024-07-24T05:26:04.216Z","dependency_job_id":"b41d0cf5-6c11-4f48-a315-be192f5bcf2f","html_url":"https://github.com/danijar/teleport","commit_stats":null,"previous_names":["danijar/teleport"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danijar%2Fteleport","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danijar%2Fteleport/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danijar%2Fteleport/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danijar%2Fteleport/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/danijar","download_url":"https://codeload.github.com/danijar/teleport/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248610407,"owners_count":21132920,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["array","distributed","multiprocessing","numpy","python","server","socket"],"created_at":"2024-10-14T10:33:42.571Z","updated_at":"2025-04-12T17:51:53.641Z","avatar_url":"https://github.com/danijar.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🌀 Teleport\n\nEfficiently send large arrays across machines.\n\n## Overview\n\nTeleport provides a `Server` that you can bind functions to and a `Client` that\ncan call the functions and receive their results. The function arguments and\nreturn values are both trees of **Numpy arrays**. The data is sent efficiently\nwithout serialization to maximize throughput.\n\n## Installation\n\n```sh\npip install git+https://github.com/danijar/teleport.git\n```\n\n## Example\n\nThis example runs the server and client in the same Python program using\nsubprocesses, but they could also be separate Python scripts running on\ndifferent machines.\n\n```python\ndef server():\n  import teleport\n  server = teleport.Server('tcp://*:2222')\n  server.bind('add', lambda data: {'result': data['foo'] + data['bar']})\n  server.bind('msg', lambda data: print('Message from client:', data['msg']))\n  server.run()\n\ndef client():\n  import teleport\n  client = teleport.Client('tcp://localhost:2222')\n  client.connect()\n  future = client.add({'foo': 1, 'bar': 1})\n  result = future.result()\n  print(result)  # {'result': 2}\n  client.msg({'msg': 'Hello World'})\n\nif __name__ == '__main__':\n  import teleport\n  server_proc = teleport.Process(server, start=True)\n  client_proc = teleport.Process(client, start=True)\n  client_proc.join()\n  server_proc.terminate()\n```\n\n## Features\n\nSeveral productivity and performance features are available:\n\n- **Request batching:** The server can batch requests together so that the user\n  function receives a dict of stacked arrays and the function result will be\n  split and sent back to the corresponding clients.\n- **Multithreading:** Servers can use a thread pool to process multiple\n  requests in parallel. Optionally, each function can also request its own\n  thread pool to allow functions to block (e.g. for rate limiting) without\n  blocking other functions.\n- **Async clients:** Clients can send multiple overlapping requests and wait\n  on the results when needed using `Future` objects. The maximum number of\n  inflight requests can be limited to avoid requests building up when the\n  server is slower than the client.\n- **Error handling:** Exceptions raised in server functions are reported to the\n  client and raised in `future.result()` or, if the user did not store the\n  future object, on the next request. Worker exception can also be reraised in\n  the server application using `server.check()`.\n- **Heartbeating:** Clients can send ping requests when they have not received\n  a result from the server for a while, allowing to wait for results that take\n  a long time to compute without assuming connection loss.\n- **Concurrency:** `Thread` and `Process` implementations with exception\n  forwarding that can be forcefully terminated by the parent, which Python\n  threads do not natively support. Stoppable threads and processes are also\n  available for coorperative shutdown.\n- **GIL load reduction:** The `ProcServer` behaves just like the normal\n  `Server` but uses a background process to batch requests and fan out results,\n  substantially reducing GIL load for the server workers in the main process.\n\n## Questions\n\nIf you have a question, please [file an issue][issue].\n\n[issue]: https://github.com/danijar/teleport/issues\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanijar%2Fteleport","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdanijar%2Fteleport","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanijar%2Fteleport/lists"}