{"id":14967130,"url":"https://github.com/golemfactory/golemrpc","last_synced_at":"2025-12-14T04:39:32.183Z","repository":{"id":34301864,"uuid":"159834848","full_name":"golemfactory/golemrpc","owner":"golemfactory","description":null,"archived":false,"fork":false,"pushed_at":"2022-11-04T19:19:36.000Z","size":1181,"stargazers_count":7,"open_issues_count":2,"forks_count":2,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-10-05T16:49:19.300Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/golemfactory.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-11-30T14:28:03.000Z","updated_at":"2022-10-27T04:48:45.000Z","dependencies_parsed_at":"2023-01-15T06:07:15.374Z","dependency_job_id":null,"html_url":"https://github.com/golemfactory/golemrpc","commit_stats":null,"previous_names":["golemfactory/raspa-poc"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/golemfactory%2Fgolemrpc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/golemfactory%2Fgolemrpc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/golemfactory%2Fgolemrpc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/golemfactory%2Fgolemrpc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/golemfactory","download_url":"https://codeload.github.com/golemfactory/golemrpc/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":219869985,"owners_count":16555427,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-24T13:37:30.001Z","updated_at":"2025-10-19T09:31:35.211Z","avatar_url":"https://github.com/golemfactory.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Quickstart\n\nGolemrpc is a python client that allows you to use reuqest tasks in Golem Network without installing Golem on your local machine.\n\nUsually, one needs to run full Golem node in order to participate in and use Golem Network. Golemrpc allows you to connect to specified Golem node that operates on your behalf in Golem Network. This way you can request your tasks and the remote Golem node handles networking and protocol. It even handles payments so you do not need to have any ETH and/or GNT. You only need to provide a script that connects to the remote Golem node and requests tasks - see our `examples`.\n\nFirst of all, you need to know your remote Golem node. Contact the Golem node's administrator and obtain\n1. IP address and port,\n2. node's certificate that authenticates it, the `.pem` file,\n3. access secret that authenticates you, the `.tck` file.\n\nOptionally, you can run your own local Golem node. Aforementioned files can be found under paths: `$datadir/crossbar/rpc_cert.pem` and `$datadir/crossbar/secrets/golemcli.tck`.\n\nSystem requirements:\n1. [*Python3.6*](https://www.python.org/downloads/release/python-360/) installed.\n\nBefore running examples, perform a one time installation:\n\n```sh\npip3 install golemrpc\n```\n\nWindows installation is a bit more complicated because one of our library's dependencies is not precompiled for this OS and `pip3` will try to compile it during installation. To successfully proceed with compilation one has to install VisualStudio with it's libraries. See our [VisualStudio installation manual](https://docs.golem.network/#/Products/Brass-Beta/Installation?id=visual-studio).\n\nTake one of our examples: `examples/dummy_task_on_aws.py` or `examples/raspa_task_on_aws.py`. Copy the example script, `.pem` file and `.tck` file to a working directory.\nIf you want to use raspa, you also need to copy `examples/raspa_data` to the working directory. If you want to use dummy task, you need to create `my_input.txt` file in your home directory and write some human readable text as test data (to know why simply examine source code of the examples).\nNow edit the script and update the connection config:\n\n```python\nrpc = RPCComponent(\n    host='35.158.100.160',\n    port=61000,\n    cli_secret_filepath='golemcli_aws.tck',\n    rpc_cert_filepath='rpc_cert_aws.pem'\n)\n```\n\nNow you can run the script:\n\n```sh\npython3 your_example.py\n```\n\nAfter a while, you will see an output: a JSON object containing information on where to find the results (paths) and metadata of the submitted task.\n\n### Complete script:\n\n```sh\npip3 install golemrpc\ngit clone https://github.com/golemfactory/golemrpc.git\ncd golemrpc/examples\n# Edit the example if needed.\npython3.6 raspa_task.py\n```\n\n# Overview\n\nGolemrpc is a python package allowing communication with a (remote) Golem node. Connection handling, golem task state handling, resources upload and results retrieval are handled automatically for the user. This package has been created mainly for RASPA2 use case. Environment used for this Golem app has only few python packages installed.\n\nRASPA2 tasks are submitted to Golem in the form of Python functions and `args` dictionary, e.g.:\n\n```python\ndef raspa_task(args):\n    '''Task to compute provider side.\n    It is possible to import RASPA2 package on provider side because\n    this package is preinstalled in a docker environment we are about\n    to use (dockerhub golemfactory/glambda:1.5).\n    '''\n    import RASPA2\n    import pybel\n\n    mol = pybel.readstring('cif', args['mol'])\n\n    return RASPA2.get_helium_void_fraction(mol)\n```\n\nIt is called RASPA2, however generic Python functions are equally acceptable.\n\n## Requirements\n\nUser must have access to a Golem node (`v0.20+`) with a public IP address and ports forwarded (e.g. AWS hosted). By having access, we mean possessing SSL certificate and CLI secret files required to establish a connection. For typical Golem installation, these are stored in `$golem_datadir/crossbar` directory.\n\n## Architecture\n\nGolemrpc allows users to make task requests without having to install Golem client and take care of port forwarding, GNT purchase etc. Library handles user defined tasks by communicating with Golem over RPC (remote procedure call) interface. This Golem node will be requesting tasks to P2P network on user's behalf. Requesting node can be hosted by anyone, anywhere, etc., for instance, on the AWS EC2 instance.\n\nGolem node running locally, i.e., as localhost, is also supported, but by default, this library will handle input/output files transfers over TCP. To disable it and take advantage of sharing the filesystem with Golem node, use `remote` argument in `RPCComponent` class.\n\n## Installation\n\nUse package installer for Python3.6:\n\n```sh\npip3 install golemrpc\n```\n\n# User guide\n\n## Basic example\n\nFirst set up a logging module:\n\n```python\nimport logging\nlogging.basicConfig(level=logging.INFO)\n```\n\nThen create and start an RPCComponent:\n\n```python\nfrom golemrpc.rpccomponent import RPCComponent\nrpc = RPCComponent(\n    host='35.158.100.160',\n    port=61000,\n    cli_secret_filepath='golemcli_aws.tck',\n    rpc_cert_filepath='rpc_cert_aws.pem'\n)\nrpc.start()\n```\n\nIt will spawn a separate thread that will handle communication with Golem on `35.158.100.160:61000`. Now define a `dummy_task` function\nto compute on Golem provider's machine:\n\n```python\ndef dummy_task(args):\n    return 1 + args['b']\n```\n\nCreate a golemrpc `CreateTask` message with task type `GLambda` (an abbreviation for Golem Lambda, a type of task that takes serialized Python function and dictionary as input and sends them to the provider for computation. For more information about messages and events see [this](https://github.com/golemfactory/golemrpc/blob/master/docs/messages.md)):\n\n```python\nmessage = {\n    'type': 'CreateTask',\n    'task': {\n        'type': 'GLambda',\n        'options': {\n            'method': dummy_task,\n            'args': {'b': 2},\n        },\n    }\n}\n```\n\nSend it to RPCComponent:\n\n```python\nrpc.post(message)\n```\n\nNow poll the RPC message queue for new events. First expected event is `TaskCreatedEvent`:\n\n```python\ntask_created_event = rpc.poll(timeout=None)\n```\n\nIt contains `task_id`assigned to this particular task by the requesting Golem node. One can use it to keep track of tasks in case there are more than one being computed at a time. Second message coming from RPCComponent should be `TaskResults` containing filepaths to actual results:\n\n```python\ntask_results = rpc.poll(timeout=None)\n```\n\nBy default there are three files listed in `TaskResults` message: `result.json`, `stderr.log` and `stdout.log`. Result returned by `dummy_task` function is serialized to a `result.json` of form:\n\n```json\n{\n    \"data\": 3,\n}\n```\n\nIf there are any errors in user supplied function the JSON object will contain `error` key:\n\n```json\n{\n    \"error\": \"Some error message\"\n}\n```\n\nPick result file, load it and verify the results:\n\n```python\nimport json\n\nresult_json_file = None\n\nfor f in task_results['results']:\n    if f.endswith('result.json'):\n        result_json_file = f\n        break\n\nwith open(result_json_file, 'r') as f:\n    result_json = json.load(f)\nassert result_json['data'] == (1 + 2)\n```\n\n## Big input files\n\nBy default, `CreateTask` message cannot exceed 0.5MB. One must use `resources` task's field instead of function `args` in order to supply bigger file inputs. Files listed in `resources` will be saved in `/golem/resources` directory (with the path structure preserved) which is accessible from the user supplied function, e.g.:\n\n```python\ndef echo_task(args):\n    my_input = None\n    with open('/golem/resources/my_input.txt') as f:\n        my_input = f.read()\n    return my_input\n```\n\nis valid if followed by message:\n\n```python\nmessage = {\n    'type': 'CreateTask',\n    'task': {\n        'type': 'GLambda',\n        'options': {\n            'method': echo_task,\n        },\n        'resources': ['my_input.txt']\n    }\n}\n```\n\n## Custom output files\n\nThere is no size limit for `result.json` file although one might want to use format different than JSON. To get back the results in a custom format, you have to write them to a file in `/golem/output` directory. Every file in this directory will be packed and sent back to you if it is listed in `outputs` field of `CreateTask` [message](https://github.com/golemfactory/golemrpc/blob/master/docs/messages.md), e.g.:\n\n```python\ndef echo_task(args):\n    with open('/golem/resources/my_input.txt', 'r') as fin,\\\n         open('/golem/output/my_output.txt', 'w') as fout:\n        fout.write(fin.read())\n```\n\nwill create `my_output.txt` result file and send it back to the user when created by message:\n\n```python\nimport os\nmessage = {\n    'type': 'CreateTask',\n    'task': {\n        'type': 'GLambda',\n        'options': {\n            'method': echo_task,\n            'outputs': ['my_output.txt']\n        },\n        'resources': ['{cwd}/my_input.txt'.format(cwd=os.getcwd())]\n    }\n}\n```\n\n## Local Golem nodes setup\n\nOne has to install Golem from source using `b0.20` branch. See Golem instructions for [running from source](https://github.com/golemfactory/golem/wiki/Installation#running-from-the-source).\n\nFurther steps assume that user has successfully installed Golem from source, and have `GOLEM_DIR` environment variable defined.\n\nSet up two Golem nodes:\n\n ```sh\n# Node listening on port 61000, requestor.\npython $GOLEM_DIR/golemapp.py --datadir=$HOME/datadir1 --password=node1 --accept-terms --rpc-address=localhost:61000\n# Node listening on port 61001, provider.\npython $GOLEM_DIR/golemapp.py --datadir=$HOME/datadir2 --password=node2 --accept-terms --rpc-address=localhost:61001 --peer=localhost:40102\n```\n\nNow, if first node acts as a requestor, one should use CLI secret and SSL cert from `$HOME/datadir1/crossbar/rpc_cert.pem` and `$HOME/datadir1/crossbar/secrets/golemcli.tck`.\n\n## Other\n\nFor more information regarding task requesting see our [examples](https://github.com/golemfactory/golemrpc/tree/master/examples).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgolemfactory%2Fgolemrpc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgolemfactory%2Fgolemrpc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgolemfactory%2Fgolemrpc/lists"}