{"id":20512041,"url":"https://github.com/wardlt/gladier-test","last_synced_at":"2025-03-05T22:44:07.258Z","repository":{"id":91621818,"uuid":"486646172","full_name":"WardLT/gladier-test","owner":"WardLT","description":null,"archived":false,"fork":false,"pushed_at":"2022-04-28T15:17:49.000Z","size":3,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-01-16T09:08:39.816Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/WardLT.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-28T15:17:47.000Z","updated_at":"2022-04-28T15:17:52.000Z","dependencies_parsed_at":null,"dependency_job_id":"3ef731ae-67e8-44ca-ab5d-14b03da50823","html_url":"https://github.com/WardLT/gladier-test","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":"globus-gladier/gladier-client-template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WardLT%2Fgladier-test","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WardLT%2Fgladier-test/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WardLT%2Fgladier-test/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WardLT%2Fgladier-test/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/WardLT","download_url":"https://codeload.github.com/WardLT/gladier-test/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242117649,"owners_count":20074435,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T20:39:30.131Z","updated_at":"2025-03-05T22:44:07.217Z","avatar_url":"https://github.com/WardLT.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# gladier-client-template\n\n## HOW TO USE THIS TEMPLATE\n\n\u003e **DO NOT FORK** this is meant to be used from **[Use this template](https://github.com/globus-gladier/gladier-client-template/generate)** feature.\n\n1. Click on **[Use this template](https://github.com/globus-gladier/gladier-client-template/generate)**\n2. Give a new name to your gladier project  \n   (e.g. `gladier_amazing_science` recommendation is to use all lowercase and underscores separation for repo names.)\n3. Wait until the first run of CI finishes  \n   (Github Actions will process the template and commit to your new repo)\n4. Then clone your new gladier and start tweaking it.\n\n\u003e **NOTE**: **WAIT** until first CI run on github actions before cloning your new project.\n\n# Your Journey starts here.\n## If you chose to accept it.\n\nHello traveler, this is your tutorial into the world of automation.\n\nThis repository is designed to be a guide through the complexities of Gladier and its tools.\nFor full automation of an experiment, there is a step of infrastructure that cannot be ignored. This is usually done once and allows the user to focus on tool development.\n\nThree main operations are included on the simpleClients:\n* Process\n* Transfer\n* Publish\n\n## Installing Gladier\n\nTo run this tutorial we advise that a new environment is created on your favorite tool. For simplicity, we will use miniconda although any development app with access to pip should suffice.\n\n    conda create -n gladier-test python pip\n    conda activate gladier-test\n    pip install gladier\n\n## First run on a Gladier Client\n\nOur first example can be found at `example_client/example_client.py`. It creates and executes a flow with one operation.\n\n   ./example_client/example_client.py\n\nTo execute a remote function (the equivalent of a `lambda`) we use the funcX service to register, retrieve and execute functions. The service requires that a small instance client is deployed at the \"processing\" machine. We have a client running at one of our example machines. The location of this client is defined by:\n\n    'funcx_endpoint_compute': '4b116d3c-1703-4f8f-9f6f-39921e5864df'\n\nNote1: The remote machine is not necessary remote. Different parts of the flow can be executed on different machines by changing the `funcx_endpoint_compute` value for each tool. This will be explored on a further section.\n\nNote2: Gladier reads the python function imported by `SimpleTool` and automatically register or re-register(in the case of changes in the local function definition) it with the funcx service. The UUID of this function is automatically populated in the flow definition. \n\n\nDuring the execution takes care of creating the flow definition and registering it with the globus service:\n\n    Flow created with ID: ddb9d6be-d48f-40df-a663-6bcc6db1bb76                                                              https://app.globus.org/flows/ddb9d6be-d48f-40df-a663-6bcc6db1bb76 \n\nAnd also created the run after the `.run()` execution:\n\n    Run started with ID: 6fa0969a-2778-4f7f-95d5-d365e89aca32                                                               https://app.globus.org/runs/6fa0969a-2778-4f7f-95d5-d365e89aca32 \n\nRunning the client again will not register a new flow with the globus service but will generate a new run instance.\n\n### Best Practices: \nThe `SimpleTool` and its driving function `simple_function` are separated into a `tools` folder in a single file. We advise to create one python file per \"action\" in the flows. This makes development and debugging and tracing errors much simpler.\n\nThe `example_client.py` itself also is separated from the other clients in the folder and only contain one `GladierBaseClient`. This prevents instances being created with the 'wrong' flow definition or common mistakes on 'what is running'.\n\n\n## Creating a flow with Transfer\n\nOur second example can be found at `example_client/example_client_transfer.py`. It transfer a file from our remote server into your workstation. \nIn order to allow for transfer, the first step is to introduce the workstation in the the creates and executes a flow with one operation.\n\n   ./example_client/example_client_transfer.py\n\n\n## Creating a flow with Publish\n\nOur Third example can be found at `example_client/example_client_publish.py`. It do a simple operation of publishing some data into a globus index.\nAs before, it requires the setup of a globus index and how to visualize it. \n\n    pip install globus-search-cli\n    globus-search login\n    globus-search index create example-index gladier-example-index\n\nThe result is a new search index on the globus-search database which will serve as a \"repository\" for the flow data.\n\n    {\n       \"@datatype\": \"GSearchIndex\",\n       \"@version\": \"2017-09-01\",\n       \"creation_date\": \"2022-04-27 21:04:30\",\n       \"description\": \"gladier-example-index\",\n       \"display_name\": \"example-index\",\n       \"id\": \"563c3d98-6fa8-4ef5-83e2-0f378efe0a5f\",\n       \"is_trial\": true,\n       \"max_size_in_mb\": 1,\n       \"num_entries\": 0,\n       \"num_subjects\": 0,\n       \"size_in_mb\": 0,\n       \"status\": \"open\",\n       \"subscription_id\": null\n    }    \n\nThe search index id `563c3d98-6fa8-4ef5-83e2-0f378efe0a5f` will be used so the flow knows where to send metadata too.\n\nTo execute our simple publish client\n\n   ./example_client/example_client_publish.py\n\nTo check if the data went to the index try this check https://acdc.alcf.anl.gov/globus-tutorial/563c3d98-6fa8-4ef5-83e2-0f378efe0a5f\n\n\n## Making this template into a client\n\n    ./scripts/create_setup.py\n\n## Installing gladier_client\n\n    conda create -n gladier pip\n    conda activate gladier\n\n    git clone https://github.com/globus-gladier/gladier-client-template\n    cd gladier-client-template\n    \n    pip install -r requirements.txt\n\n    python setup.py develop\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwardlt%2Fgladier-test","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwardlt%2Fgladier-test","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwardlt%2Fgladier-test/lists"}