{"id":24641148,"url":"https://github.com/elixir-cloud-aai/protes","last_synced_at":"2025-09-11T02:39:26.921Z","repository":{"id":38847958,"uuid":"157390813","full_name":"elixir-cloud-aai/proTES","owner":"elixir-cloud-aai","description":"Proxy service for injecting middleware into GA4GH TES requests","archived":false,"fork":false,"pushed_at":"2025-08-18T14:39:57.000Z","size":524,"stargazers_count":8,"open_issues_count":21,"forks_count":6,"subscribers_count":28,"default_branch":"dev","last_synced_at":"2025-08-18T16:22:58.039Z","etag":null,"topics":["api-gateway","ga4gh","hacktoberfest","task-execution"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elixir-cloud-aai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-11-13T14:12:14.000Z","updated_at":"2025-08-18T14:40:00.000Z","dependencies_parsed_at":"2023-02-19T04:20:25.541Z","dependency_job_id":"a5ebef31-1581-4e14-9c70-bb001766b3d2","html_url":"https://github.com/elixir-cloud-aai/proTES","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/elixir-cloud-aai/proTES","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elixir-cloud-aai%2FproTES","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elixir-cloud-aai%2FproTES/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elixir-cloud-aai%2FproTES/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elixir-cloud-aai%2FproTES/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elixir-cloud-aai","download_url":"https://codeload.github.com/elixir-cloud-aai/proTES/tar.gz/refs/heads/dev","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elixir-cloud-aai%2FproTES/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274568594,"owners_count":25309282,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-11T02:00:13.660Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api-gateway","ga4gh","hacktoberfest","task-execution"],"created_at":"2025-01-25T12:12:27.436Z","updated_at":"2025-09-11T02:39:26.862Z","avatar_url":"https://github.com/elixir-cloud-aai.png","language":"Python","readme":"# proTES\n\n[![license][badge-license]][badge-url-license]\n[![chat][badge-chat]][badge-url-chat]\n[![ci][badge-ci]][badge-url-ci]\n\n## Synopsis\n\nproTES is a robust and scalable [Global Alliance for Genomics and Health\n(GA4GH)][res-ga4gh] [Task Execution Service (TES) API][res-ga4gh-tes] gateway\nthat may play a pivotal role in augmenting the capabilities of your GA4GH Cloud\necosystem by offering flexible middleware injection for effectively federating\natomic, containerized workloads across on premise, hybrid and multi-cloud\nenvironments composed of GA4GH TES nodes.\n\n## Description\n\nproTES gateway may serve as a crucial component in federated compute networks\nbased on the GA4GH Cloud ecosystem. Its primary purpose is to provide\ncentralized features to a federated network of independently operated  GA4GH TES \ninstances. As such, it can serve, for example, as a compatibility layer, a load\nbalancer workload distribution layer, a public entry point to an enclave of\nindependent compute nodes, or a means of collecting telemetry.\n\nWhen TES requests are received, proTES applies a configured middlewares before\nforwarding the requests to appropriate TES instances in the network. A plugin\nsystem makes it easy to write and inject middlewares tailored to specific\nrequirements, such as for access control, request/response processing or\nvalidation, or the selection of suitable endpoints considering data use\nrestrictions and client preferences.\n\n### Built-in middleware plugins\n\nCurrently, there are two plugins shipped with proTES that each serve as\nproof-of-concept examples for different task distribution scenarios:\n\n* **Load balancing**: The `pro_tes.middleware.task_distribution.random` plugin\n  evenly (actually: randomly!) distributes workloads across a network of TES\n  endpoints\n* **Bringing compute to the data**: The\n  `pro_tes.middleware.task_distribution.distance` plugin selects TES endpoints \n  to relay incoming requests to in such a way that the distance the (input) data\n  of a task has to travel across the network of TES endpoints is minimized. \n\n### Implementation notes\n\nproTES is a [Flask][res-flask] microservice that supports\n[OAuth2][res-oauth2]-based authorization out of the box (bearer authentication)\nand stores information about incoming and outgoing tasks in a NoSQL database\n([MongoDB][res-mongodb]). Based on our [FOCA][res-foca] microservice archetype,\nit is highly configurable in a declarative (YAML-based!) manner. Forwarded tasks\nare tracked asynchronously via a [RabbitMQ][res-rabbitmq] broker and\n[Celery][res-celery] workers that can be easily scaled up. Both a\n[Helm][res-helm] chart and a [Docker Compose][res-docker-compose] configuration\nare provided for easy deployment in native cloud-based production and\ndevelopment environments, respectively.\n\n![proTES-overview][image-protes-overview]\n\n## Installation\n\nFor production-grade [Kubernetes][res-kubernetes]-based deployment, see\n[separate instructions][docs-deploy]. For testing/development purposes, you can\nuse the instructions described below.\n\n### Requirements\n\nEnsure you have the following software installed:\n\n* [Docker][res-docker] (18.06.1-ce, build e68fc7a)\n* [docker-compose][res-docker-compose] (1.22.0, build f46880fe)\n* [Git][res-git] (1.8.3.1)\n\n\u003e **Note:** These indicated versions are those that were used for\n\u003e developing/testing. Other versions may or may not work.\n\n### Prerequisites\n\nCreate data directory and required subdiretories\n\n```bash\nexport PROTES_DATA_DIR=/path/to/data/directory\nmkdir -p $PROTES_DATA_DIR/{db,specs}\n```\n\n\u003e **Note:** If the `PROTES_DATA_DIR` environment variable is not set, proTES\n\u003e will require the following default directories to be available:\n\u003e\n\u003e * `../data/pro_tes/db`\n\u003e * `../data/pro_tes/specs`\n\nClone repository\n\n```bash\ngit clone https://github.com/elixir-europe/proTES.git\n```\n\nTraverse to app directory\n\n```bash\ncd proTES\n```\n\n### Configure (optional)\n\nThe following user-configurable files are available:\n\n* [app configuration](pro_tes/config/app_config.yaml)\n* [deployment configuration](docker-compose.yaml)\n\n### Deploy\n\nBuild/pull and run services\n\n```bash\ndocker-compose up -d --build\n```\n\nVisit Swagger UI\n\n```bash\nfirefox http://localhost:8080/ga4gh/tes/v1/ui\n```\n\n\u003e **Note:** Host and port may differ if you have changed the configuration or\n\u003e use an HTTP server to reroute calls to a different host.\n\n## Contributing\n\nThis project is a community effort and lives off your contributions, be it in\nthe form of bug reports, feature requests, discussions, ideas, fixes, or other\ncode changes. Please read [these guidelines][docs-contributing] if you want to\ncontribute. And please mind the [code of conduct][docs-coc] for all\ninteractions with the community.\n\n## Versioning\n\nThe project adopts the [semantic versioning][semver] scheme for versioning.\nCurrently the service is in beta stage, so the API may change and even break\nwithout further notice. However, once we deem the service stable and \"feature\ncomplete\", the major, minor and patch version will shadow the supported TES\nversion, with the build version representing proTES-internal updates.\n\n## License\n\nThis project is covered by the [Apache License 2.0][badge-url-license] also\n[shipped with this repository][docs-license].\n\n## Contact\n\nproTES is part of [ELIXIR Cloud \u0026 AAI][res-elixir-cloud-aai], a multinational\neffort at establishing and implementing FAIR data sharing and promoting\nreproducible data analyses and responsible data handling in the life sciences.\n\nIf you have suggestions for or find issue with this app, please use the\n[issue tracker][contact-issue-tracker]. If you would like to reach out to us\nfor anything else, you can join our [Slack board][badge-url-chat], start a\nthread in our [Q\u0026A forum][contact-qa], or send us an [email][contact-email].\n\n[![GA4GH logo](images/logo-ga4gh.png)](https://www.ga4gh.org/)\n[![ELIXIR logo](images/logo-elixir.png)](https://www.elixir-europe.org/)\n[![ELIXIR Cloud \u0026 AAI logo](images/logo-elixir-cloud.png)](https://elixir-europe.github.io/cloud/)\n\n[badge-chat]: \u003chttps://img.shields.io/static/v1?label=chat\u0026message=Slack\u0026color=ff6994\u003e\n[badge-ci]: \u003chttps://github.com/elixir-cloud-aai/proTES/actions/workflows/checks.yaml/badge.svg\u003e\n[badge-license]: \u003chttps://img.shields.io/badge/license-Apache%202.0-blue.svg\u003e\n[badge-url-chat]: \u003chttps://join.slack.com/t/elixir-cloud/shared_invite/zt-1uvebyx1e-NcC0Hof2guT9df~haBdQaw\u003e\n[badge-url-ci]: \u003chttps://github.com/elixir-cloud-aai/proTES/actions/workflows/checks.yaml\u003e\n[badge-url-license]: \u003chttp://www.apache.org/licenses/LICENSE-2.0\u003e\n[contact-email]: \u003cmailto:cloud-service@elixir-europe.org\u003e\n[contact-issue-tracker]: \u003chttps://github.com/elixir-cloud-aai/landing-page/issues\u003e\n[contact-qa]: \u003chttps://github.com/elixir-cloud-aai/elixir-cloud-aai/discussions\u003e\n[docs-coc]: \u003chttps://github.com/elixir-cloud-aai/elixir-cloud-aai/blob/dev/CODE_OF_CONDUCT.md\u003e\n[docs-contributing]: \u003chttps://elixir-cloud-aai.github.io/guides/guide-contributor/\u003e\n[docs-deploy]: deployment/README.md\n[docs-license]: LICENSE\n[GA4GH TES OpenAPI specification]:\u003chttps://github.com/ga4gh/task-execution-schemas\u003e\n[image-protes-overview]: \u003cimages/overview.svg\u003e\n[res-celery]: \u003chttp://www.celeryproject.org/\u003e\n[res-connexion]: \u003chttps://github.com/zalando/connexion\u003e\n[res-docker]: \u003chttps://www.docker.com/\u003e\n[res-docker-compose]: \u003chttps://docs.docker.com/compose/\u003e\n[res-elixir-cloud-aai]: \u003chttps://elixir-cloud.dcc.sib.swiss/\u003e\n[res-flask]: \u003chttp://flask.pocoo.org/\u003e\n[res-foca]: \u003chttps://github.com/elixir-cloud-aai/foca\u003e\n[res-ga4gh]: \u003chttps://www.ga4gh.org/\u003e\n[res-ga4gh-cloud]: \u003chttps://www.ga4gh.org/work_stream/cloud/\u003e\n[res-ga4gh-tes]: \u003chttps://github.com/ga4gh/task-execution-schemas\u003e\n[res-git]: \u003chttps://git-scm.com/\u003e\n[res-helm]: \u003chttps://helm.sh/\u003e\n[res-kubernetes]: \u003chttps://kubernetes.io/\u003e\n[res-mondodb]: \u003chttps://www.mongodb.com/\u003e\n[res-ouath2]: \u003chttps://oauth.net/2/\u003e\n[res-rabbitmq]: \u003chttps://www.rabbitmq.com/\u003e\n[res-sem-ver]: \u003chttps://semver.org/\u003e\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felixir-cloud-aai%2Fprotes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felixir-cloud-aai%2Fprotes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felixir-cloud-aai%2Fprotes/lists"}