{"id":24641199,"url":"https://github.com/elixir-cloud-aai/mock-tes","last_synced_at":"2025-10-07T02:30:48.559Z","repository":{"id":45233757,"uuid":"190434305","full_name":"elixir-cloud-aai/mock-TES","owner":"elixir-cloud-aai","description":"Connexion-based mock-up service implementing the GA4GH Task Execution Service API schema","archived":false,"fork":false,"pushed_at":"2021-12-29T14:24:34.000Z","size":96,"stargazers_count":3,"open_issues_count":3,"forks_count":1,"subscribers_count":18,"default_branch":"dev","last_synced_at":"2023-07-11T09:26:25.571Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elixir-cloud-aai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-06-05T16:56:21.000Z","updated_at":"2023-07-11T09:26:25.572Z","dependencies_parsed_at":"2022-09-17T08:11:03.677Z","dependency_job_id":null,"html_url":"https://github.com/elixir-cloud-aai/mock-TES","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elixir-cloud-aai%2Fmock-TES","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elixir-cloud-aai%2Fmock-TES/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elixir-cloud-aai%2Fmock-TES/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elixir-cloud-aai%2Fmock-TES/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elixir-cloud-aai","download_url":"https://codeload.github.com/elixir-cloud-aai/mock-TES/tar.gz/refs/heads/dev","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":235575672,"owners_count":19012159,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-25T12:12:37.328Z","updated_at":"2025-10-07T02:30:43.294Z","avatar_url":"https://github.com/elixir-cloud-aai.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mock-TES\n\n[![Apache License](https://img.shields.io/badge/license-Apache%202.0-orange.svg?style=flat\u0026color=important)](http://www.apache.org/licenses/LICENSE-2.0)\n\n[Connexion]-based mockup service implementing parts of the GA4GH [Task Execution\nService] (TES) API schema. The service was developed to implement and test\n[TEStribute], a task distribution logic package for TES instances. It features\nan extended TES specification to provide parameters required for the model\nunderlying the task distribution logic.\n\n## Usage\n\nOnce deployed and started ([see below](#Deployment)), the service will be\navailable at:  \n\u003chttp://localhost:9001/ga4gh/tes/v1/\u003e\n\nYou can explore the service via the Swagger UI:\n\n```bash\nfirefox http://localhost:9001/ga4gh/tes/v1/ui/\n```\n\nThe specifications, in JSON format, can be retrieved with:\n\n```bash\nwget http://localhost:9001/ga4gh/tes/v1/swagger.json\n```\n\n\u003e Note that host and port can be set manually in the [config] file. In that\n\u003e case, the values in the URLs above need to be replaced as well.\n\nThe client [TES-cli] can be used to send requests to the service.\n\n## Deployment\n\n`mock-TES` can be deployed via containers (preferred) or after manual\ninstallation of all dependencies.\n\nIn both cases, the repository first needs to be cloned with:\n\n```bash\ngit clone git@github.com:elixir-europe/mock-TES.git\n```\n\nAfterwards traverse to the repository's root directory:\n\n```bash\ncd mock-TES\n```\n\n### Containerized deployment\n\n\u003e \"Production-like\" containerized deployment without HTTP server/load balancer\n\u003e etc.\n\n#### Requirements (Dockerized deployment)\n\n* [Git] (tested with version 2.17.1)\n* [Docker] (tested with version 18.09.6)\n* [docker-compose] (tested with version 1.24.0)\n\n#### Building \u0026 starting the service\n\n```bash\n# Build application image\n# [NOTE] Image re-building is not always necessary. Inspect the `Dockerfile`\n#        to check which changes will need re-building.\ndocker-compose build\n# Start service\ndocker-compose up -d\n```\n\n#### Other useful commands\n\n```bash\n# Check logs\ndocker-compose logs\n# Shut down service\ndocker-compose down\n```\n\n### Non-containerized deployment\n\n\u003e Deployment for local development without containers, HTTP server/load\n\u003e balancer etc.\n\n#### Requirements\n\n* [Git] (tested with version 2.17.1)\n* [Python] (tested with versions 2.7.15+ \u0026 3.6.8)\n* [pip] (tested with version 19.1.1)\n* [virtualenv] (tested with version 15.1.0)\n\n#### Installing \u0026 starting the service\n\n```bash\n# Set up Python virtual environment\nvirtualenv -p `which python3` venv\nsource venv/bin/activate\n\n# Install dependencies\npip install -r requirements.txt\n\n# Install app\npython setup.py develop\n\n# Run service\npython mock_tes/Server.py\n```\n\n## Implementation details\n\nNote that only those parts of the service are implemented that are required for\ndeveloping, testing and debugging [TEStribute]. For all other endpoints only\nstubs are implemented that return the most basic valid response (typically an\n\"empty\" JSON object).\n\n### Modifications to TES specs\n\nAn endpoint `/tasks/task-info` was added to commit [`d55bf88`] of the [Task\nExecution Service] [OpenAPI] specification. It provides parameters required for\n[TEStribute]'s task distribution logic.\n\nThe [modified specifications] as well as a snapshot of the [unmodified\nspecifications] are available in this repository.\n\nThis section outlines the definitions of the endpoint and underlying request\nand response models.\n\n#### Endpoint `/tasks/task-info`\n\nThe endpoint `/tasks/task-info` returns estimations of queue time and incurred\ncosts if a task with a given set of resource requirements were to be computed\non this TES instance right now. Within the framework of [TEStribute], these\nparameters allow informed decisions with regards to which of a number of TES\ninstances a given task should be sent to. The endpoint is defined by:\n\n```yaml\n/tasks/task-info:\n    post:\n      summary: |-\n        Provides estimates for the queue time and incurred costs for a task\n        with the given resource requirements.\n      operationId: GetTaskInfo\n      responses:\n        '200':\n          description: ''\n          schema:\n            $ref: '#/definitions/tesTaskInfo'\n      parameters:\n        - name: body\n          in: body\n          required: true\n          schema:\n            $ref: '#/definitions/tesResources'\n      tags:\n        - TaskService\n      x-swagger-router-controller: ga4gh.tes.server\n```\n\n#### Request model `tesResources`\n\nA property `execution_time_sec` was added to the model describing a task's\nresource requirements. The entire model is now defined as follows:\n\n```yaml\n tesResources:\n    type: object\n    properties:\n      execution_time_sec:\n        type: number\n        format: double\n        description: Requested execution in seconds (s)\n      cpu_cores:\n        type: integer\n        format: int64\n        description: Requested number of CPUs\n      preemptible:\n        type: boolean\n        format: boolean\n        description: Is the task allowed to run on preemptible compute instances (e.g. AWS Spot)?\n      ram_gb:\n        type: number\n        format: double\n        description: Requested RAM required in gigabytes (GB)\n      disk_gb:\n        type: number\n        format: double\n        description: Requested disk size in gigabytes (GB)\n      zones:\n        type: array\n        items:\n          type: string\n        description: Request that the task be run in these compute zones.\n    description: Resources describes the resources requested by a task.\n```\n\n#### Response model `tesTaskInfo`\n\nA valid request to the endpoint results in a response defined in the following\nmodel:\n\n```yaml\n    type: object\n    properties:\n      estimated_compute_costs:\n        $ref: '#/definitions/tesCosts'\n        description: |-\n          Estimated total incurred compute costs for running a task with the\n          indicated resource requirements on this TES instance. Excludes data\n          storage and transfer costs.\n      estimated_storage_costs:\n        $ref: '#/definitions/tesCosts'\n        description: |-\n          Estimated total incurred data storage costs for running a task with\n          the indicated resource requirements on this TES instance. Excludes\n          compute and data transfer costs. Currently this is reported as a flat\n          fee for conceptually indefinite storage.\n      estimated_queue_time_sec:\n        type: number\n        format: double\n        description: |-\n          Given the current load on this TES instance, returns an estimate of\n          the time, in seconds (s), that a task with the indicated resource\n          requirements will spend in the task queue.\n      unit_costs_data_transfer:\n        $ref: '#/definitions/tesCosts'\n        description: |-\n          Unit costs for transferring 1 gigabyte (GB) of data (inputs, outputs)\n          across 1000 kilometers (km).\n    description: |-\n      Given a set of resource requirements, returns the estimated queue time\n      and total incurred costs. Allows informed decisions with regard to which\n      TES instance a given task should be sent to.\n```\n\n##### `tesCosts`\n\nThe response model further relies on model `tesCosts` which specifies an\n`amount` and a `currency` property:\n\n```yaml\n  tesCosts:\n    type: object\n    properties:\n      amount:\n        type: number\n        format: double\n        description: Numeric value specifying an amount of money.\n      currency:\n        type: string\n        enum:\n          - AUD\n          - BRL\n          - BTC\n          - CAD\n          - CHF\n          - CNH\n          - EUR\n          - GBP\n          - HKD\n          - INR\n          - KRW\n          - JPY\n          - MXN\n          - NOK\n          - NZD\n          - RUB\n          - SEK\n          - SGD\n          - TRY\n          - USD\n          - ZAR\n        description: Currency/unit of the costs.\n    description: Generic object specifying an amount of money.\n```\n\n### Service configuration\n\nThe service can be configured with different unit costs for CPU and memory\nusage, data transfer and storage. Default values for the corresponding\nparameters are listed in `task_info` section of the service's [config] and they\ncan be edited by the user before starting the service.\n\nAlternatively (and preferably), these parameters can be modified in the running\nservice via the `/update-config` endpoint, which is particularly useful for\nsetting up environments for various testing scenarios for [TEStribute]. The\nendpoint is defined in the [update config] [OpenAPI] specifications:\n\n```yaml\n  /update-config:\n    post:\n      summary: Update task info config\n      operationId: UpdateTaskInfoConfig\n      responses:\n        '200':\n          description: ''\n          schema:\n            $ref: '#/definitions/tesTaskInfoConfig'\n      parameters:\n        - name: body\n          in: body\n          schema:\n            $ref: '#/definitions/tesTaskInfoConfig'\n          required: true\n          description: ''\n      tags:\n        - TaskService\n      x-swagger-router-controller: ga4gh.tes.server\n```\n\nIt relies on the following models:\n\n```yaml\n  tesTaskInfoConfig:\n    type: object\n    properties:\n      currency:\n        type: string\n        enum:\n          - AUD\n          - BRL\n          - BTC\n          - CAD\n          - CHF\n          - CNH\n          - EUR\n          - GBP\n          - HKD\n          - INR\n          - KRW\n          - JPY\n          - MXN\n          - NOK\n          - NZD\n          - RUB\n          - SEK\n          - SGD\n          - TRY\n          - USD\n          - ZAR\n        description: Currency/unit of the costs.\n      unit_costs:\n        $ref: '#/definitions/tesTaskInfoCosts'\n  tesTaskInfoCosts:\n    type: object\n    properties:\n      cpu_usage:\n        type: integer\n        format: int64\n        description: costs per core\n      memory_consumption:\n        type: integer\n        format: int64\n        description: costs of computation per GB\n      data_storage:\n        type: integer\n        format: int64\n        description:  cost of data storage  GB\n      data_transfer:\n        type: integer\n        format: int64\n        description: cost of data transfer per GB and 1000 km\n```\n\n[TES-cli] can be used to communicate with the `/update-config` endpoint.\n\n\u003e Note that while the `/update-config` endpoint can be accessed via the same\n\u003e root URI (and explored via the Swagger UI), it was not included in the\n\u003e [modified TES specifications], but rather is added to it _on the fly_ when the\n\u003e service is started.\n\n## Contributing\n\nThis project is a community effort and lives off your contributions, be it in\nthe form of bug reports, feature requests, discussions, or fixes and other code\nchanges. Please read the [contributing guidelines] if you want to contribute.\nAnd please mind the [code of conduct] for all interactions with the community.\n\n## Versioning\n\nDevelopment of the app is currently still in alpha stage, and current versioning\nis for internal use only. In the future, we are aiming to adopt [semantic\nversioning] that is synchronized to the versioning of [TEStribute] and\n[TES-cli] in order to ensure that these apps will be compatible as long as both\ntheir major and minor versions match.\n\n## License\n\nThis project is covered by the [Apache License 2.0] also available [shipped\nwith this repository](LICENSE).\n\n## Contact\n\nPlease contact the [project leader](mailto:alexander.kanitz@sib.swiss) for\ninquiries, proposals, questions etc. that are not covered by the\n[Contributing](#Contributing) section.\n\n## Acknowledgments\n\nThe project is a collaborative effort under the umbrella of the [ELIXIR Cloud\nand AAI] group. It was started during the [2019 Google Summer of Code] as part\nof the [Global Alliance for Genomics and Health] [organization].\n\n![logo banner]\n\n[2019 Google Summer of Code]: \u003chttps://summerofcode.withgoogle.com/projects/#6613336345542656\u003e\n[Apache License 2.0]: \u003chttps://www.apache.org/licenses/LICENSE-2.0\u003e\n[code of conduct]: CODE_OF_CONDUCT.md\n[config]: mock_tes/config/app_config.yaml\n[Connexion]: \u003chttps://github.com/zalando/connexion\u003e\n[contributing guidelines]: CONTRIBUTING.md\n[`d55bf88`]: \u003chttps://github.com/ga4gh/task-execution-schemas/tree/d55bf880062442288afc95665aa0e21fbba77b20\u003e\n[Docker]: \u003chttps://docs.docker.com/install/\u003e\n[docker-compose]: \u003chttps://docs.docker.com/compose/install/\u003e\n[ELIXIR Cloud and AAI]: \u003chttps://elixir-europe.github.io/cloud/\u003e\n[Git]: \u003chttps://git-scm.com/book/en/v2/Getting-Started-Installing-Git\u003e\n[Global Alliance for Genomics and Health]: \u003chttps://www.ga4gh.org/\u003e\n[logo banner]: logos/logo-banner.svg\n[modified specifications]: mock_tes/specs/schema.task_execution_service.d55bf88.openapi.modified.yaml\n[modified TES specifications]: mock_tes/specs/schema.task_execution_service.d55bf88.openapi.modified.yaml\n[OpenAPI]: \u003chttps://swagger.io/specification/\u003e\n[organization]: \u003chttps://summerofcode.withgoogle.com/organizations/6643588285333504/\u003e\n[pip]: \u003chttps://pip.pypa.io/en/stable/installing/\u003e\n[Python]: \u003chttps://www.python.org/downloads/\u003e\n[semantic versioning]: \u003chttps://semver.org/\u003e\n[Task Execution Service]: \u003chttps://github.com/ga4gh/task-execution-schemas\u003e\n[TES-cli]: \u003chttps://github.com/elixir-europe/TES-cli\u003e\n[TEStribute]: \u003chttps://github.com/elixir-europe/TEStribute\u003e\n[unmodified specifications]: mock_tes/specs/schema.task_execution_service.d55bf88.openapi.yaml\n[update config]: mock_tes/specs/schema.task_execution_service.config_update.openapi.yaml\n[virtualenv]: \u003chttps://virtualenv.pypa.io/en/stable/installation/\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felixir-cloud-aai%2Fmock-tes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felixir-cloud-aai%2Fmock-tes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felixir-cloud-aai%2Fmock-tes/lists"}