{"id":20697444,"url":"https://github.com/epomatti/azure-batch","last_synced_at":"2026-04-14T14:33:05.545Z","repository":{"id":189381683,"uuid":"610779368","full_name":"epomatti/azure-batch","owner":"epomatti","description":"Azure Batch architecture","archived":false,"fork":false,"pushed_at":"2024-01-23T20:53:19.000Z","size":204,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-08-13T21:41:35.774Z","etag":null,"topics":["appi","application-insights","azure","azure-batch","batch","hpc","python","terraform"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/epomatti.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-07T13:18:48.000Z","updated_at":"2024-05-11T17:57:06.000Z","dependencies_parsed_at":"2024-01-09T17:59:08.511Z","dependency_job_id":"54bc7988-e69f-4c3f-96a0-15199254f199","html_url":"https://github.com/epomatti/azure-batch","commit_stats":null,"previous_names":["epomatti/azure-batch"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/epomatti/azure-batch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epomatti%2Fazure-batch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epomatti%2Fazure-batch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epomatti%2Fazure-batch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epomatti%2Fazure-batch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/epomatti","download_url":"https://codeload.github.com/epomatti/azure-batch/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epomatti%2Fazure-batch/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31801384,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-14T11:13:53.975Z","status":"ssl_error","status_checked_at":"2026-04-14T11:13:53.299Z","response_time":153,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["appi","application-insights","azure","azure-batch","batch","hpc","python","terraform"],"created_at":"2024-11-17T00:18:05.391Z","updated_at":"2026-04-14T14:33:05.522Z","avatar_url":"https://github.com/epomatti.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Azure Batch\n\n\u003e You might have to submit a support to increase Batch quotas\n\n\u003cimg src=\".docs/azure-batch.png\" /\u003e\n\n## Setup\n\nCopy the `.auto.tfvars` file template:\n\n```sh\ncp config/template.tfvars .auto.tfvars\n```\n\nCreate the account infrastructure:\n\n```sh\nterraform init\nterraform apply -auto-approve\n```\n\nℹ️ The pool will be provisioned with 0 nodes. Adjust your preferences accordingly.\n\nOptionally, Configure your Batch account logs to be sent to the Log Analytics Workspace by setting up the Diagnostic Settings using the portal.\n\nUse the script to create the application package:\n\n```sh\ncd scripts\n\nbash scripts/create-batch-application.sh\n```\n\nSet the application to the pool:\n\n```sh\naz batch pool set \\\n  --account-name bafastbrains \\\n  --pool-id dev \\\n  --application-package-references \"molecular-analysis#1.0\"\n```\n\nOne option to easily interact with the CLI is to login to the Batch account:\n\n```sh\naz batch account login \\\n  --name bafastbrains \\\n  --resource-group rg-fastbrains \\\n  --shared-key-auth\n```\n\nAlternatively, if you need to use the keys, add `--shared-key-auth`.\n\nRun a task:\n\n```sh\naz batch task create --task-id sciTask001 --command-line \"echo task001\" --job-id dev-job\n```\n\nView task status:\n\n```sh\naz batch task show \\\n  --job-id dev-job \\\n  --task-id sciTask001\n```\n\nView task output:\n\n```sh\naz batch task file list \\\n  --job-id dev-job \\\n  --task-id sciTask001 \\\n  --output table\n```\n\nIt is possible to create a task with the [`--json-file`][1] option:\n\n\u003e The file containing the task(s) to create in JSON(formatted to match REST API request body). When submitting multiple tasks, accepts either an array of tasks or a TaskAddCollectionParameter. If this parameter is specified, all other parameters are ignored.\n\nAdditional functionality for the CLI is available through extensions:\n\n```sh\naz extension add --name azure-batch-cli-extensions\n```\n\nThe jump box already has System-Assigned Identity. To use it:\n\n```sh\n# Using the System-Assigned identity within the VM\naz login --identity\n```\n\nThis is not required if you use `az batch account login`, but another option to interact with a private endpoint Batch/pools using the jump box:\n\n```sh\nexport AZURE_BATCH_ACCOUNT=\"\"\nexport AZURE_BATCH_ENDPOINT=\"\"\nexport AZURE_BATCH_ACCESS_KEY=\"\"\n```\n\nNow it is possible to use the private endpoints:\n\n```sh\naz batch pool list\n```\n\n## Connect to nodes\n\nUse the node IP and frontend port:\n\n```sh\nssh batch@\u003cip\u003e -p 50000\n```\n\nBatch files are located in the [temporary storage][2].\n\nFor the mounted storage, on [Ubuntu batch][3] the path is `/mnt/batch/tasks/fsmounts/blobs`.\n\n## Metrics\n\nDetailed usage of the `applicationinsights` package can be found in the [full documentation][5]. For details on Azure Monitor custom metrics, [link][6] for the guidelines.\n\nConfiguration of the instruction can be done via `opentelemetry` available [variables][7]:\n\n```sh\nOTEL_RESOURCE_ATTRIBUTES=\"service.namespace=\u003cnamespace\u003e,service.instance.id=\u003cinstance\u003e\"\nOTEL_SERVICE_NAME=\"\u003cservice\u003e\"\n```\n\nFrom the renewed [official docs][4]:\n\n```sh\npython3 -m venv .venv\nsource .venv/bin/activate\n\npython3 -m pip install --upgrade pip\npython3 -m pip install -r requirements.txt\n```\n\nFor troubleshooting:\n\n```sh\nwhich python\n.venv/bin/python\n```\n\n## User access\n\nA user named `BatchUser` will be created.\n\nThis user has `Reader` permissions in the resource group, and `Contributor` access in the Batch account.\n\nIt should be possible to created Pools with this user.\n\n## Reference\n\n- [Azure Batch permissions](https://techcommunity.microsoft.com/t5/azure-paas-blog/the-usage-of-managed-identity-in-the-azure-batch-account-and/ba-p/3607014)\n- [Private Endpoints + VM](https://learn.microsoft.com/en-us/troubleshoot/azure/general/azure-batch-pool-creation-failure#cause-1-public-network-access-is-disabled-but-batch-account-doesnt-have-private-endpoint)\n\n[1]: https://learn.microsoft.com/en-us/cli/azure/batch/task?view=azure-cli-latest#az-batch-task-create\n[2]: https://learn.microsoft.com/en-us/azure/batch/files-and-directories\n[3]: https://learn.microsoft.com/en-us/azure/batch/virtual-file-mount?tabs=linux\n[4]: https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/\n[5]: https://shipit.dev/python-appinsights/\n[6]: https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/metrics-custom-overview\n[7]: https://learn.microsoft.com/en-us/azure/azure-monitor/app/opentelemetry-configuration?tabs=python\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepomatti%2Fazure-batch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fepomatti%2Fazure-batch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepomatti%2Fazure-batch/lists"}