{"id":15683961,"url":"https://github.com/braun-steven/torch-utils","last_synced_at":"2025-10-04T10:54:51.848Z","repository":{"id":88952195,"uuid":"193223867","full_name":"braun-steven/torch-utils","owner":"braun-steven","description":"Utility functions that reoccur in my PyTorch experiments.","archived":false,"fork":false,"pushed_at":"2019-09-27T11:12:13.000Z","size":39,"stargazers_count":10,"open_issues_count":0,"forks_count":2,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-03-31T11:02:13.641Z","etag":null,"topics":["pytorch","utility"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/braun-steven.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-06-22T10:58:57.000Z","updated_at":"2020-08-09T07:47:20.000Z","dependencies_parsed_at":"2023-03-13T18:17:07.928Z","dependency_job_id":null,"html_url":"https://github.com/braun-steven/torch-utils","commit_stats":{"total_commits":24,"total_committers":1,"mean_commits":24.0,"dds":0.0,"last_synced_commit":"e4a7057217de3daf7eeef144ca7b12bc909b8473"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/braun-steven%2Ftorch-utils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/braun-steven%2Ftorch-utils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/braun-steven%2Ftorch-utils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/braun-steven%2Ftorch-utils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/braun-steven","download_url":"https://codeload.github.com/braun-steven/torch-utils/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252897315,"owners_count":21821420,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["pytorch","utility"],"created_at":"2024-10-03T17:09:21.102Z","updated_at":"2025-10-04T10:54:46.798Z","avatar_url":"https://github.com/braun-steven.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Torch Utilities\n\nThis repo contains some utility functions that I constantly reuse when writing experiment code using PyTorch. \n\nFeel free to contribute your helper functions which you could not live without!\n\n## Utility Functions\n\nThe following functions are available:\n\n- `time_delta_now`: Create a human readable time string of the time passed until now, given a timestamp.\n- `ensure_dir`: Ensure that a directory exists.\n- `count_params`: Count the number of learnable parameters in an `nn.Module` object.\n- `generate_run_base_dir`: Generate a base directory for experiment runs.\n- `setup_logging`: Setup python logging with the `logging` module. Starts logging to a file and `stdout`\n- `set_seed`: Set the seed for python, numpy and torch.\n- `set_cuda_device`: Set the `CUDA_VISIBLE_DEVICES` environment variable.\n- `make_multi_gpu`: Convert a `nn.Module` to a multi-gpu module.\n- `load_args`: Load stored commandline arguments from the `argparse` module.\n- `save_args`: Store commandline arguments from `argparse` in a file.\n- `clone_args`: Clone an arguments object from `argparse`.\n- `plot_samples`: Plot `(x, y)` samples in a grid.\n\n## qdaq: Cuda Experiment Queue\n\n`qdaq` makes running multiple PyTorch or Tensorflow experiments on more than one GPU easy.\nThe typical use case is e.g. a gridsearch over hyperparameters for the same experiment with a bunch of GPUs available. Usually it would be necessary to manually split the hyperparameter space and start the experiments on different GPUs. This has multiple drawbacks:\n\n- It's tedious to split experiment setups and start them on the appropriate GPU device by hand.\n- If 10 experiments are sequentially started on GPU0 and 10 on GPU1 it might happen that GPU0 finishes way earlier while GPU1 is still running with a queue of experiments. This results in unused GPU time on GPU0 (bad load balancing).\n\nThe main function is to provide a multiprocessing queue with available cuda devices. The idea is to define a `Job`, e.g. your experiment, create a bunch of jobs with different settings, specify a list of available cuda devices and let the internals handle the rest. In the background each job is put into a queue and grabs a cuda device as soon as it is available.\n\n### Quick Start\n\nThe following is a quick example on how to use qdaq:\n\n```python\nimport torch\nfrom utils.qdaq import Job, start\n\n\n# Create a job class that implements `run`\nclass Foo(Job):\n    def __init__(self, q):\n        # Save some job parameters\n        self.q = q\n\n    def run(self, cuda_device_id):\n        # Get cuda device\n        device = torch.device(f\"cuda:{cuda_device_id}\")\n\n        # Send data to device\n        x = torch.arange(1, 3).to(device)\n\n        # Compute stuff\n        y = x.pow(self.q)\n        print(f\"Cuda device {cuda_device_id}, Exponent {self.q}, Result {y}\")\n\n\nif __name__ == \"__main__\":\n    exps = [Foo(i) for i in range(9)]\n    start(exps, [1, 3])\n\n# Output:\n# Cuda device 3, Exponent 1, Result tensor([1, 2], device='cuda:3')\n# Cuda device 1, Exponent 0, Result tensor([1, 1], device='cuda:1')\n# Cuda device 3, Exponent 2, Result tensor([1, 4], device='cuda:3')\n# Cuda device 1, Exponent 3, Result tensor([1, 8], device='cuda:1')\n# Cuda device 1, Exponent 5, Result tensor([ 1, 32], device='cuda:1')\n# Cuda device 3, Exponent 4, Result tensor([ 1, 16], device='cuda:3')\n# Cuda device 1, Exponent 6, Result tensor([ 1, 64], device='cuda:1')\n# Cuda device 3, Exponent 8, Result tensor([  1, 256], device='cuda:3')\n# Cuda device 1, Exponent 7, Result tensor([  1, 128], device='cuda:1')\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbraun-steven%2Ftorch-utils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbraun-steven%2Ftorch-utils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbraun-steven%2Ftorch-utils/lists"}