{"id":34057750,"url":"https://github.com/arawxx/torch-utils","last_synced_at":"2026-04-02T01:01:34.795Z","repository":{"id":228095882,"uuid":"773150265","full_name":"arawxx/torch-utils","owner":"arawxx","description":"A library containing useful and frequently used PyTorch functions and classes.","archived":false,"fork":false,"pushed_at":"2024-03-28T00:51:02.000Z","size":24,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-12-16T00:13:49.111Z","etag":null,"topics":["deep-learning","deeplearning","fine-tuning","finetuning","library","llm","llms","pytorch","scheduler","schedulers","torch","torch-utils","utils","utils-library"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/arawxx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-03-16T22:07:53.000Z","updated_at":"2024-12-14T06:31:00.000Z","dependencies_parsed_at":"2024-03-22T11:28:41.256Z","dependency_job_id":null,"html_url":"https://github.com/arawxx/torch-utils","commit_stats":null,"previous_names":["arawxx/torch-utils"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/arawxx/torch-utils","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arawxx%2Ftorch-utils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arawxx%2Ftorch-utils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arawxx%2Ftorch-utils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arawxx%2Ftorch-utils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/arawxx","download_url":"https://codeload.github.com/arawxx/torch-utils/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arawxx%2Ftorch-utils/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31293631,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T21:15:39.731Z","status":"ssl_error","status_checked_at":"2026-04-01T21:15:34.046Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","deeplearning","fine-tuning","finetuning","library","llm","llms","pytorch","scheduler","schedulers","torch","torch-utils","utils","utils-library"],"created_at":"2025-12-14T03:51:53.504Z","updated_at":"2026-04-02T01:01:34.779Z","avatar_url":"https://github.com/arawxx.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# torch-utils\n[![Downloads](https://static.pepy.tech/badge/pytorch-utilities)](https://pepy.tech/project/pytorch-utilities)\n\nThis repository contains useful functions and classes for Deep Learning engineers using PyTorch.\n\n# Installation\n\nYou can install this package using pip. The name of the package in PyPI is **pytorch-utilities**:\n\n`pip install pytorch-utilities`\n\n## Cosine Annealing with Linear Warmup Learning Rate\n\nUsing this scheduler is as simple as using a default PyTorch scheduler.\n\nExample usage:\n\n```python\nimport torch\nfrom torch.optim import AdamW\nfrom torchutils.schedulers import CosineAnnealingLinearWarmup\n\n\n# Initialize your model and dataloader\n# model = ...\n# dataloader = ...\n# loss_fn = ...\n\n# Initialize the optimizer and scheduler\noptimizer = AdamW(model.parameters(), lr=0.0005)\nscheduler = CosineAnnealingLinearWarmup(optimizer, warmup_epochs=5, max_epochs=100)\n\n# If you want to step the scheduler after each iteration (batch), adjust the warmup_epochs and max_epochs accordingly\n# scheduler = CosineAnnealingLinearWarmup(optimizer, warmup_epochs=5 * len(dataloader), max_epochs=100 * len(dataloader))\n\n# Training loop\nfor epoch in range(100):\n    for inputs, targets in dataloader:\n        optimizer.zero_grad()\n  \n        # Forward pass\n        outputs = model(inputs)\n  \n        # Compute loss\n        loss = loss_fn(outputs, targets)\n  \n        # Backward pass and optimization\n        loss.backward()\n        optimizer.step()\n\n        # If you want to step the scheduler after each iteration (batch), uncomment the following line\n        # scheduler.step()\n  \n    # If you're stepping the scheduler after each epoch, do it here\n    scheduler.step()\n```\n\n## Layer-wise Learning Rate Decay\n\nUsing `layerwise_lrd`, you can set different learning rates for different layers in your model, from the first layer to the last in an ascending order. This is a widely used fine-tuning technique in Deep Vision models that ensures the model keeps most of its learned parameters in the first layers, as the features extracted in these layers are usually low level such as edges and shapes which are beneficial in most image domains and do not need much of a change.\n\nCurrently, only ViT models implemented in `timm`, or with layer names like the ones implemented in it are supported.\n\nExample Usage:\n\n```python\nimport timm\nimport torch\nfrom torchutils.schedulers import layerwise_lrd\n\n\n# Load the model\nmodel = timm.create_model('vit_base_patch14_dinov2.lvd142m', num_classes=1000)\n\n# Fetch model's parameter groups (in place of `model.parameters()`)\nparam_groups = layerwise_lrd(\n    model,\n    weight_decay=0.05,\n    no_weight_decay_list=model.no_weight_decay(),\n    layer_decay=0.75,\n)\n\n# Set the optimizer\noptimizer = torch.optim.AdamW(param_groups, lr=0.001)\n\n# Rest of your training code as usual\n# ...\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farawxx%2Ftorch-utils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farawxx%2Ftorch-utils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farawxx%2Ftorch-utils/lists"}