{"id":31580567,"url":"https://github.com/anto18671/pretraining-custom-timm","last_synced_at":"2026-05-19T07:33:19.370Z","repository":{"id":312495977,"uuid":"1047688630","full_name":"anto18671/pretraining-custom-timm","owner":"anto18671","description":"A flexible and extensible PyTorch pretraining script built atop the timm library.","archived":false,"fork":false,"pushed_at":"2025-09-03T19:42:09.000Z","size":21,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-04T05:52:56.424Z","etag":null,"topics":["computer-vision","pretraining","pytorch","timm"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/anto18671.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-31T01:49:53.000Z","updated_at":"2025-09-15T00:10:39.000Z","dependencies_parsed_at":"2025-08-31T04:11:56.081Z","dependency_job_id":"8750e573-ea63-4647-a274-79d8d3435caa","html_url":"https://github.com/anto18671/pretraining-custom-timm","commit_stats":null,"previous_names":["anto18671/pretraining-custom-timm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/anto18671/pretraining-custom-timm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anto18671%2Fpretraining-custom-timm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anto18671%2Fpretraining-custom-timm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anto18671%2Fpretraining-custom-timm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anto18671%2Fpretraining-custom-timm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/anto18671","download_url":"https://codeload.github.com/anto18671/pretraining-custom-timm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anto18671%2Fpretraining-custom-timm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278520487,"owners_count":26000493,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-05T02:00:06.059Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","pretraining","pytorch","timm"],"created_at":"2025-10-05T21:15:05.227Z","updated_at":"2025-10-05T21:15:06.680Z","avatar_url":"https://github.com/anto18671.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pretraining-custom-timm\n\nA flexible and extensible **PyTorch pretraining script** built atop the `timm` library. Designed for pretraining custom vision models with modern techniques like mixed precision, advanced augmentations, learning rate scheduling, EMA, and checkpointing. Updated **August 31, 2025**.\n\n---\n\n## Features\n\n- **Model Configuration**: Leverages `timm` for easily switching architectures.\n- **Data Loading \u0026 Augmentation**: Includes standard and advanced augment techniques (e.g., RandAugment, RandomErasing).\n- **Mixed Precision Training**: Built-in support for AMP using PyTorch's `torch.amp`.\n- **Learning Rate Scheduling**: Configurable warmup + cosine decay.\n- **EMA (Exponential Moving Average)**: Maintains smoothed model weights for robust validation performance.\n- **Checkpoint Strategy**: Saves robust checkpoints: `last.pth`, `best.pth`, `last_ema.pth`, and `best_ema.pth`.\n- **Training Logging**: Progress tracking with `tqdm`; summary of model via `torchsummary`.\n\n---\n\n## Installation\n\n```bash\ngit clone https://github.com/anto18671/pretraining-custom-timm.git\ncd pretraining-custom-timm\npip install -r requirements.txt\n```\n\n**Requirements** (aligning with this repo’s `requirements.txt`):\n\n- `torch`\n- `torchvision`\n- `timm`\n- `tqdm`\n- `torchsummary`\n- `datasets` _(if Hugging Face datasets are used)_\n  _(Adjust based on what's listed in the actual `requirements.txt`.)_\n\n---\n\n## Usage\n\nRun the training script:\n\n```bash\npython train.py\n```\n\n### Default Hyperparameters\n\n```text\nImage size: 256\nBatch size: 128\nEpochs: 300\nWarmup epochs: 20\nBase LR: scaled as 5e-4 × (batch_size / 512)\nWeight decay: 0.05\nMixup alpha: 0.8\nCutMix alpha: 1.0\nEMA decay: 0.9999\n```\n\n_(Update these defaults if your script uses different values.)_\n\n---\n\n## Checkpoints \u0026 Logging\n\n- **Checkpoints**:\n\n  - `last.pth`: Last epoch weights.\n  - `best.pth`: Best validation performance (raw model).\n  - `last_ema.pth`: EMA model at last epoch.\n  - `best_ema.pth`: EMA model at best validation accuracy.\n\n- **Training Logs**: Displayed via `tqdm` (loss, accuracy, learning rate).\n\n---\n\n## Example Training Console Output\n\n```text\nEpoch 001/300 | train_loss=5.4321 train_acc=0.0123 | \\\nval_loss=5.4210 val_acc=0.0145 | \\\nema_val_loss=5.4100 ema_val_acc=0.0152 | \\\nbest_acc=0.0145 best_ema_acc=0.0152 | lr=0.000012\n```\n\n---\n\n## Additional Usage Examples\n\n### Resuming from a Checkpoint\n\nTo resume training from the best checkpoint:\n\n```python\ncheckpoint = torch.load(\"best.pth\")\nmodel.load_state_dict(checkpoint['state_dict'])\nstart_epoch = checkpoint['epoch'] + 1\n```\n\n### Evaluating EMA Weights\n\n```python\nema_checkpoint = torch.load(\"best_ema.pth\")\nema_model.load_state_dict(ema_checkpoint['state_dict'])\n# Proceed with validation using ema_model...\n```\n\n---\n\n## Future Extensions\n\n- Distributed training with PyTorch (`torch.distributed`).\n- Logging with TensorBoard or Weights \u0026 Biases.\n- Fine-tuning scripts with custom head layers.\n- Additional augmentations and data sampling strategies.\n\n---\n\n## License\n\nThis project is licensed under the **MIT License** — see the `LICENSE` file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanto18671%2Fpretraining-custom-timm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanto18671%2Fpretraining-custom-timm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanto18671%2Fpretraining-custom-timm/lists"}