{"id":15642968,"url":"https://github.com/warner-benjamin/fastxtend","last_synced_at":"2025-10-08T23:23:53.496Z","repository":{"id":38209205,"uuid":"399950725","full_name":"warner-benjamin/fastxtend","owner":"warner-benjamin","description":"Train fastai models faster (and other useful tools)","archived":false,"fork":false,"pushed_at":"2025-06-04T05:13:21.000Z","size":43370,"stargazers_count":70,"open_issues_count":2,"forks_count":7,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-08-14T09:48:30.420Z","etag":null,"topics":["deep-learning","fastai","machine-learning","python","pytorch"],"latest_commit_sha":null,"homepage":"https://fastxtend.benjaminwarner.dev","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/warner-benjamin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-08-25T20:35:50.000Z","updated_at":"2025-07-15T06:09:39.000Z","dependencies_parsed_at":"2023-10-16T16:26:06.778Z","dependency_job_id":"eefce033-bec3-4c24-a510-2134651543f1","html_url":"https://github.com/warner-benjamin/fastxtend","commit_stats":null,"previous_names":[],"tags_count":24,"template":false,"template_full_name":null,"purl":"pkg:github/warner-benjamin/fastxtend","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/warner-benjamin%2Ffastxtend","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/warner-benjamin%2Ffastxtend/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/warner-benjamin%2Ffastxtend/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/warner-benjamin%2Ffastxtend/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/warner-benjamin","download_url":"https://codeload.github.com/warner-benjamin/fastxtend/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/warner-benjamin%2Ffastxtend/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279000736,"owners_count":26082862,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-08T02:00:06.501Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","fastai","machine-learning","python","pytorch"],"created_at":"2024-10-03T11:58:26.054Z","updated_at":"2025-10-08T23:23:53.466Z","avatar_url":"https://github.com/warner-benjamin.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fastxtend\n\n\n\u003c!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! --\u003e\n\n### Train fastai models faster (and other useful tools)\n\n![fastxtend accelerates\nfastai](https://github.com/warner-benjamin/fastxtend/blob/main/nbs/images/imagenette_benchmark.png?raw=true)\n\nTrain fastai models faster with fastxtend’s [fused\noptimizers](https://fastxtend.benjaminwarner.dev/optimizer.fused.html),\n[Progressive\nResizing](https://fastxtend.benjaminwarner.dev/callback.progresize.html)\ncallback, integrated [FFCV\nDataLoader](https://fastxtend.benjaminwarner.dev/ffcv.tutorial.html),\nand integrated [PyTorch\nCompile](https://fastxtend.benjaminwarner.dev/callback.compiler.html)\nsupport.\n\n## Feature overview\n\n**Train Models Faster**\n\n- Drop in [fused\n  optimizers](https://fastxtend.benjaminwarner.dev/optimizer.fused.html),\n  which are 21 to 293 percent faster then fastai native optimizers.\n- Up to 75% optimizer memory savings with integrated\n  [bitsandbytes](https://github.com/TimDettmers/bitsandbytes) [8-bit\n  optimizers](https://fastxtend.benjaminwarner.dev/optimizer.eightbit.html).\n- Increase GPU throughput and decrease training time with the\n  [Progressive\n  Resizing](https://fastxtend.benjaminwarner.dev/callback.progresize.html)\n  callback.\n- Use the highly optimized [FFCV\n  DataLoader](https://fastxtend.benjaminwarner.dev/ffcv.tutorial.html),\n  fully integrated with fastai.\n- Integrated support for `torch.compile` via the\n  [Compile](https://fastxtend.benjaminwarner.dev/callback.compiler.html)\n  callbacks.\n\n**General Features**\n\n- Fused implementations of modern optimizers, such as\n  [Adan](https://fastxtend.benjaminwarner.dev/optimizer.adan.html),\n  [Lion](https://fastxtend.benjaminwarner.dev/optimizer.lion.html), \u0026\n  [StableAdam](https://fastxtend.benjaminwarner.dev/optimizer.stableadam.html).\n- Hugging Face [Transformers\n  compatibility](https://fastxtend.benjaminwarner.dev/text.huggingface.html)\n  with fastai\n- Flexible [metrics](https://fastxtend.benjaminwarner.dev/metrics.html)\n  which can log on train, valid, or both. Backwards compatible with\n  fastai metrics.\n- Easily use [multiple\n  losses](https://fastxtend.benjaminwarner.dev/multiloss.html) and log\n  each individual loss on train and valid.\n- [Multiple\n  profilers](https://fastxtend.benjaminwarner.dev/callback.profiler.html)\n  for profiling training and identifying bottlenecks.\n- A fast [Exponential Moving\n  Average](https://fastxtend.benjaminwarner.dev/callback.ema.html)\n  callback for smoother training.\n\n**Vision**\n\n- Apply\n  [`MixUp`](https://fastxtend.benjaminwarner.dev/callback.cutmixup.html#mixup),\n  [`CutMix`](https://fastxtend.benjaminwarner.dev/callback.cutmixup.html#cutmix),\n  or Augmentations at once with\n  [`CutMixUp`](https://fastxtend.benjaminwarner.dev/callback.cutmixup.html#cutmixup)\n  or\n  [`CutMixUpAugment`](https://fastxtend.benjaminwarner.dev/callback.cutmixup.html#cutmixupaugment).\n- Additional [image\n  augmentations](https://fastxtend.benjaminwarner.dev/vision.augment.batch.html).\n- Support for running fastai [batch transforms on\n  CPU](https://fastxtend.benjaminwarner.dev/vision.data.html).\n- More\n  [attention](https://fastxtend.benjaminwarner.dev/vision.models.attention_modules.html)\n  and\n  [pooling](https://fastxtend.benjaminwarner.dev/vision.models.pooling.html)\n  modules\n- A flexible implementation of fastai’s\n  [`XResNet`](https://fastxtend.benjaminwarner.dev/vision.models.xresnet.html#xresnet).\n\nCheck out the documentation for additional splitters, callbacks,\nschedulers, utilities, and more.\n\n## Documentation\n\n\u003chttps://fastxtend.benjaminwarner.dev\u003e\n\n## Install\n\nfastxtend is avalible on pypi:\n\n``` bash\npip install fastxtend\n```\n\nfastxtend can be installed with task-specific dependencies for `vision`,\n`ffcv`, `text`, `audio`, or `all`:\n\n``` bash\npip install \"fastxtend[all]\"\n```\n\nTo easily install most prerequisites for all fastxtend features, use\n[Conda](https://docs.conda.io/en/latest) or\n[Miniconda](https://docs.conda.io/en/latest/miniconda.html):\n\n``` bash\nconda create -n fastxtend python=3.11 \"pytorch\u003e=2.1\" torchvision torchaudio \\\npytorch-cuda=12.1 fastai nbdev pkg-config libjpeg-turbo \"opencv\u003c4.8\" tqdm psutil \\\nterminaltables numpy \"numba\u003e=0.57\" librosa timm kornia rich typer wandb \\\n\"transformers\u003e=4.34\" \"tokenizers\u003e=0.14\" \"datasets\u003e=2.14\" ipykernel ipywidgets \\\n\"matplotlib\u003c3.8\" -c pytorch -c nvidia -c fastai -c huggingface -c conda-forge\n\nconda activate fastxtend\n\npip install \"fastxtend[all]\"\n```\n\nreplacing `pytorch-cuda=12.1` with your preferred [supported version of\nCuda](https://pytorch.org/get-started/locally).\n\nTo create an editable development install:\n\n``` bash\ngit clone https://github.com/warner-benjamin/fastxtend.git\ncd fastxtend\npip install -e \".[dev]\"\n```\n\n## Usage\n\nLike fastai, fastxtend provides safe wildcard imports using python’s\n`__all__`.\n\n``` python\nfrom fastai.vision.all import *\nfrom fastxtend.vision.all import *\nfrom fastxtend.ffcv.all import *\n```\n\nIn general, import fastxtend after all fastai imports, as fastxtend\nmodifies fastai. Any method modified by fastxtend is backwards\ncompatible with the original fastai code.\n\n## Examples\n\nUse a fused ForEach optimizer:\n\n``` python\nLearner(..., opt_func=adam(foreach=True))\n```\n\nOr a bitsandbytes 8-bit optimizer:\n\n``` python\nLearner(..., opt_func=adam(eightbit=True))\n```\n\nSpeed up image training using Progressive Resizing:\n\n``` python\nLearner(... cbs=ProgressiveResize())\n```\n\nLog an accuracy metric on the training set as a smoothed metric and\nvalidation set like normal:\n\n``` python\nLearner(..., metrics=[Accuracy(log_metric=LogMetric.Train, metric_type=MetricType.Smooth),\n                      Accuracy()])\n```\n\nLog multiple losses as individual metrics on train and valid:\n\n``` python\nmloss = MultiLoss(loss_funcs=[nn.MSELoss, nn.L1Loss],\n                  weights=[1, 3.5], loss_names=['mse_loss', 'l1_loss'])\n\nLearner(..., loss_func=mloss, metrics=RMSE(), cbs=MultiLossCallback)\n```\n\nCompile a model with `torch.compile`:\n\n``` python\nfrom fastxtend.callback import compiler\n\nlearn = Learner(...).compile()\n```\n\nProfile a fastai training loop:\n\n``` python\nfrom fastxtend.callback import simpleprofiler\n\nlearn = Learner(...).profile()\nlearn.fit_one_cycle(2, 3e-3)\n```\n\n## Benchmark\n\nTo run the benchmark on your own machine, see the [example\nscripts](https://github.com/warner-benjamin/fastxtend/tree/main/examples)\nfor details on how to replicate.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwarner-benjamin%2Ffastxtend","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwarner-benjamin%2Ffastxtend","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwarner-benjamin%2Ffastxtend/lists"}