{"id":13578129,"url":"https://github.com/inferno-pytorch/speedrun","last_synced_at":"2026-02-18T23:39:05.970Z","repository":{"id":55091989,"uuid":"134756095","full_name":"inferno-pytorch/speedrun","owner":"inferno-pytorch","description":"Research code need not be ugly. ","archived":false,"fork":false,"pushed_at":"2023-09-13T09:41:33.000Z","size":217,"stargazers_count":77,"open_issues_count":3,"forks_count":5,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-02T04:52:56.647Z","etag":null,"topics":["deep-learning","neural-networks","pytorch","research"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/inferno-pytorch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-05-24T18:48:50.000Z","updated_at":"2024-11-06T00:50:00.000Z","dependencies_parsed_at":"2024-11-05T15:38:50.259Z","dependency_job_id":"b0f4c030-cdff-46ba-b038-2ee892564a77","html_url":"https://github.com/inferno-pytorch/speedrun","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inferno-pytorch%2Fspeedrun","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inferno-pytorch%2Fspeedrun/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inferno-pytorch%2Fspeedrun/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inferno-pytorch%2Fspeedrun/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/inferno-pytorch","download_url":"https://codeload.github.com/inferno-pytorch/speedrun/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247366334,"owners_count":20927494,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","neural-networks","pytorch","research"],"created_at":"2024-08-01T15:01:27.765Z","updated_at":"2025-04-05T16:31:54.046Z","avatar_url":"https://github.com/inferno-pytorch.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# speedrun\n\n## What? \n\nA no-strings-attached toolkit to help you deploy and manage your machine learning experiments. The idea is to equip you with the tools you need to have well-documented and reproducible experiments going, but without _getting in your way_. Think of it as a swiss army knife for dealing with the code-hell research projects typically tend to evolve to. \n\n### Installation\nOn python 3.6+:\n\n```bash\n# Clone the repository\ngit clone https://github.com/nasimrahaman/speedrun.git\ncd speedrun/\n# To embark on an adventure, uncomment the following line:\n# git checkout dev\n# Install\npython setup.py install\n```\n\nOptionally, \n\n```bash\n# Install tensorboard\npip install tensorboardX\n# Install dill\npip install dill\n# Install Weights and Biases\npip install wandb\n```\n\n## How? \n\nAt the most basic level, speedrun provides the base-class `BaseExperiment` for your experiments to inherit from. This already enables you to read from configuration files and manages the experiment directories for you, so you have the choice of not worrying about file-paths and other nasty low-level details. \n\nBut in addition, there are numerous fully optional power-ups that you can bolt on to your experiment class to make it what you need. These are called `Mixin`s, and the growing catalogue includes: \n\n- `WandBMixin` for out-of-the-box logging to [Weights and Biases](https://www.wandb.com/).\n- `TensorboardMixin` to log runs locally to a tensorboard log-file (wraps [tensorboardX](https://github.com/lanpa/tensorboard-pytorch)). \n- `WandBSweepMixin` and `SweepRunner` to configure and launch hyper-parameter sweeps with [Weights and Biases Sweeps](https://docs.wandb.com/sweeps). \n- `IOMixin` to supply utilities for e.g. writing text or images to file, progress bars, etc.\n- `MatplotlibMixin` to convert matplotlib figures to images that you can then log to tensorboard with the `TensorboardMixin` or dump to file with the `IOMixin`. \n- `FirelightMixin` to interact with [Firelight](https://github.com/inferno-pytorch/firelight), a tool for visualizing high-dimensional embeddings. \n- `InfernoMixin` to interact with [Inferno](https://github.com/inferno-pytorch/inferno), a tool to abstract away the training loop for your pytorch models. \n- `WaiterMixin` to have your code wait for a running process to finish (and release resources). \n\n... and more underway. Check out [speedrun-springboard](https://github.com/inferno-pytorch/speedrun-springboard) for a prefabricated experimental set-up, or the small pseudo-example below: \n\n```python\nfrom speedrun import BaseExperiment, TensorboardMixin\n\n\nclass MyFirstExperiment(BaseExperiment, TensorboardMixin):\n    def __init__(self):\n        super(MyFirstExperiment, self).__init__()\n        # This is where the magic happens\n        self.auto_setup()\n        # Set up your experiment here\n        ...\n        self.my_cool_module = SomeModule(**self.get('my_cool_module/kwargs'))\n        self.another_component = SomeComponent(**self.get('another_component/kwargs', default={}))\n        # Say you have a component that gets messy and uses unpickleable objects. For checkpointing \n        # to still work, you'll need to tell the base experiment to not try pickle it. \n        self.ugly_messy_component = UglyMessyComponent()\n        self.register_unpickleable('ugly_messy_component')\n        ...\n    \n    def some_basic_logic(self, *args):\n        # ...\n        return self.bundle(result_1=..., result_2=..., result_3=...)\n    \n    def moar_logics(self):\n        # ...\n        # Uh oh, we need a global variable\n        if 'one_time_computation_result' not in self.cache_keys:\n            # Do the one time computation\n            one_time_computation_result = self.some_basic_logic(self.step % 10)\n            self.write_to_cache('one_time_computation_result', one_time_computation_result)\n        else:\n            one_time_computation_result = self.read_from_cache('one_time_computation_result')\n        # ...\n        return self.bundle(result_1=...)\n    \n    def run(self):\n        # ...\n        for iteration in range(self.get('training/num_iterations')):\n            # training maybe? \n            basic_results = self.some_basic_logic()\n            new_result = self.moar_logics(basic_results.result_1, basic_results.result_2)\n            output_sample = ...\n            loss = ...\n            if self.log_scalars_now: \n                self.log_scalar('training/loss', loss)\n            if self.log_images_now: \n                self.log_image('training/output_sample', output_sample)\n            # force=False would checkpoint if the step count matches current iteration\n            self.checkpoint(force=False)\n            # This increments the step counter\n            self.next_step()\n\nif __name__=='__main__': \n    MyFirstExperimet().run()\n```\n\nNow, there are a few simple steps before we can run the first experiment of the project. The subsequent experiments are a breeze! \n\nFirst, we make a directory to store the _experiment templates_ (you can call the directory anything you like).\n```bash\nmkdir templates\n```\nNext, let's make the first experiment template. You can call the template anything you like, but we'll call it `BASIC-X`. \n```bash\nmkdir -p templates/BASIC-X/Configurations\nnano templates/BASIC-X/Configurations/train_config.yml\n```\n\nNow, we paste in the following configuration in `train_config.yml`. Note that the `.../Configurations/train_config.yml` structure is required for speedrun to find it. \n\n```yml\nmy_cool_module:\n  kwargs: \n    a: 1\n    b: 2\nanother_module:\n  kwargs: \n    c: 3\n    d: 4\ntraining: \n  num_iterations: 100000\n  checkpoint_every: 10000\ntensorboard: \n  log_images_every: 100\n  log_scalars_every: 10\n```\n\nFinally, we make a directory for our actual experiments (not the templates) to live in. You can call it anything you like, but we'll call it `experiments`: \n```bash\nmkdir experiments\n```\n\nThat's it, we're all set! To launch our first experiment, which we call `BASIC-0`, we could do:  \n```bash\npython my_experiment.py experiments/BASIC-0 --inherit templates/BASIC-X\n```\nThis will create a directory `experiments/BASIC-0` with multiple subdirectories. The configuration will be dumped in `experiments/BASIC-0/Configurations`, the tensorboard logs in `experiments/BASIC-0/Logs` and the checkpoints in `experiments/BASIC-0/Weights`. \n\nSo you fire up your first experiments but you think the keyword argument `a` of `my_cool_module` should instead be `42` and `d` of `another_module` should be `21`. All you need to do is: \n```bash\npython my_experiment.py experiments/BASIC-1 --inherit experiments/BASIC-0 --config.my_cool_module.kwargs.a 42 --config.another_module.kwargs.d 21\n```\n\nThis will _inherit_ the configuration from `BASIC-0`, but override `a` in kwargs of `my_cool_module` and `d` in kwargs of `another_module`. The resulting configuration will be dumped in `experiments/BASIC-1/Configurations/train_config.yml` for future experiments to inherit from! This way, you can iterate over your experiments and be confident that every run is self-contained and reproducible. To know the exact difference between the two experiments, you can always: \n```bash\ndiff experiments/BASIC-0/Configurations/train_config.yml experiments/BASIC-1/Configurations/train_config.yml\n```\n\nThe tools might be nice, but it's not just just about that - organizing experiments in classes is a great way of reusing code, which in turn helps keep your experiments reproducible. Say when you're done with the first round of experiments, it's super easy to iterate on your ideas simply by inheriting from your `MyFirstExperiment`, perhaps in a different file: \n\n```python\nfrom main import MyFirstExperiment\n\nclass MySecondExperiment(MyFirstExperiment):\n    def moar_logics(self):\n        # Your shiny new logics go in here\n        # ...\n        return self.bundle(result_1=...)\n\nif __name__=='__main__':\n    MySecondExperiment().run()\n```\n\nThis way, when you fix a bug in `MyFirstExperiment.some_basic_logic`, it's automatically fixed in `MySecondExperiment` as well. Fine print: it's hard to know in advance what parts of the experiment would eventually need to be replaced - so you might need to refactor `MyFirstExperiment` and move bits of logic to their own methods, which you can then overload in `MySecondExperiment`. But more often than not, it's totally worth the effort. \n\n## Why?\n![shitcode](https://i.imgur.com/qG08mar.jpg)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finferno-pytorch%2Fspeedrun","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finferno-pytorch%2Fspeedrun","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finferno-pytorch%2Fspeedrun/lists"}