{"id":20841059,"url":"https://github.com/benchopt/benchmark_resnet_classif","last_synced_at":"2025-10-19T14:33:48.838Z","repository":{"id":38192828,"uuid":"471037137","full_name":"benchopt/benchmark_resnet_classif","owner":"benchopt","description":"Benchopt benchmark for ResNet fitting on a classification task","archived":false,"fork":false,"pushed_at":"2023-09-19T00:08:17.000Z","size":728,"stargazers_count":11,"open_issues_count":22,"forks_count":4,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-05-08T22:06:32.964Z","etag":null,"topics":["benchmark","deep-learning","image-classification","optimization","pytorch","resnet","tensorflow"],"latest_commit_sha":null,"homepage":"https://benchopt.github.io/results/benchmark_resnet_classif.html","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/benchopt.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-03-17T15:33:18.000Z","updated_at":"2025-04-03T01:14:41.000Z","dependencies_parsed_at":"2023-01-21T14:02:43.517Z","dependency_job_id":null,"html_url":"https://github.com/benchopt/benchmark_resnet_classif","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":"benchopt/template_benchmark","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benchopt%2Fbenchmark_resnet_classif","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benchopt%2Fbenchmark_resnet_classif/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benchopt%2Fbenchmark_resnet_classif/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benchopt%2Fbenchmark_resnet_classif/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/benchopt","download_url":"https://codeload.github.com/benchopt/benchmark_resnet_classif/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253154974,"owners_count":21862622,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","deep-learning","image-classification","optimization","pytorch","resnet","tensorflow"],"created_at":"2024-11-18T01:18:39.968Z","updated_at":"2025-10-19T14:33:48.710Z","avatar_url":"https://github.com/benchopt.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Benchmark repository for ResNet fitting on classification\n=====================\n|Build Status| |Python 3.6+| |TensorFlow 2.8+| |PyTorch 1.10+| |PyTorch-Lightning 1.6+|\n\nBenchOpt is a package to simplify and make more transparent and\nreproducible the comparisons of optimization algorithms.\nThis benchmark is dedicated to solver of the ResNet classification fitting problem:\n\n.. math::\n\n    \\min_{w} \\sum_i L(f_w(x_i), y_i)\n\nwhere :math:`i` is the sample index, :math:`x_i` is the input image, :math:`y_i` is the sample label, and :math:`L` is the cross-entropy loss function.\n\n\nUse\n--------\n\nThis benchmark can be run using the following commands:\n\n.. code-block::\n\n   $ pip install -U benchopt\n   $ git clone https://github.com/benchopt/benchmark_resnet_classif\n   $ benchopt run benchmark_resnet_classif\n\nWhile this command would run the entire benchmark, which includes several models, datasets and solvers, sequentially,\nyou can restrict the run to a specific model, dataset and/or solver by passing the corresponding arguments.\nFor example if I want to run the benchmark for the ResNet18 model on CIFAR10 dataset with the Adam solver, without a validation set, and for a single random seed:\n\n.. code-block::\n\n\t$ benchopt run . -o \"*[model_size=18]\" -d \"cifar[random_state=42,with_validation=False]\" -s \"adam-torch[batch_size=128,coupled_weight_decay=0.0,data_aug=True,decoupled_weight_decay=0.02,lr_schedule=cosine]\"  --max-runs 200 --n-repetitions 1\n\nUse `benchopt run -h` for more details about these options, or visit https://benchopt.github.io/api.html.\n\nExtension\n---------\n\nIf you want to add a new solver, you need to inherit one of the base solver classes from PyTorch, TensorFlow or PyTorch-Lightning.\nFor example, to implement a new PyTorch-based solver with the Adam optimizer, you can add the following python file in the `solvers \u003csolvers\u003e`_ folder:\n\n.. code:: python\n\n   from benchopt import BaseSolver, safe_import_context\n\n   with safe_import_context() as import_ctx:\n      import torch\n\n\n   class Solver(BaseSolver):\n\n      parameters = {\n         'lr': [1e-3],\n         'batch_size': [128],\n      }\n\n      def skip(self, framework, **_kwargs,):\n         if framework != 'pytorch':\n            return True, 'Not a torch dataset/objective'\n         return False, None\n\n      def set_objective(self, dataset, model_init_fn, **_kwargs):\n         self.model_init_fn = model_init_fn\n         self.dataloader = torch.utils.data.DataLoader(\n            dataset,\n            batch_size=self.batch_size,\n            num_workers=10,\n            persistent_workers=True,\n            pin_memory=True,\n            shuffle=True,\n         )\n\n      @staticmethod\n      def get_next(stop_val):\n         # evaluate the model at every epoch.\n         return stop_val + 1\n\n      def run(self, callback):\n         # model weight initialization\n         model = self.model_init_fn()\n         criterion = torch.nn.CrossEntropyLoss()\n\n         max_epochs = callback.stopping_criterion.max_runs\n         optimizer = torch.optim.Adam(model.parameters(), lr=self.lr)\n         # Initial evaluation\n         while callback(model):\n            for X, y in self.dataloader:\n                  if torch.cuda.is_available():\n                     X, y = X.cuda(), y.cuda()\n                  optimizer.zero_grad()\n                  loss = criterion(model(X), y)\n                  loss.backward()\n\n                  optimizer.step()\n\n         self.model = model\n\n      def get_result(self):\n         return self.model\n\nIf you want to use a more complex solver, using a learning rate scheduler, as well as data augmentation,\nyou can subclass the `TorchSolver \u003cutils/torch_solver.py\u003e`_ class we provide:\n\n.. code:: python\n\n   from benchopt import safe_import_context\n\n\n   with safe_import_context() as import_ctx:\n      from torch.optim import Adam\n\n   TorchSolver = import_ctx.import_from('torch_solver', 'TorchSolver')\n\n\n   class Solver(TorchSolver):\n      \"\"\"Adam solver\"\"\"\n      name = 'Adam-torch'\n\n      # any parameter defined here is accessible as a class attribute\n      parameters = {\n         **TorchSolver.parameters,\n         'lr': [1e-3],\n         'weight_decay': [0.0, 5e-4],\n      }\n\n      def set_objective(self, **kwargs):\n         super().set_objective(**kwargs)\n         self.optimizer_klass = Adam\n         self.optimizer_kwargs = dict(\n               lr=self.lr,\n               weight_decay=self.weight_decay,\n         )\n\nIf you want to modify the data augmentation policy you will have to override the :code:`set_objective` function.\nIf you want to use a different learning rate scheduler, you will have to override the :code:`set_lr_schedule_and_optimizer` function.\nWe are in the process of making these functions more modular to enable easier customization.\n\n\n\n.. |Build Status| image:: https://github.com/benchopt/benchmark_resnet_classif/workflows/Tests/badge.svg\n   :target: https://github.com/benchopt/benchmark_resnet_classif/actions\n.. |Python 3.6+| image:: https://img.shields.io/badge/python-3.6%2B-blue\n   :target: https://www.python.org/downloads/release/python-360/\n.. |TensorFlow 2.8+| image:: https://img.shields.io/badge/TensorFlow-2.8%2B-orange\n   :target: https://www.tensorflow.org/?hl=fr\n.. |PyTorch 1.10+| image:: https://img.shields.io/badge/PyTorch-1.10%2B-red\n   :target: https://pytorch.org/\n.. |PyTorch-Lightning 1.6+| image:: https://img.shields.io/badge/PyTorch--Lightning-1.6%2B-blueviolet\n   :target: https://pytorch-lightning.readthedocs.io/en/latest/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenchopt%2Fbenchmark_resnet_classif","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbenchopt%2Fbenchmark_resnet_classif","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenchopt%2Fbenchmark_resnet_classif/lists"}