{"id":20826765,"url":"https://github.com/cedrickchee/dawnbench-analysis","last_synced_at":"2025-03-12T07:23:42.390Z","repository":{"id":138118153,"uuid":"140240041","full_name":"cedrickchee/dawnbench-analysis","owner":"cedrickchee","description":"DAWNBench analysis of CIFAR-10 time-to-accuracy.","archived":false,"fork":false,"pushed_at":"2018-07-09T09:23:24.000Z","size":452,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-18T17:49:44.499Z","etag":null,"topics":["benchmark","benchmark-scripts","cifar10","dawnbench","deeplearning","machinelearning"],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cedrickchee.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-07-09T06:19:46.000Z","updated_at":"2024-11-01T11:37:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"bceb5925-7d11-4717-9d5c-b884e0f2347f","html_url":"https://github.com/cedrickchee/dawnbench-analysis","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedrickchee%2Fdawnbench-analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedrickchee%2Fdawnbench-analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedrickchee%2Fdawnbench-analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedrickchee%2Fdawnbench-analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cedrickchee","download_url":"https://codeload.github.com/cedrickchee/dawnbench-analysis/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243174035,"owners_count":20248220,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","benchmark-scripts","cifar10","dawnbench","deeplearning","machinelearning"],"created_at":"2024-11-17T23:09:57.601Z","updated_at":"2025-03-12T07:23:42.383Z","avatar_url":"https://github.com/cedrickchee.png","language":"Jupyter Notebook","readme":"# DAWNBench Analysis\n\n## Introduction\n\n[DAWNBench](https://dawn.cs.stanford.edu/benchmark/) is an end-to-end deep learning training and inference benchmark. Computation time and cost are critical resources in building deep models, yet many existing benchmarks focus solely on model accuracy.\n\nThis repository contains the analysis of DAWNBench results. We will analyze two common deep learning workloads, CIFAR-10 and ImageNet time-to-accuracy and training cost across different optimization strategies, model architectures, software frameworks, clouds, and hardware.\n\nFor the start, we will analyze only CIFAR-10 workload across two model architectures:\n\n- Custom Wide ResNet\n- ResNet18 (TODO)\n\n|Rank | Time to 94% Accuracy | Model | Framework | Hardware |\n| --- | --- | --- | --- | --- |\n| 1 | 00:52:11 | Custom Wide ResNet + AdamW + modified 1 cycle policy \u003cbr /\u003e Cedric Chee \u003cbr /\u003e [source](/src/models/wrn) | fastai / PyTorch  0.4.0 | 1 K80 (AWS p2.xlarge) |\n| 2 | 00:55:51 | Custom Wide ResNet + modified 1 cycle policy \u003cbr /\u003e Cedric Chee \u003cbr /\u003e [source](/src/models/wrn) | fastai / PyTorch  0.4.0 | 1 K80 (AWS p2.xlarge) |\n| 3 | 0:59:38 | Custom DarkNet \u003cbr /\u003e Cedric Chee \u003cbr /\u003e [source](https://nbviewer.jupyter.org/github/cedrickchee/fastai/blob/master/courses/dl2/cifar10-darknet.ipynb) | PyTorch 0.3.1 | 1 K80 (AWS p2.xlarge) |\n| 4 | 1:18:22 | Custom Wide ResNet \u003cbr /\u003e fast.ai students team \u003cbr /\u003e [source](https://github.com/fastai/imagenet-fast/commit/1bc5aeec765572397c0ebe4a5d616d03beeeeec1) | fastai / PyTorch 0.4.0 | 1 K80 (AWS p2.xlarge) |\n\n## Analysis\n\n### 1. Custom Wide ResNet fast.ai students team DAWNBench submission (Baseline)\n\n[Jupyter Notebook / Source](https://nbviewer.jupyter.org/github/cedrickchee/dawnbench-analysis/blob/master/src/cifar10_custom_wrn_dawnbench.ipynb#fastai-DAWN-bench-submission)\n\n![Training](/images/cifar10_fastai_dawnbench_submission_training.png)\n\n![Loss and learning rate plot](/images/cifar10_fastai_dawnbench_submission_loss_lr_plot.png)\n\n### 2. My Custom Wide ResNet + AdamW + modified 1 cycle policy hyper-parameters\n\n[Jupyter Notebook / Source](https://nbviewer.jupyter.org/github/cedrickchee/dawnbench-analysis/blob/master/src/cifar10_custom_wrn_adamw.ipynb)\n\n![Training](/images/cifar10_fastai_adamw_training.png)\n\n![Loss and learning rate plot](/images/cifar10_fastai_adamw_loss_lr_plot.png)\n\n### 3. My Custom Wide ResNet + modified 1 cycle policy hyper-parameters\n\n[Jupyter Notebook / Source](https://nbviewer.jupyter.org/github/cedrickchee/dawnbench-analysis/blob/master/src/cifar10_custom_wrn_dawnbench.ipynb#With-tweaks-for-training-on-AWS-p2.xlarge-K80-GPU)\n\n![Training](/images/cifar10_custom_wrn_training.png)\n\n![Loss and learning rate plot](/images/cifar10_custom_wrn_loss_lr_plot.png)\n\n## Code to replicate all analyses\n\n### Instructions for building and running the container.\n\nYou will need to have [Docker](https://docs.docker.com/install/linux/docker-ce/ubuntu/) and [nvidia-docker](https://github.com/NVIDIA/nvidia-docker) installed in order to run this.\n\n1. cd into cloned repo.\n2. `docker build -t cifar10 .`\n3. `./run_container.sh cifar10`\n\nOnce the Docker container started, all you have to do is access Jupyter Notebook through this url, https://localhost:8888 in your web browser. Then, enter `jupyter` as password. Open the notebook and click 'run all cells'.","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcedrickchee%2Fdawnbench-analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcedrickchee%2Fdawnbench-analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcedrickchee%2Fdawnbench-analysis/lists"}