https://github.com/hughplay/deepcodebase
:building_construction: A template for deep learning projects.
https://github.com/hughplay/deepcodebase
codebase deep-learning docker hydra mnist pytorch pytorch-lightning template
Last synced: about 1 month ago
JSON representation
:building_construction: A template for deep learning projects.
- Host: GitHub
- URL: https://github.com/hughplay/deepcodebase
- Owner: hughplay
- License: mit
- Created: 2022-07-02T14:29:41.000Z (over 3 years ago)
- Default Branch: master
- Last Pushed: 2024-09-13T11:26:30.000Z (about 1 year ago)
- Last Synced: 2024-09-14T01:10:34.842Z (about 1 year ago)
- Topics: codebase, deep-learning, docker, hydra, mnist, pytorch, pytorch-lightning, template
- Language: Python
- Homepage:
- Size: 373 KB
- Stars: 11
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://pytorch.org/get-started/locally/)
[](https://pytorchlightning.ai/)
[](https://hydra.cc)
[](#license)
*A template for deep learning projects.*
[[Use this template]](https://github.com/hughplay/DeepCodebase/generate) [[Document]](https://hongxin2019.github.io/deepcodebase/)
## Pre-requirements
- [Docker](https://docs.docker.com/engine/install)
- [Nvidia-Docker](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker)
- [docker-compose](https://docs.docker.com/compose/install/)
## Quick Start
TLDR; We have prepare the commands in the `Makefile` for you:
```bash
make init # run in host, build a docker image for api server and launch the container
make in # run in host, ctrl + d to exit the container
# experiment
make wandb_login # run in container, login to wandb
make train # run in container, train the model
make table # run in container, generate the latex table
# commit
make copy_git # run in host, copy git config to the container home
make precommit_install # run in container, prepare pre-commit for tidy code
git add . && git commit -m "xxx" # run in container, commit code, it will run pre-commit check
# profiling
python train.py experiment=mnist_lenet debug=profiler # run in container, using pytorch profiler to profile the code
make nsightrun cmd="python train.py experiment=mnist_lenet debug=nsight" # run in container, launch a nsight system profiler
```
Below is the detailed instructions.
**1. Create new project from template**
Click [[Use this template]](https://github.com/hughplay/DeepCodebase/generate) to create a new project from this template.
**2. Clone the new project to local**
You can find the repository url in the new project page.
```bash
git clone
```
**3. Prepare docker environment**
Build the docker image and launch the container with:
```bash
python docker.py startd --build
```
**4. Enter the docker container**
Enter the container with:
```bash
python docker.py
```
The docker container is our development environment.
**5. Train a model**
Launch training with:
```bash
python train.py experiment=mnist_lenet
```
or launch multiple training at once:
```bash
python batchrun.py scripts/train/train_baselines.sh --quotas 1
```
**6. View results on wandb**
The training results will be automatically uploaded to wandb. You can view the results on [wandb.ai](https://wandb.ai).
**7. Generate Latex table**
After training multiple models, you can generate a latex table with:
```bash
python scripts/gen_latex_table.py --table baseline
```
This will generate a table like this:
```latex
\usepackage{booktabs}
\begin{table}[t]
\centering
\small
\caption{Performance on the MNIST dataset. }
\label{tab:baseline}
\begin{tabular}{lr}
\toprule
Model & Accuracy \\
\midrule
DNN & 0.974566 \\
LeNet & \textbf{0.983000} \\
\bottomrule
\end{tabular}
\end{table}
```
**8. Custom your own project**
Modify the code to fit your own deep learning project. Check our [document](https://hongxin19.github.io/deepcodebase/) for more details.
**9. Commit you modifications**
We strongly recommend you to use [pre-commit](https://pre-commit.com/) to check your code before commiting. Before you first commit, you need to install the pre-commit hook with:
```bash
pre-commit install
```
Then, you can commit your modifications with normal git commands.
```bash
git add .
git commit -m "commit message"
```
It will show you the errors if your code does not pass the pre-commit checks. After you fix the errors, you need to git add the modified files and commit again.
## Cite us
Give a star and cite us if you find this project useful.
[](https://github.com/hughplay/DeepCodebase)
```md
[](https://github.com/hughplay/DeepCodebase)
```
*This is a project based on [DeepCodebase](https://github.com/hughplay/DeepCodebase) template.*
``` md
*This is a project based on [DeepCodebase](https://github.com/hughplay/DeepCodebase) template.*
```
## Projects based on DeepCodebase
- [TVR](https://github.com/hughplay/TVR)
## License
DeepCodebase is released under the [MIT license](LICENSE).