https://github.com/ethho/deeplabcut
Fork of DeepLabCut/DeepLabCut
https://github.com/ethho/deeplabcut
Last synced: 10 months ago
JSON representation
Fork of DeepLabCut/DeepLabCut
- Host: GitHub
- URL: https://github.com/ethho/deeplabcut
- Owner: ethho
- License: lgpl-3.0
- Created: 2021-02-08T18:14:30.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2022-10-22T13:01:33.000Z (over 3 years ago)
- Last Synced: 2025-06-21T10:41:59.522Z (about 1 year ago)
- Language: Python
- Size: 142 MB
- Stars: 0
- Watchers: 1
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Governance: docs/Governance.md
- Roadmap: docs/roadmap.md
- Authors: AUTHORS
Awesome Lists containing this project
README
# Ethan's Fork of DeepLabCut v2.1.10
The purpose of this fork is to enable usage of DeepLabCut v2.1.10 on TACC GPU systems.
## Usage
### (Optional) Build and push the Docker image
1. Change the `IMAGE` variable in the [Makefile](./Makefile) to your desired Docker tag
2. `make image` to build the image using Docker
3. `make push` to push the built image
4. `make shell` to run a container in Docker and open an interactive bash session
### Pull and run the Docker image via idev and Singularity on TACC GPU
1. Open an SSH session on a TACC system that supports GPU computation
2. Clone this repository to your $WORK directory on a GPU-enabled TACC system. Note: if your workflow is I/O intensive, please copy your working directory to $SCRATCH; please see [TACC's I/O best practices](https://portal.tacc.utexas.edu/tutorials/managingio) for details.
```bash
cd $WORK # or cd $SCRATCH if I/O intensive
git clone https://github.com/eho-tacc/DeepLabCut.git deeplabcut-fork
cd deeplabcut-fork
```
3. Change `IMAGE` in the [Makefile](./Makefile) to the same tag that you specified above. This is not necessary if you are okay with using the [image I have pushed to Dockerhub](https://hub.docker.com/repository/docker/enho/deeplabcut).
4. Change the `SIF` variable in the [Makefile](./Makefile) to a file path (usually somewhere on $WORK) where you would like to store the `*.sif` file. By default, this is set to a path in the current working directory.
5. Launch an idev session and load the following modules:
1. `module load cuda/10.0`
2. `module load nccl cudnn tacc-singularity`
3. My `module list`:
```bash
Currently Loaded Modules:
1) intel/19.1.1 4) autotools/1.2 7) pmix/3.1.4 10) TACC 13) cudnn/7.6.2 (g)
2) impi/19.0.9 5) python3/3.7.0 8) hwloc/1.11.12 11) cuda/10.0 (g) 14) ooops/1.4
3) git/2.24.1 6) cmake/3.16.1 9) xalt/2.10.2 12) nccl/2.4.7 (g) 15) tacc-singularity/3.6.3
```
6. In the idev session, pull the Docker image to a `*.sif` file:
```bash
idev
make sif
```
7. `make sing-dlc-demo` to run a [DeepLabCut demo notebook](https://github.com/DeepLabCut/DeepLabCut/blob/06c4a16828c6c335f7d332da3060516d29857893/examples/Demo_labeledexample_Openfield.ipynb) as a `*.py` script (converted via nbconvert)
* This script was generated using the following command:
```bash
python3 -m nbconvert --to script examples/Demo_labeledexample_Openfield.ipynb
```
8. Alternatively, `make sing-shell` to run a container in Singularity and open an interactive bash session
### Running Jupyter within the Singularity image on TACC GPU
This workflow is currently supported for Maverick2 and Frontera GPU nodes. It is similar to running the sbatch script `/share/doc/slurm/job.jupyter`, except that it launches `jupyter-notebook` from within the image built [above](#optional-build-and-push-the-docker-image).
1. Complete steps 1-6 in the idev workflow described [above](#pull-and-run-the-docker-image-via-idev-and-singularity-on-tacc-gpu)
2. Change the `ALLOCATION` in the [Makefile](./Makefile) from "SD2E-Community" to a valid allocation. You can view your allocations on the [TACC User Portal](https://portal.tacc.utexas.edu/projects-and-allocations).
3. From a login node, `make jupyter-mav2` or `make jupyter-frontera` to launch a SLURM job running Jupyter in the `SIMG` container, on a Maverick2 GTX node or Fronterat RTX node, respectively. This step is similar to running `sbatch /share/doc/slurm/job.jupyter`.
4. Wait patiently until `tail -f ./jupyter.out` prints a URL to which you should direct your web browser.
5. Make sure to `scancel` your job after you are done working.
## Additional Docs
* [Singularity on TACC systems](https://containers-at-tacc.readthedocs.io/en/latest/singularity/01.singularity_basics.html)
* [DeepLabCut](https://github.com/DeepLabCut/DeepLabCut)
* [TACC-ML base images](https://github.com/TACC/tacc-ml)
----
[](https://badge.fury.io/py/deeplabcut)
[](https://pepy.tech/project/deeplabcut)
[](https://pypistats.org/packages/deeplabcut)
[](https://github.com/AlexEMG/DeepLabCut)
[](CONTRIBUTING.md)
[](https://www.gnu.org/licenses/lgpl-3.0)
[](https://forum.image.sc/tag/deeplabcut)
[](https://gitter.im/DeepLabCut/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
[](https://twitter.com/DeepLabCut)
[](https://github.com/AlexEMG/DeepLabCut)
DeepLabCut is a toolbox for markerless pose estimation of animals performing various tasks. [Read a short development and application summary below](https://github.com/AlexEMG/DeepLabCut#why-use-deeplabcut). As long as you can see (label) what you want to track, you can use this toolbox, as it is animal and object agnostic.
**Latest updates:**
:purple_heart: DeepLabCut supports multi-animal pose estimation (BETA release, plese give us your feedback! `pip install deeplabcut==2.2b8`).
:purple_heart: We have a real-time package available! http://DLClive.deeplabcut.org
# [Installation: how to install DeepLabCut](docs/installation.md)
Very quick start: `pip install deeplabcut`
* you also need tensorflow and wxPython see [here](https://github.com/DeepLabCut/DeepLabCut/blob/master/conda-environments/README.md#creating-your-own-customized-conda-env-recommended-route-for-linux-ubuntu-centos-mint-etc)
# [Documentation: The DeepLabCut Process](docs/UseOverviewGuide.md)
An overview of the pipeline and workflow for project management. For a step-by-step user guide, please also read the [Nature Protocols paper](https://doi.org/10.1038/s41596-019-0176-0)!
For a deeper understanding and more resources for you to get started with Python and DeepLabCut, please check out our free online course! http://DLCcourse.deeplabcut.org
# [DEMO the code](/examples)
We provide several Jupyter Notebooks: one that walks you through a demo dataset to test your installation, and another Notebook to run DeepLabCut from the beginning on your own data. We also show you how to use the code in Docker, and on Google Colab.
# Why use DeepLabCut?
In 2018, we demonstrated the capabilities for [trail tracking](https://vnmurthylab.org/), [reaching in mice](http://www.mousemotorlab.org/) and various Drosophila behaviors during egg-laying (see [Mathis et al.](https://www.nature.com/articles/s41593-018-0209-y) for details). There is, however, nothing specific that makes the toolbox only applicable to these tasks and/or species. The toolbox has already been successfully applied (by us and others) to [rats](http://www.mousemotorlab.org/deeplabcut), humans, various fish species, bacteria, leeches, various robots, cheetahs, [mouse whiskers](http://www.mousemotorlab.org/deeplabcut) and [race horses](http://www.mousemotorlab.org/deeplabcut). DeepLabCut utilizes the feature detectors (ResNets + readout layers) of one of the state-of-the-art algorithms for human pose estimation by Insafutdinov et al., called DeeperCut, which inspired the name for our toolbox (see references below). Furthermore, we have added faster variants with MobileNetV2 backbones (see [Pretraining boosts out-of-domain robustness for pose estimation](https://arxiv.org/abs/1909.11229)). Additionally, we have improved the inference speed and provided additional augmentation methods (via tensorpack and imgaug), and added real-time and mutli-animal support in a beta release (more to come ...)
**Left:** Due to transfer learning it requires **little training data** for multiple, challenging behaviors (see [Mathis et al. 2018](https://www.nature.com/articles/s41593-018-0209-y) for details). **Mid Left:** The feature detectors are robust to video compression (see [Mathis/Warren](https://www.biorxiv.org/content/early/2018/10/30/457242) for details). **Mid Right:** It allows 3D pose estimation with a single network and camera (see [Mathis/Warren](https://www.biorxiv.org/content/early/2018/10/30/457242)). **Right:** It allows 3D pose estimation with a single network trained on data from multiple cameras together with standard triangulation methods (see [Nath* and Mathis* et al. 2019](https://doi.org/10.1038/s41596-019-0176-0)).
**DeepLabCut** is embedding in a larger open-source eco-system, providing behavioral tracking for neuroscience, ecology, medical, and technical applications. Moreover, many new tools are being actively developed. See [DLC-Utils](https://github.com/DeepLabCut/DLCutils) for some helper code.
## Code contributors:
DLC code was originally developed by [Alexander Mathis](https://github.com/AlexEMG) & [Mackenzie Mathis](https://github.com/MMathisLab), and was extended in 2.0 with [Tanmay Nath](http://www.mousemotorlab.org/team), and currently actively developed with [Jessy Lauer](https://github.com/jeylau). The feature detector code is based on Eldar Insafutdinov's TensorFlow implementation of [DeeperCut](https://github.com/eldar/pose-tensorflow). DeepLabCut is an open-source tool and has benefited from suggestions and edits by many individuals including Mert Yuksekgonul, Tom Biasi, Richard Warren, Ronny Eichler, Hao Wu, Federico Claudi, Gary Kane and Jonny Saunders as well as the [contributors](https://github.com/AlexEMG/DeepLabCut/graphs/contributors). Please see [AUTHORS](https://github.com/AlexEMG/DeepLabCut/blob/master/AUTHORS) for more details!
This is an actively developed package and we welcome community development and involvement.
## Community Support, Developers, & Help:
- We are a community partner on the [](https://forum.image.sc/tag/deeplabcut). Please post help and support questions on the forum with the tag DeepLabCut. Check out their mission statement [Scientific Community Image Forum: A discussion forum for scientific image software](https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3000340).
- If you encounter a previously unreported bug/code issue, please post here (we encourage you to search issues first): https://github.com/DeepLabCut/DeepLabCut/issues
- For quick discussions amongst users, please see here: [](https://gitter.im/DeepLabCut/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
- If you want to contribute to the code, please read our guide [here!](CONTRIBUTING.md)
- The project [road map](docs/roadmap.md). Get in touch with us if you want to help!
## References:
If you use this code or data we kindly as that you please [cite Mathis et al, 2018](https://www.nature.com/articles/s41593-018-0209-y) and, if you use the Python package (DeepLabCut2.x) please also cite [Nath, Mathis et al, 2019](https://doi.org/10.1038/s41596-019-0176-0). If you utilize the MobileNetV2s or EfficientNets please cite [Mathis, Biasi et al. 2020](https://arxiv.org/abs/1909.11229).
DOIs (#ProTip, for helping you find citations for software, check out [CiteAs.org](http://citeas.org/)!):
- Mathis et al 2018: [10.1038/s41593-018-0209-y](https://doi.org/10.1038/s41593-018-0209-y)
- Nath, Mathis et al 2019: [10.1038/s41596-019-0176-0](https://doi.org/10.1038/s41596-019-0176-0)
Please check out the following references for more details:
@article{Mathisetal2018,
title={DeepLabCut: markerless pose estimation of user-defined body parts with deep learning},
author = {Alexander Mathis and Pranav Mamidanna and Kevin M. Cury and Taiga Abe and Venkatesh N. Murthy and Mackenzie W. Mathis and Matthias Bethge},
journal={Nature Neuroscience},
year={2018},
url={https://www.nature.com/articles/s41593-018-0209-y}}
@article{NathMathisetal2019,
title={Using DeepLabCut for 3D markerless pose estimation across species and behaviors},
author = {Nath*, Tanmay and Mathis*, Alexander and Chen, An Chi and Patel, Amir and Bethge, Matthias and Mathis, Mackenzie W},
journal={Nature Protocols},
year={2019},
url={https://doi.org/10.1038/s41596-019-0176-0}}
@article{insafutdinov2016eccv,
title = {DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model},
author = {Eldar Insafutdinov and Leonid Pishchulin and Bjoern Andres and Mykhaylo Andriluka and Bernt Schiele},
booktitle = {ECCV'16},
url = {http://arxiv.org/abs/1605.03170}}
@article{Mathis2020DeepLT,
title={Deep learning tools for the measurement of animal behavior in neuroscience},
author={Mackenzie W. Mathis and Alexander Mathis},
journal={Current Opinion in Neurobiology},
year={2020},
volume={60},
pages={1-11}}
@article{Mathis2020Primer,
title={A Primer on Motion Capture with Deep Learning: Principles, Pitfalls, and Perspectives},
author={Alexander Mathis and Steffen Schneider and Jessy Lauer and Mackenzie W. Mathis},
journal={Neuron},
year={2020},
volume={108},
pages={44-65}}
Our open-access pre-prints:
@misc{Mathis2019_pretraining,
title={Pretraining boosts out-of-domain robustness for pose estimation},
author={Alexander Mathis and Mert Y\"uksekg\"on\"ul and Byron Rogers and Matthias Bethge and Mackenzie W. Mathis},
year={2019},
eprint={1909.11229},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@article{NathMathis2018,
author = {Nath*, Tanmay and Mathis*, Alexander and Chen, An Chi and Patel, Amir and Bethge, Matthias and Mathis, Mackenzie W},
title = {Using DeepLabCut for 3D markerless pose estimation across species and behaviors},
year = {2018},
doi = {10.1101/476531},
publisher = {Cold Spring Harbor Laboratory},
URL = {https://www.biorxiv.org/content/early/2018/11/24/476531},
eprint = {https://www.biorxiv.org/content/early/2018/11/24/476531.full.pdf},
journal = {bioRxiv}
}
@article{mathis2018markerless,
title={Markerless tracking of user-defined features with deep learning},
author={Mathis, Alexander and Mamidanna, Pranav and Abe, Taiga and Cury, Kevin M and Murthy, Venkatesh N and Mathis, Mackenzie W and Bethge, Matthias},
journal={arXiv preprint arXiv:1804.03142},
year={2018}
}
@article{MathisWarren2018speed,
author = {Mathis, Alexander and Warren, Richard A.},
title = {On the inference speed and video-compression robustness of DeepLabCut},
year = {2018},
doi = {10.1101/457242},
publisher = {Cold Spring Harbor Laboratory},
URL = {https://www.biorxiv.org/content/early/2018/10/30/457242},
eprint = {https://www.biorxiv.org/content/early/2018/10/30/457242.full.pdf},
journal = {bioRxiv}
}
## License:
This project is licensed under the GNU Lesser General Public License v3.0. Note that the software is provided "as is", without warranty of any kind, express or implied. If you use the code or data, please cite us!. Note, artwork and images are copyrighted; please do not take or use these images without written permission.
## Versions:
VERSION 2.2: Multi-animal pose estimation and tracking with DeepLabCut.
VERSION 2.0-2.1: This is the **Python package** of [DeepLabCut](https://www.nature.com/articles/s41593-018-0209-y) that was originally released with our [Nature Protocols](https://doi.org/10.1038/s41596-019-0176-0) paper (preprint [here](https://www.biorxiv.org/content/10.1101/476531v1)).
This package includes graphical user interfaces to label your data, and take you from data set creation to automatic behavioral analysis. It also introduces an active learning framework to efficiently use DeepLabCut on large experimental projects, and data augmentation tools that improve network performance, especially in challenging cases (see [panel b](https://camo.githubusercontent.com/77c92f6b89d44ca758d815bdd7e801247437060b/68747470733a2f2f737461746963312e73717561726573706163652e636f6d2f7374617469632f3537663664353163396637343536366635356563663237312f742f3563336663316336373538643436393530636537656563372f313534373638323338333539352f636865657461682e706e673f666f726d61743d37353077)).
VERSION 1.0: The initial, Nature Neuroscience version of [DeepLabCut](https://www.nature.com/articles/s41593-018-0209-y) can be found in the history of git, or here: https://github.com/AlexEMG/DeepLabCut/releases/tag/1.11
## News (and in the news):
- Jan 2021: [Pretraining boosts out-of-domain robustness for pose estimation](https://openaccess.thecvf.com/content/WACV2021/html/Mathis_Pretraining_Boosts_Out-of-Domain_Robustness_for_Pose_Estimation_WACV_2021_paper.html) published in the IEEE Winter Conference on Applications of Computer Vision. We also added EfficientNet backbones to DeepLabCut, those are best trained with cosine decay (see paper). To use them, just pass "efficientnet-b0" to "efficientnet-b6" when creating the trainingset!
- Dec 2020: We released a real-time package that allows for online pose estimation and real-time feedback. See [DLClive.deeplabcut.org](http://DLClive.deeplabcut.org).
- 5/22 2020: We released 2.2beta5. This beta release has some of the features of DeepLabCut 2.2, whose major goal is to integrate multi-animal pose estimation to DeepLabCut.
- Mar 2020: Inspired by suggestions we heard at this weeks CZI's Essential Open Source Software meeting in Berkeley, CA we updated our [docs](docs/UseOverviewGuide.md). Let us know what you think!
- Feb 2020: Our [review on animal pose estimation is published!](https://www.sciencedirect.com/science/article/pii/S0959438819301151)
- Nov 2019: DeepLabCut was recognized by the Chan Zuckerberg Initiative (CZI) with funding to support this project. Read more in the [Harvard Gazette](https://news.harvard.edu/gazette/story/newsplus/harvard-researchers-awarded-czi-open-source-award/), on [CZI's Essential Open Source Software for Science site](https://chanzuckerberg.com/eoss/proposals/) and in their [Medium post](https://medium.com/@cziscience/how-open-source-software-contributors-are-accelerating-biomedicine-1a5f50f6846a)
- Oct 2019: DLC 2.1 released with lots of updates. In particular, a Project Manager GUI, MobileNetsV2, and augmentation packages (Imgaug and Tensorpack). For detailed updates see [releases](https://github.com/AlexEMG/DeepLabCut/releases)
- Sept 2019: We published two preprints. One showing that [ImageNet pretraining contributes to robustness](https://arxiv.org/abs/1909.11229) and a [review on animal pose estimation](https://arxiv.org/abs/1909.13868). Check them out!
- Jun 2019: DLC 2.0.7 released with lots of updates. For updates see [releases](https://github.com/AlexEMG/DeepLabCut/releases)
- Feb 2019: DeepLabCut joined [twitter](https://twitter.com/deeplabcut) [](https://twitter.com/DeepLabCut)
- Jan 2019: We hosted workshops for DLC in Warsaw, Munich and Cambridge. The materials are available [here](https://github.com/AlexEMG/DeepLabCut-Workshop-Materials)
- Jan 2019: We joined the Image Source Forum for user help: [](https://forum.image.sc/tag/deeplabcut)
- Nov 2018: We posted a detailed guide for DeepLabCut 2.0 on [BioRxiv](https://www.biorxiv.org/content/early/2018/11/24/476531). It also contains a case study for 3D pose estimation in cheetahs.
- Nov 2018: Various (post-hoc) analysis scripts contributed by users (and us) will be gathered at [DLCutils](https://github.com/AlexEMG/DLCutils). Feel free to contribute! In particular, there is a script guiding you through
importing a project into the new data format for DLC 2.0
- Oct 2018: new pre-print on the speed video-compression and robustness of DeepLabCut on [BioRxiv](https://www.biorxiv.org/content/early/2018/10/30/457242)
- Sept 2018: Nature Lab Animal covers DeepLabCut: [Behavior tracking cuts deep](https://www.nature.com/articles/s41684-018-0164-y)
- Kunlin Wei & Konrad Kording write a very nice News & Views on our paper: [Behavioral Tracking Gets Real](https://www.nature.com/articles/s41593-018-0215-0)
- August 2018: Our [preprint](https://arxiv.org/abs/1804.03142) appeared in [Nature Neuroscience](https://www.nature.com/articles/s41593-018-0209-y)
- August 2018: NVIDIA AI Developer News: [AI Enables Markerless Animal Tracking](https://news.developer.nvidia.com/ai-enables-markerless-animal-tracking/)
- July 2018: Ed Yong covered DeepLabCut and interviewed several users for the [Atlantic](https://www.theatlantic.com/science/archive/2018/07/deeplabcut-tracking-animal-movements/564338).
- April 2018: first DeepLabCut preprint on [arXiv.org](https://arxiv.org/abs/1804.03142)