https://github.com/mdda/deep-learning-workshop
Deep Learning Workshop : Including a VirtualBox VM with pre-configured Jupyter, Tensorflow, PyTorch, models and data
https://github.com/mdda/deep-learning-workshop
Last synced: 6 months ago
JSON representation
Deep Learning Workshop : Including a VirtualBox VM with pre-configured Jupyter, Tensorflow, PyTorch, models and data
- Host: GitHub
- URL: https://github.com/mdda/deep-learning-workshop
- Owner: mdda
- License: mit
- Created: 2016-02-07T11:08:32.000Z (over 10 years ago)
- Default Branch: master
- Last Pushed: 2019-07-06T16:18:47.000Z (almost 7 years ago)
- Last Synced: 2025-07-19T21:55:27.363Z (11 months ago)
- Language: Jupyter Notebook
- Homepage:
- Size: 17.9 MB
- Stars: 474
- Watchers: 30
- Forks: 120
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## Deep Learning Workshop
This repo includes all scripts required to build a VirtualBox 'Appliance' (an easy-to-install pre-configured VM)
that can be used by Deep Learning Workshop participants.
This workshop consists of an introduction to deep learning (from single layer networks-in-the-browser,
then using the VM/Jupyter setup to train networks using both Theano (+Lasagne for model components) and Tensorflow (+some sugar layers).
The modules also include pretrained state-of-the-art networks, such as GoogLeNet, in various applications) :
* [FOSSASIA 2016](http://2016.fossasia.org/) : Deep Learning Workshop (2 hours)
* Application : Generative art (~style transfer)
* Application : Classifying unknown classes of images (~transfer learning)
* Slides for the talk are [here](http://redcatlabs.com/2016-03-19_FOSSASIA-Workshop/),
with an accompanying [blog post](http://blog.mdda.net/ai/2016/03/19/workshop-at-fossasia-2016)
* [PyCon-SG 2016](https://pycon.sg/schedule/presentation/94/) : Deep Learning Workshop (1.5 hours)
* Unfortunately, due to 'demand' for speaker slots, PyCon has only scheduled 1h30 for the workshop, rather than the 3h00 they originally suggested...
* Application : Reinforcement Learning
* Slides for the talk are [here](http://redcatlabs.com/2016-06-23_PyConSG-Workshop/),
with an accompanying [blog post](http://blog.mdda.net/ai/2016/06/23/workshop-at-pycon-sg-2016), which includes a video link
* [DataScienceSG MeetUp](http://www.meetup.com/DataScience-SG-Singapore/) : 'Hardcore' session about Deep Learning (2.5 hours)
* Application : Anomaly Detection (mis-shaped MNIST digits)
* Application : Classifying unknown classes of images (~transfer learning)
* Slides for the talk are [here](http://redcatlabs.com/2016-07-23_DataScienceSG-DeepLearning-Workshop/),
with an accompanying [blog post](http://blog.mdda.net/ai/2016/07/23/datascience-sg-workshop), which includes a video link
* [Fifth Elephant, India](https://fifthelephant.in/2016/) : Deep Learning Workshop (6 hours : 4x 1.5hr classes in one day)
* Application : Classifying unknown classes of images (~transfer learning)
* Application : Generative art (~style transfer)
* Application : RNN Tagger
* Application : RNN Fun (work-in-progress)
* Application : Anomaly Detection (mis-shaped MNIST digits)
* Application : Reinforcement Learning
* Slides for the talk are [here](http://redcatlabs.com/2016-07-30_FifthElephant-DeepLearning-Workshop/),
with an accompanying [blog post](http://blog.mdda.net/ai/2016/07/30/workshop-at-fifth-elephant-2016)
* [PyDataSG MeetUp](https://www.meetup.com/PyData-SG/events/235981761/) : Talk on RNNs and NLP (1.5 hours)
* Application : RNN Tagger (cleaned up a little)
* Slides for the talk are [here](http://redcatlabs.com/2017-01-23_PyDataSG_RNNs-for-NLP/),
with an accompanying [blog post](http://blog.mdda.net/ai/2017/01/23/presentation-at-pydata), which includes a video link
* [TensorFlow & Deep Learning MeetUp](https://www.meetup.com/TensorFlow-and-Deep-Learning-Singapore/events/237032130/) : Talk on transfer learning (0.5 hours)
* Application : Classifying unknown classes of images (~transfer learning) *in TensorFlow*
* Slides for the talk are [here](http://redcatlabs.com/2017-02-16_TFandDL_TransferLearning/#/),
with an accompanying [blog post](http://blog.mdda.net/ai/2017/02/16/presentation-at-tensorflow), which includes a video link
* [FOSSASIA 2017](http://2017.fossasia.org/) : Deep Learning Workshop (1 hour)
* Application : Speech Recognition using a CNN
* Slides for the talk are [here](http://redcatlabs.com/2017-03-18_FOSSASIA-Workshop/#/),
with an accompanying [blog post](http://blog.mdda.net/ai/2017/03/18/presentation-at-FOSSASIA), which includes a video link
* [TensorFlow & Deep Learning MeetUp](https://www.meetup.com/TensorFlow-and-Deep-Learning-Singapore/events/237860964/) : Talk on CNNs (0.5 hours)
* Application : Speech Recognition using a CNN (non-workshop version)
* Slides for the talk are [here]((http://redcatlabs.com/2017-03-20_TFandDL_IntroToCNNs/#/),
with an accompanying [blog post](http://blog.mdda.net/ai/2017/03/20/presentation-at-tensorflow), which includes a video link
* [TensorFlow & Deep Learning MeetUp](https://www.meetup.com/TensorFlow-and-Deep-Learning-Singapore/events/238584480/) : Generative Art : Style-Transfer (0.5 hours)
* Application : Generative Art (Style-Transfer)
* Slides for the talk are [here](http://redcatlabs.com/2017-04-13_TFandDL_ArtStyleTransfer/#/)
* [APAC Machine Learning & Data Science Community Summit](http://m.onoffmix.com/event/97444/content) : In the news : AlphaGo and Reinforcement Learning (0.75 hours)
* Application : Bubble-Breaker in TensorFlow / Keras
* Slides for the talk are [here](http://redcatlabs.com/2017-05-20_APAC-Summit/)
with an accompanying [blog post](http://blog.mdda.net/ai/2017/05/20/presentation-in-korea)
* [TensorFlow & Deep Learning MeetUp](https://www.meetup.com/TensorFlow-and-Deep-Learning-Singapore/events/239252636/) : Text : Embeddings, RNNs and NER (~1 hour)
* Application : BiDirectional RNNs for Case-Insensitive NER
* Slides for the talk (including a more general introduction to NLP) are [here](http://redcatlabs.com/2017-05-25_TFandDL_TextAndRNNs/#/)
with an accompanying [blog post](http://blog.mdda.net/ai/2017/05/25/presentation-at-tensorflow), which includes a video link
* [TensorFlow & Deep Learning MeetUp](https://www.meetup.com/TensorFlow-and-Deep-Learning-Singapore/events/240386386/) : Advanced Text and Language (0.75 hours)
* Application : Image Captioning (Flickr30k)
* Slides for the talk are [here](http://redcatlabs.com/2017-06-22_TFandDL_Captioning/#/)
with an accompanying [blog post](http://blog.mdda.net/ai/2017/06/22/presentation-at-tensorflow), which includes a video link
* [FOSSASIA 2018](http://2018.fossasia.org/) : Deep Learning Workshop (1 hour)
* Application : OpenAI's meta-learning "Reptile" sines demo
* Slides for the talk are [here](http://redcatlabs.com/2018-03-25_FOSSASIA-Workshop/#/),
with an accompanying [blog post](http://blog.mdda.net/ai/2018/03/25/presentation-at-FOSSASIA), which includes a video link
**NB : Ensure Conference Workshop announcement / blurb includes VirtualBox warning label**
* Also : for the Art (and potentially other image-focussed) modules, having a few 'personal' images available might be entertaining *
The VM itself includes :
* Jupyter (iPython's successor)
* Running as a server available to the host machine's browser
* Data
* MNIST training and test sets
* Trained models from two of the 'big' ImageNet winners
* Test Images for both recognition, 'e-commerce' and style-transfer modules
* Corpuses and pretrained GloVe for the language examples
* Locally-runnable versions of a CNN demonstrator, and OpenAI's '3-boxes' Reptile demo
* Tool chain(s) (Python-oriented)
* Theano / Lasagne
* Tensorflow and Keras
* PyTorch (CPU version)
And this repo can itself be run in 'local mode', using scripts in ```./local/``` to :
* Set up the virtual environment correctly
* Run ```jupyter``` with the right flags, paths etc
### Status : Workshop WORKS!
#### Currently working well
* Scripts to create working Fedora 25 installation inside VM
* Has working ```Python3.x``` ```virtualenv``` with ```Jupyter``` and ```TensorFlow / TensorBoard```
* Script to transform the VM into a VirtualBox appliance
* Exposing ```Jupyter```, ```TensorBoard``` and ```ssh``` to host machine
* Locally hosted Convnet.js for :
* Demonstration of gradient descent ('painting')
* Locally hosted TensorFlow Playground for :
* Visualising hidden layer, and effect of features, etc
* Locally hosted cnn demo for :
* Demonstration of how a single CNN 3x3 filter works
* Existing workshop notebooks :
* Basics
* MNIST
* MNIST CNN
* ImageNet : GoogLeNet
* ImageNet : Inception 3
* CNN for simple Voice Recognition
* 'Anomaly Detection' - identifying mis-shaped MNIST digits
* 'Commerce' - repurpose a trained network to classify our stuff
* 'Art' - Style transfer with Lasagne, but using GoogLeNet features for speed
* 'Reinforcement Learning' - learning to play "Bubble Breaker"
* 'RNN-Tagger' - Processing text, and learning to do case-less Named Entity Recognition
* Notebook Extras
* U - VM Upgrade tool
* X - BLAS configuration fiddle tool
* Z - GPU chooser (needs Python's ```BeautifulSoup```)
* Create rsync-able image containing :
* VirtualBox appliance image
+ including data sets and pre-trained models
* VirtualBox binaries for several likely platforms
* Write to thumb-drives for actual workshop
* and/or upload to DropBox
* Workshop presentation materials
#### Still Work-in-Progress
* Create sync-to-latest-workbooks script to update existing (taken-home) VMs
* Create additional 'applications' modules (see 'ideas.md')
* Monitor TensorBoard - to see whether it reduces its memory footprint enough to switch from Theano...
* 'RNN-Fun' - Discriminative and Generative RNNs
### Notes
#### Running the environment locally
See the [local/README file](./local/README.md).
Also worth investigating : [Google Colab](https://colab.research.google.com/), which
allows the Free (as in Beer) use of a K40 GPU in a Jupyter-notebook-like interface. In fact,
there is also the possibility of pulling up GitHub-based notebooks directly using the
url :
```
https://colab.research.google.com/github/USER/REPO/blob/master/NOTEBOOK
```
For a concrete example, look at this link to [the recent revamped Reptile code from OpenAI](https://colab.research.google.com/github/mdda/deep-learning-workshop/blob/master/notebooks/8-MetaLearning/2-Reptile-Sines.ipynb) that
is in the MetaLearning folder of this repo.
#### Git-friendly iPython Notebooks
Using the code from : http://pascalbugnion.net/blog/ipython-notebooks-and-git.html (and
https://gist.github.com/pbugnion/ea2797393033b54674af ),
you can enable this kind of feature just on one repository,
rather than installing it globally, as follows...
Within the repository, run :
```
# Set the permissions for execution :
chmod 754 ./bin/ipynb_optional_output_filter.py
git config filter.dropoutput_ipynb.smudge cat
git config filter.dropoutput_ipynb.clean ./bin/ipynb_optional_output_filter.py
```
this will add suitable entries to ``./.git/config``.
or, alternatively, create the entries manually by ensuring that your ``.git/config`` includes the lines :
```
[filter "dropoutput_ipynb"]
smudge = cat
clean = ./bin/ipynb_output_filter.py
```
Note also that this repo includes a ``/.gitattributes`` file containing the following:
```
*.ipynb filter=dropoutput_ipynb
```
Doing this causes git to run ``ipynb_optional_output_filter.py`` in the ``REPO/bin`` directory,
which only uses ``import json`` to parse the notebook files (and so can be executed as a plain script).
To disable the output-cleansing feature in a notebook (to disable the cleansing on a per-notebook basis),
simply add to its metadata (Edit-Metadata) as a first-level entry (``true`` is the default):
```
"git" : { "suppress_outputs" : false },
```
#### Git-friendly iPython Notebooks (Looks promising, but...)
[nbstripout](https://github.com/kynan/nbstripout) seems to do what we want, and can be installed more easily.
Within the local python environment (or do this globally, as root, if you're committed) :
```
pip install nbstripout
````
### Useful resources
* [MathJax](http://nbviewer.ipython.org/github/olgabot/ipython/blob/master/examples/Notebook/Typesetting%20Math%20Using%20MathJax.ipynb)
* [Bokeh](http://bokeh.pydata.org/en/latest/docs/quickstart.html)