{"id":15031713,"url":"https://github.com/pkmital/cadl","last_synced_at":"2025-05-15T17:09:11.125Z","repository":{"id":48126506,"uuid":"62756493","full_name":"pkmital/CADL","owner":"pkmital","description":"ARCHIVED: Contains historical course materials/Homework materials for the FREE MOOC course on \"Creative Applications of Deep Learning w/ Tensorflow\" #CADL","archived":false,"fork":false,"pushed_at":"2019-05-06T07:18:18.000Z","size":330445,"stargazers_count":1482,"open_issues_count":23,"forks_count":732,"subscribers_count":116,"default_branch":"master","last_synced_at":"2025-03-31T21:49:46.187Z","etag":null,"topics":["deep-learning","dockerfile","jupyter-notebook","machine-learning","mooc","neural-network","tensorflow","tutorial","workshop"],"latest_commit_sha":null,"homepage":"https://www.kadenze.com/courses/creative-applications-of-deep-learning-with-tensorflow/info","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pkmital.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-07-06T22:07:55.000Z","updated_at":"2025-03-22T02:24:05.000Z","dependencies_parsed_at":"2022-08-12T19:10:32.388Z","dependency_job_id":null,"html_url":"https://github.com/pkmital/CADL","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pkmital%2FCADL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pkmital%2FCADL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pkmital%2FCADL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pkmital%2FCADL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pkmital","download_url":"https://codeload.github.com/pkmital/CADL/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247737788,"owners_count":20987721,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","dockerfile","jupyter-notebook","machine-learning","mooc","neural-network","tensorflow","tutorial","workshop"],"created_at":"2024-09-24T20:16:23.474Z","updated_at":"2025-04-07T22:10:33.151Z","avatar_url":"https://github.com/pkmital.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build Status](https://travis-ci.org/pkmital/CADL.svg?branch=master)](https://travis-ci.org/pkmital/CADL) [![Slack Channel](https://cadl.herokuapp.com/badge.svg)](https://cadl.herokuapp.com)\n\n# \u003ca href=\"https://www.kadenze.com/courses/creative-applications-of-deep-learning-with-tensorflow/info\"\u003eCreative Applications of Deep Learning w/ Tensorflow\u003c/a\u003e\n\nThis repository contains lecture transcripts and homework assignments as Jupyter Notebooks for the first of three \u003ca href=\"https://www.kadenze.com/partners/kadenze-academy\"\u003eKadenze Academy\u003c/a\u003e courses on \u003ca href=\"https://www.kadenze.com/courses/creative-applications-of-deep-learning-with-tensorflow/info\"\u003eCreative Applications of Deep Learning w/ Tensorflow\u003c/a\u003e.  It also contains a python package containing all the code developed during all three courses.\n\n**COURSE 1: Creative Applications of Deep Learning with TensorFlow I**  \nSession 1: Introduction to TensorFlow  \nSession 2: Training A Network W/ TensorFlow  \nSession 3: Unsupervised And Supervised Learning  \nSession 4: Visualizing And Hallucinating Representations  \nSession 5: Generative Models  \n\n**COURSE 2: Creative Applications of Deep Learning with TensorFlow II**  \nSession 1: Cloud Computing, GPUs, Deploying  \nSession 2: Mixture Density Networks  \nSession 3: Modeling Attention with RNNs, DRAW  \nSession 4: Image-to-Image Translation with GANs  \n\n**COURSE 3: Creative Applications of Deep Learning with TensorFlow III**  \nSession 1: Modeling Music and Art: Google Brain’s Magenta Lab  \nSession 2: Modeling Language: Natural Language Processing  \nSession 3: Autoregressive Image Modeling w/ PixelCNN  \nSession 4: Modeling Audio w/ Wavenet and NSynth  \n\n# Github Contents Overview\n\n| | Session | Description | Transcript | Homework |\n| --- | --- | --- | --- | --- |\n| Python Package | **[pycadl](https://github.com/pkmital/pycadl)** | Python package required for courses 2 and 3 | N/A | N/A |\n|Installation| **[Installation](#installation-preliminaries)** | Setting up Python/Notebook and necessary libraries. | N/A | N/A |\n|Preliminaries| **[Preliminaries with Python](session-0)** | Basics of working with Python and images. | N/A | N/A |\n|1| **[Computing with Tensorflow](session-1)** | Working with a small dataset of images.  Dataset preprocessing.  Tensorflow basics.  Sorting/organizing a dataset. | [lecture-1.ipynb](session-1/lecture-1.ipynb) [Colab](https://colab.research.google.com/notebook#fileId=1bjXwBG0AqGcFbSP5pjjL1A981mBhhN0f) | [session-1.ipynb](session-1/session-1.ipynb) |\n|2| **[Basics of Neural Networks](session-2)** | Learn how to create a Neural Network.  Learn to use a neural network to paint an image.  Apply creative thinking to the inputs, outputs, and definition of a network. | [lecture-2.ipynb](session-2/lecture-2.ipynb) | [session-2.ipynb](session-2/session-2.ipynb) |\n|3| **[Unsupervised and Supervised Learning](session-3)** | Build an autoencoder.  Extend it with convolution, denoising, and variational layers.  Build a deep classification network.  Apply softmax and onehot encodings to classify audio using a Deep Convolutional Network. | [lecture-3.ipynb](session-3/lecture-3.ipynb) | [session-3.ipynb](session-3/session-3.ipynb) |\n|4| **[Visualizing Representations](session-4)** | Visualize backpropped gradients, use them to create Deep Dream, extend Deep Dream w/ regularization.  Stylize images or synthesize new images with painterly or hallucinated aesthetics of another image. | [lecture-4.ipynb](session-4/lecture-4.ipynb) | [session-4.ipynb](session-4/session-4.ipynb) |\n|5| **[Generative Models](session-5)** | Build a Generative Adversarial Network and extend it with a Variational Autoencoder.  Use the latent space of this network to perform latent arithmetic.  Build a character level Recurrent Neural Network using LSTMs.  Understand different ways of inferring with Recurrent Networks.  | [lecture-5.ipynb](session-5/lecture-5.ipynb) | [session-5-part-1.ipynb](session-5/session-5-part-1.ipynb), [session-5-part-2.ipynb](session-5/session-5-part-2.ipynb) |\n\n\u003ca name=\"installation-preliminaries\"\u003e\u003c/a\u003e\n# Installation Preliminaries\n\n\u003c!-- MarkdownTOC autolink=true autoanchor=true bracket=round --\u003e\n\n- [Quickstart Guide](#quickstart-guide)\n    - [Method 1: pip Install](#method-1-pip-install)\n    - [Method 2: Docker Installation](#method-2-docker-installation)\n- [What is Notebook?](#what-is-notebook)\n- [Docker Toolbox](#docker-toolbox)\n- [Jupyter Notebook](#jupyter-notebook)\n    - [OSX/Linux](#osxlinux)\n    - [Windows/Docker Containers](#windowsdocker-containers)\n- [Navigating to Notebook](#navigating-to-notebook)\n- [Installing Python Packages](#installing-python-packages)\n    - [Ubuntu/Linux 64-bit for Python 3.4](#ubuntulinux-64-bit-for-python-34)\n    - [Ubuntu/Linux 64-bit for Python 3.5](#ubuntulinux-64-bit-for-python-35)\n    - [OSX for Python 3.4 or Python 3.5](#osx-for-python-34-or-python-35)\n    - [Other Linux/OSX varieties](#other-linuxosx-varieties)\n- [CUDA/GPU instructions](#cudagpu-instructions)\n- [Testing it](#testing-it)\n- [CUDA/GPU instructions for MacOS](#cudagpu-instructions-for-macos)\n- [Troubleshooting](#troubleshooting)\n    - [ImportError: No module named 'tensorflow'](#importerror-no-module-named-tensorflow)\n    - [AttributeError: module 'tensorflow' has no attribute '\\_\\_version\\_\\_'](#attributeerror-module-tensorflow-has-no-attribute-%5C%5Cversion%5C%5C)\n    - [GPU-related issues](#gpu-related-issues)\n    - [Protobuf library related issues](#protobuf-library-related-issues)\n    - [Cannot import name 'descriptor'](#cannot-import-name-descriptor)\n    - [Can't find setup.py](#cant-find-setuppy)\n    - [SSLError: SSL_VERIFY_FAILED](#sslerror-sslverifyfailed)\n    - [Jupyter Notebook Kernel is always busy \\(Windows\\)](#jupyter-notebook-kernel-is-always-busy-windows)\n    - [Something Else!](#something-else)\n\n\u003c!-- /MarkdownTOC --\u003e\n\nThe first course makes heavy usage of Jupyter Notebook.  This will be necessary for submitting the homeworks and interacting with the guided session notebooks I will provide for each assignment.  Follow along this guide and we'll see how to obtain all of the necessary libraries that we'll be using.  By the end of this, you'll have installed Jupyter Notebook, NumPy, SciPy, and Matplotlib.  While many of these libraries aren't necessary for performing the Deep Learning which we'll get to in later lectures, they are incredibly useful for manipulating data on your computer, preparing data for learning, and exploring results.\n\n\u003ca name=\"quickstart-guide\"\u003e\u003c/a\u003e\n## Quickstart Guide\n\n**Please skip this section and read the rest of this readme if you are unfamiliar w/ Jupyter Notebook or installing Python libraries.  This section is only for advanced users who want to get started quickly.**\n\nThere are two ways to get started.  You can use a native pip installation or use Docker.  There is a quickstart guide for both methods below.  If you have trouble with these, then please skip to the more in depth guides below these sections.\n\n\u003ca name=\"method-1-pip-install\"\u003e\u003c/a\u003e\n### Method 1: pip Install\n\nFor those of you that are proficient w/ Python programming, you'll need Python 3.4+ and the latest TensorFlow which you can install via pip, e.g.:\n\n```bash\n$ pip install tensorflow\n```\n\nor w/ CUDA as:\n\n```bash\n$ pip install tensorflow-gpu\n```\n\n\u003ca name=\"method-2-docker-installation\"\u003e\u003c/a\u003e\n### Method 2: Docker Installation\n\nIf you want a controlled environment w/ all dependencies installed for you, and are proficient w/ Docker and Jupyter, you can get started w/ this repo like so:\n\n```bash\n$ cd\n$ git clone --recursive https://github.com/pkmital/CADL.git\n$ cd CADL\n$ docker build -t cadl .\n$ docker run -it -p 8888:8888 -p 6006:6006 -v /$(pwd)/session-1:/notebooks --name tf cadl /bin/bash\n```\n\nNote that you can skip the build step and download from docker hub instead like so:\n\n```bash\n$ docker run -it -p 8888:8888 -p 6006:6006 -v /$(pwd)/session-1:/notebooks --name tf pkmital/cadl /bin/bash\n```\n\nBe sure to replace \"session-1\" with whichever session you are working on, e.g. \"session-2\", \"session-3\"...  This will give you a bash prompt with the files for each session:\n\n```bash\nroot@39c4441bcde8:/notebooks# ls\nREADME.md  lecture-1.ipynb  libs  session-1.ipynb  tests\n```\n\nWhich you can use to launch jupyter like so:\n\n```bash\nroot@39c4441bcde8:/notebooks# jupyter notebook --allow-root\n[I 01:45:27.712 NotebookApp] [nb_conda_kernels] enabled, 2 kernels found\n[I 01:45:27.715 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret\n[W 01:45:27.729 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.\n[I 01:45:27.799 NotebookApp] [nb_anacondacloud] enabled\n[I 01:45:27.802 NotebookApp] [nb_conda] enabled\n[I 01:45:27.856 NotebookApp] ✓ nbpresent HTML export ENABLED\n[W 01:45:27.856 NotebookApp] ✗ nbpresent PDF export DISABLED: No module named 'nbbrowserpdf'\n[I 01:45:27.858 NotebookApp] Serving notebooks from local directory: /notebooks\n[I 01:45:27.858 NotebookApp] 0 active kernels\n[I 01:45:27.858 NotebookApp] The Jupyter Notebook is running at: http://[all ip addresses on your system]:8888/?token=dd68eeffd8f227dd789327c981d16b24631866e909bd6469\n[I 01:45:27.858 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).\n```\n\nJupyter should then be running if you navigate Google Chrome (suggested!) to \"http://localhost:8888\".  If you navigate to the session-1.ipynb file, you will see the homework, or to \"lecture-1.ipynb\", to find the lecture transcripts.  The same goes for every other session.\n\nIf you need to relaunch the docker image again, you can write:\n\n```bash\n$ cd\n$ cd CADL\n$ docker start -i tf\n```\n\nIf you want to use a GPU version, and have a Linux machine, and have an NVIDIA GPU, you can use [nvidia-docker](https://github.com/NVIDIA/nvidia-docker) (this only works for Linux machines! for non-Linux machines that want to use GPU, please follow the expanded directions below, or the quickstart pip installation above):\n\n```bash\n$ wget -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.0-rc.3/nvidia-docker_1.0.0.rc.3-1_amd64.deb\n$ sudo dpkg -i /tmp/nvidia-docker*.deb \u0026\u0026 rm /tmp/nvidia-docker*.deb\n$ nvidia-docker build -t cadl-gpu -f Dockerfile-gpu .\n$ nvidia-docker run -it -p 8888:8888 -p 6006:6006 -v /$(pwd)/session-1:/notebooks --name tf cadl-gpu /bin/bash \n$ nvidia-docker start -i tf\n```\n\nIf you had any trouble w/ this setup then please go through the rest of this document which provides much more in depth details.\n\n\n\u003ca name=\"what-is-notebook\"\u003e\u003c/a\u003e\n## What is Notebook?\n\nJupyter Notebook, previously called \"iPython Notebook\" prior to version 4.0, is a way of interacting with Python code using a web browser.  It is a very useful instructional tool that we will be using for all of our homework assignments.  Notebooks have the file extensions \"ipynb\" which are abbreviations of \"iPython Notebook\".  Some websites such as [nbviewer.ipython.org](http://nbviewer.ipython.org) or [www.github.com](http://www.github.com) can view `.ipynb` files directly as rendered HTML.  However, these are not *interactive* versions of the notebook, meaning, they are not running the python kernel which evaluates/interacts with the code.  So the notebook is just a static version of the code contained inside of it.\n\nIn order to interact with notebook and start coding, you will need to launch Terminal (for Mac and Linux users).  For Windows users, or for anyone having any problems with the Linux/Mac instructions, please follow the next section on [Docker Toolbox](#docker-toolbox) very closely!  If you are not a Windows user, please first try skipping over the next section and use the installation instructions in [Jupyter Notebook](#jupyter-notebook) before trying Docker as this solution will be much faster than running Docker.\n\n\u003ca name=\"docker-toolbox\"\u003e\u003c/a\u003e\n## Docker Toolbox\n\nCurrently, Windows users can only install Tensorflow via [pip using a 64-bit Python 3.5 environment](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/get_started/os_setup.md#pip-installation-on-windows) or using Docker, as outlined below.\n\nThe easiest way to get up an running on any type of system is to use Docker.  Docker is a way of managing a \"virtual\" Linux machine on your computer which will aid the creation a machine capable of running Tensorflow.  First, please download and install the Docker Toolbox:\n\nhttps://www.docker.com/products/docker-toolbox\n\nLinux users can install docker using their favorite package manager.\n\nFor OSX and Windows users, you'll then need to run the \"Docker Quickstart Terminal\" which will launch a Terminal environment running on a virtual Linux machine on your computer. A virtual machine is basically an emulation of another machine. This is important because we'll use this machine to run Linux and install all of the necessary libraries for running Tensorflow.\n\nNote for Windows users, if you have trouble launching the Docker Quickstart Terminal because you have \"Hyper-V\", please instead try using https://docs.docker.com/docker-for-windows/.  Then launch the newly installed \"Docker CLI\" program.\n\nOnce the Terminal is launched, either via Docker CLI or Docker Quickstart Terminal, run the following command (ignoring the `$` sign at the beginning of each line, which just denote that each line is a terminal command that you should type out exactly and then hit ENTER afterwards):\n\n```shell\n$ cd\n$ docker-machine ip\n```\n\nIf you are using Docker Toolbox, you should see your virtual machine's IP address as a result of the last command.  This is the location of your virtual machine.  \u003cb\u003eNOTE THIS IP ADDRESS\u003c/b\u003e, as we'll need it in a second.  If you are using \"Docker for Windows\" instead, then you won't need this IP as we'll just use \"localhost\".\n\nThis next command will move to your \"home\" directory.  We'll then \"clone\" the github repo.  This will download everything for the course using \"git\".  If you have trouble w/ this step, make sure you have installed [git](https://git-scm.com/downloads).\n\n```shell\n$ cd\n$ git clone --recursive https://github.com/pkmital/CADL.git\n```\n\nWe'll now print out what the full path to that directory is.  PLEASE NOTE DOWN THIS DIRECTORY.  This is where everything will happen, and I'll explain that in a minute.\n\n```shell\n$ echo /$(pwd)/CADL\n```\n\nNow run the following command, which will download everything we need to run tensorflow, python, and jupyter notebook (again, ignore the \"$\" at the beginning of the line only)!\n\n```shell\n$ docker run -it -p 8888:8888 -p 6006:6006 -v /$(pwd)/CADL:/notebooks --name tf pkmital/cadl\n```\n\nWhat this is doing is:\n    * Running the docker image [pkmital/cadl](https://hub.docker.com/r/pkmital/cadl/)\n    * --name is giving it a shorthand name of \"tf\"\n    * -v is mirroring the directory \"/$(pwd)/CADL\" to the virtual machine's directory of \"/notebooks\"\n    * -p is forwarding ports from the virtual machine to your local machine so that you can access the virtual machine's port\n    * -it is running it as an interactive process\n\nYou will want to put files inside the \"/notebooks\" directory *only*.  If you place files on the virtual machine outside of the \"/notebooks\" directory, which is the SAME as the \"CADL\" directory on your local machine, they will *not* be saved.  We are using Docker to mirror the \"CADL\" directory on a virtual machine which has everything necessary for us to code in Python and Tensorflow.  _Whatever is in that directory will be mirrored on the virtual machine's directory under `/notebooks`._\n\nYou can also try running the docker run command with any other directory. For instance:\n\n```shell\n$ docker run -it -p 8888:8888 -p 6006:6006 -v /Users/YOURUSERNAME/Desktop:/notebooks --name tf pkmital/cadl\n```\n\nWhich would mean that your Desktop is where you can move files around so that on the virtual machine, you can interact with them under the `/notebooks`directory.\n\nFor OSX users, if you are installing Docker because you had installation problems using Anaconda and pip, you would instead write the following command (note the missing slash):\n\n```shell\n$ docker run -it -p 8888:8888 -p 6006:6006 -v $(pwd)/CADL:/notebooks --name tf pkmital/cadl\n```\n\nWhen you want to start this machine, you will launch the Docker Quickstart Terminal and then write:\n\n```shell\n$ cd\n$ docker start -i tf\n```\n\nNotice that the command prompt will now be `#` instead of `$`.  You should have a new folder \"tensorflow\" inside your Home directory.  This directory will be empty to begin with.  Please make sure you do everything inside this directory only or else any files you make on your virtual machine WILL BE ERASED once it is shutdown!  When you clone the CADL repository, or expand the zip file downloads contents inside this directory via your Windows machine (it will be in your Home directory under a folder \"cadl\"), then you will be able to access it via your Docker instance.\n\nFor instance, after running the `docker start -i tf` command, try going into the directory `/notebooks`:\n\n```shell\n# cd /notebooks\n```\n\n\u003ca name=\"jupyter-notebook\"\u003e\u003c/a\u003e\n## Jupyter Notebook\n\n\u003ca name=\"osxlinux\"\u003e\u003c/a\u003e\n### OSX/Linux\n\nNote: Windows/Docker users should scroll past this section to [\"Windows/Docker\"](#windows-docker-containers).  For OSX/Linux users, the easiest way to ensure you have Python 3.4 or higher and Jupter Notebook is to install Anaconda for Python 3.5 located here:\n\n[OSX](https://docs.continuum.io/anaconda/install#anaconda-for-os-x-command-line-install) or [Linux](https://docs.continuum.io/anaconda/install#linux-install)\n\nMake sure you restart your Terminal after you install Anaconda as there are some PATH variables that have to be set.\n\nThen run the following:\n\n```shell\n$ curl https://bootstrap.pypa.io/ez_setup.py -o - | python\n```\n\nIf you already have conda, but only have Python 2, you can very easily [add a new environment w/ Python 3](http://conda.pydata.org/docs/py2or3.html#create-a-python-3-5-environment) and switch back and forth as needed.  Or if you do not have Anaconda, but have a system based install, I'd really recommend either using Anaconda or [pyenv](https://github.com/yyuu/pyenv) to help you manage both python installations.\n\nWith Anaconda installed, you will have python and the package \"ipython[notebook]\", along with a ton of other very useful packages such as numpy, matplotlib, scikit-learn, scikit-image, and many others.\n\nWith everything installed, restart your Terminal application (on OSX, you can use Spotlight to find the Terminal application), and then navigate to the directory containing the \"ipynb\", or \"iPython Notebook\" file, by \"cd'ing\" (pronounced, see-dee-ing), into that directory.  This involves typing the command: \"cd some_directory\".  Once inside the directory of the notebook file, you will then type: \"jupyter notebook\".  If this command does not work, it means you do not have notebook installed!  Try installed anaconda as above, restart your Terminal application, or manually install notebook like so (ignore the \"$\" signs which just denote that this is a Terminal command that you should type out exactly and then hit ENTER!):\n\n```shell\n$ pip3 install ipython[notebook]\n$ jupyter notebook\n```\n\nIf you run into issues that say something such as:\n\n```\n[W 20:37:40.543 NotebookApp] Kernel not found: None\n```\n\nThen please try first running:\n\n```shell\n$ ipython3 kernel install\n```\n\n\u003ca name=\"windows-docker-containers\"\u003e\n\u003ca name=\"windowsdocker-containers\"\u003e\u003c/a\u003e\n### Windows/Docker Containers\n\nFor users running firewalls, you must make sure you have an exception as per [Jupyter Notebooks Firewall Instructions](http://jupyter-notebook.readthedocs.io/en/latest/public_server.html#firewall-setup) otherwise you may not be able to interact with the notebook.  Namely, you will need to allow connections from 127.0.0.1 (localhost) on ports from 49152 to 65535.  Once inside your Docker container as outlined above, you can now launch notebook like so:\n\n```shell\n$ cd /notebooks\n$ jupyter notebook \u0026\n```\n\nNote on Virtual versus Windows Directories:\n\nThis is tricky to grasp, mostly because I didn't explain it. Docker is \"virtual\" computer running inside your computer. It has its own filesystem and its own directories. So you can't reference your Windows machine's directories inside this machine. When you first ran docker (e.g. `$ docker run -it -p 8888:8888 -p 6006:6006 -v /$(pwd)/tensorflow:/notebooks --name tf pkmital/cadl`) it included as part of its command: `-v /$(pwd)/tensorflow:/notebooks`. What that was doing is \"mirroring\" a directory on your Windows machine inside your Virtual machine. So whatever was in your Windows machine under the directory `/$(pwd)/tensorflow` would appear in the Virtual machine under `/notebooks`. That Windows directory is likely `/Users/\u003cYOURUSERNAME\u003e/tensorflow`. So _ONLY_ inside that directory, create it if it doesn't exist, should you put files in order to access it on the Virtual machine.\n\nSo let's say your Username was \"pkmital\". Then your home directory would be `/Users/pkmital`, and you would have mirrored `/Users/pkmital/tensorflow` on your Windows Machine to the Virtual machine under `/notebook`. Now let's say I create a directory `/Users/pkmital/tensorflow/images` on my Windows Machine, and then put a bunch of png files in there. I will then see them in my Virtual machine under `/notebook/images`.  If I put the CADL repository inside `/Users/pkmital/tensorflow`, then I should have `/Users/pkmital/tensorflow/CADL/session-1/session-1.ipynb` and on the Virtual machine, it will be in `/notebooks/CADL/session-1/session-1.ipynb` - From this notebook, running on the virtual machine, accessed with Jupyter Notebook, I would access my images like so:\n\n```python\nimport os\nos.listdir('../../images')\n```\n\n\u003ca name=\"navigating-to-notebook\"\u003e\u003c/a\u003e\n## Navigating to Notebook\n\nAfter running \"jupyter notebook \u0026\", you should see a message similar to:\n\n```shell\nroot@182bd64f27d2:~# jupyter notebook \u0026\n[I 21:15:33.647 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret\n[W 21:15:33.712 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.\n[W 21:15:33.713 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using authentication. This is highly insecure and not recommended.\n[I 21:15:33.720 NotebookApp] Serving notebooks from local directory: /root\n[I 21:15:33.721 NotebookApp] 0 active kernels\n[I 21:15:33.721 NotebookApp] The IPython Notebook is running at: http://[all ip addresses on your system]:8888/\n[I 21:15:33.721 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).\n```\n\nDon't worry if the IP address or command prompt look different.  Note where it says: `The IPython Notebook is running at`.  If you are running Docker (Windows users), this is where we need that IP address.  For OSX/Linux users, we'll use \"localhost\" so don't worry about this.  Now open up Chrome/Safari/Firefox whatever browser you like, and then navigate to:\n\nhttp://localhost:8888\n\nor for Windows users:\n\nhttp://ADDRESS:8888\n\nwhere ADDRESS is the ip address you should have noted down before. For instance, on my machine, I would visit the website:\n\nhttp://192.168.99.100:8888\n\nThis will launch the Jupyter Notebook where you will be able to interact with the homework assignments!\n\n\u003ca name=\"installing-python-packages\"\u003e\u003c/a\u003e\n## Installing Python Packages\n\nPackages are libraries or useful extensions to the standard python libraries.  In this course, we'll be using a few including Tensorflow, NumPy, MatPlotLib, SciPy, SciKit-Image, and SciKit-Learn.  Windows users will already have these libraries since the Docker container includes these.  However, if you needed to, you can install these using \"pip\", which is the python package manager.  OSX/Linux users should follow these steps just to be sure they have the latest versions of these packages. In Python 3.4 and higher, `pip` comes with any standard python installation.  In order to use `pip`, first make sure you are using the correct version.  One way to do this is check which pip you are running:\n\n```shell\n$ which pip\n$ which pip3\n```\n\nUse which `pip` points to the install path that makes the most sense (e.g. Anaconda for OSX users for some reason does not symlink pip3 to the python3 pip, and instead points to the system version of python3).\n\nThen you'll write:\n\n```shell\n$ pip3 install -U pip setuptools\n```\n\nTo make sure you have an up to date pip, then:\n\n```shell\n$ pip3 install some_package\n```\n\nTo get the necessary libraries:\n\n```shell\n$ pip3 install \"scikit-image\u003e=0.11.3\" \"numpy\u003e=1.11.0\" \"matplotlib\u003e=1.5.1\" \"scikit-learn\u003e=0.17\"\n```\n\nThis should get you all of the libraries we need for the course, EXCEPT for tensorflow.  Tensorflow is a special case, but can be `pip` installed in much the same way by pointing pip to the github repo corresponding to your OS like so.\n\n\u003ca name=\"ubuntulinux-64-bit-for-python-34\"\u003e\u003c/a\u003e\n### Ubuntu/Linux 64-bit for Python 3.4\n\n```shell\n$ pip3 install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.11.0rc1-cp34-cp34m-linux_x86_64.whl\n```\n\n\u003ca name=\"ubuntulinux-64-bit-for-python-35\"\u003e\u003c/a\u003e\n### Ubuntu/Linux 64-bit for Python 3.5\n\n```shell\n$ pip3 install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.11.0rc1-cp35-cp35m-linux_x86_64.whl\n```\n\n\u003ca name=\"osx-for-python-34-or-python-35\"\u003e\u003c/a\u003e\n### OSX for Python 3.4 or Python 3.5\n\n```shell\n$ pip3 install --upgrade https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-0.11.0rc1-py3-none-any.whl\n```\n\n\u003ca name=\"other-linuxosx-varieties\"\u003e\u003c/a\u003e\n### Other Linux/OSX varieties\n\nYou can pip install Tensorflow for most OSX/Linux setups including those that are making use of NVIDIA GPUs and CUDA using one the packages listed on this link:\nhttps://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/get_started/os_setup.md#pip-installation\n\nIf you are having trouble with pip installation, try looking here first: [Common Installation Problems](https://github.com/tensorflow/tensorflow/blob/37451589519d15207448dc2d9b1c0309de15d8db/tensorflow/g3doc/get_started/os_setup.md#common-problems).  Failing that, reach out to us on the forums, or else you may want to instead run a Docker instance as outlined in the Windows instructions above: [Setting up a Docker Container](#docker-toolbox).\n\n\u003ca name=\"cudagpu-instructions\"\u003e\u003c/a\u003e\n## CUDA/GPU instructions\n\nNote that I have not provided instructions on getting setup w/ CUDA as it is beyond the scope of this course!  If you are interested in using GPU acceleration, I highly recommend using Ubuntu Linux and setting up a machine on [Nimbix](https://www.nimbix.net) or [Amazon EC2](https://aws.amazon.com/ec2/\n) using the instructions here: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/get_started/os_setup.md#optional-install-cuda-gpus-on-linux.  If you're using Nimbix, you can skip the install process as there is already a machine pre-installed w/ Tensorflow.  Similarly, for Amazon EC2, there are many existing \"images\" of machines that have Tensorflow already installed.\n\n\n\u003ca name=\"testing-it\"\u003e\u003c/a\u003e\n## Testing it\n\nTo confirm it worked, try running:\n\n```shell\n$ python3 -c 'import tensorflow as tf; print(tf.__version__)'\n```\n\nYou should see 1.0.0 printed, depending on which version you have installed.\n\n\n\u003ca name=\"cudagpu-instructions-for-macos\"\u003e\u003c/a\u003e\n## CUDA/GPU instructions for MacOS\n\nWhen your Mac is equipped with a NVidia graphics card, you can use the GPU for computing with Tensorflow. GPU enabled computing is not supported for Macs with ATI or Intel graphics cards. \n\nIf you have a previous cpu installation of tensorflow, uninstall it first:\n\n```\n$ pip3 uninstall tensorflow\n```\n\nUsing homebrew, install the following packages:\n\n```\n$ brew install coreutils\n$ brew tap caskroom/cask\n$ brew cask install cuda\n```\nOnce you have the CUDA Toolkit installed you will need to setup the required environment variables by adding the \nfollowing to your `~/.profile`:\n```\nexport CUDA_HOME=/usr/local/cuda\nexport DYLD_LIBRARY_PATH=\"$DYLD_LIBRARY_PATH:$CUDA_HOME/lib\"\nexport PATH=\"$CUDA_HOME/bin:$PATH\"\n```\nTensorflow needs the library libcuda.1.dylib, so we have to create an additional symbolic link:\n```\nsudo ln -sf /usr/local/cuda/lib/libcuda.dylib /usr/local/cuda/lib/libcuda.1.dylib\n```\nFinally, you will also want to install the **CUDA Deep Neural Network** (cuDNN v5) library which currently requires an \n[_Accelerated Computing Developer Program_](https://developer.nvidia.com/cudnn) account. Once you have it downloaded \nlocally, you can unzip and move the header and libraries to your local CUDA Toolkit folder:\n```\n$ sudo mv include/cudnn.h /Developer/NVIDIA/CUDA-8.0/include/\n$ sudo mv lib/libcudnn* /Developer/NVIDIA/CUDA-8.0/lib\n$ sudo ln -s /Developer/NVIDIA/CUDA-8.0/lib/libcudnn* /usr/local/cuda/lib/\n```\nThen, finally, install tensorflow with GPU support with:\n```\n$ pip3 install --ignore-installed --upgrade tensorflow-gpu\n```\n\nAccording to the instructions of the TensorFlow website, this should work. However, on MacOS 10.11 (El Capitan) and \nabove, the environment variable `DYLD_LIBRARY_PATH` is ignored, resulting in an error in the interactive python console \nand JetBrains PyCharm IDE. The dynamic library `libcudart.8.0.dylib` fails to load. This\nis due to a new protection meganism in MacOS 10.11 and higher. El Capitan ships with a new OS X feature: System \nIntegrity Protection (SIP), also known as “rootless” mode. This reduces the attack surface for malware that relies on \nmodifying system files by preventing any user, whether with system administrator (“root”) privileges or not from \nmodifying a number of operating system directories and files.\n\n**Warning:** The point of SIP is to prevent malware and other unwanted modifications into system files. Consider whether \nor not you want to dispense with this protection.\nFollow these steps to disable SIP:\n\n* Restart your Mac.\n* Before OS X starts up, hold down Command-R and keep it held down until you see an Apple icon and a progress bar. Release. This boots you into Recovery.\n* From the Utilities menu, select Terminal.\n* At the prompt type exactly the following and then press Return: `csrutil disable`\n* Terminal should display a message that SIP was disabled.\n* From the  menu, select Restart.\n\nYou can re-enable SIP by following the above steps, but using `csrutil enable` instead.\n\n\n\n\u003ca name=\"troubleshooting\"\u003e\u003c/a\u003e\n## Troubleshooting\n\n\u003ca name=\"importerror-no-module-named-tensorflow\"\u003e\u003c/a\u003e\n### ImportError: No module named 'tensorflow'\n\nYou may have different versions of Python installed.  You can troubleshoot this by looking at the output of:\n\n```shell\n$ which python3\n$ which pip3\n$ python3 --version\n$ pip3 --version\n$ which python\n$ which pip\n$ python --version\n$ pip --version\n```\n\nYou may simply need to install tensorflow using `pip` instead of `pip3` and/or use `python` instead of `python3`, assuming they point to a version of python which is Python 3 or higher.\n\n\u003ca name=\"attributeerror-module-tensorflow-has-no-attribute-%5C%5Cversion%5C%5C\"\u003e\u003c/a\u003e\n### AttributeError: module 'tensorflow' has no attribute '\\_\\_version\\_\\_'\n\nYou could be running python inside a directory that contains the folder \"tensorflow\".  Try running python inside a different directory.\n\n\n\u003ca name=\"gpu-related-issues\"\u003e\u003c/a\u003e\n### GPU-related issues\n\nIf you encounter the following when trying to run a TensorFlow program:\n\n```python\nImportError: libcudart.so.7.0: cannot open shared object file: No such file or directory\n```\n\nMake sure you followed the GPU installation [instructions](#optional-install-cuda-gpus-on-linux).\nIf you built from source, and you left the Cuda or cuDNN version empty, try specifying them\nexplicitly.\n\n\u003ca name=\"protobuf-library-related-issues\"\u003e\u003c/a\u003e\n### Protobuf library related issues\n\nTensorFlow pip package depends on protobuf pip package version\n3.0.0b2. Protobuf's pip package downloaded from [PyPI](https://pypi.python.org)\n(when running `pip install protobuf`) is a Python only library, that has\nPython implementations of proto serialization/deserialization which can be 10x-50x\nslower than the C++ implementation. Protobuf also supports a binary extension\nfor the Python package that contains fast C++ based proto parsing. This\nextension is not available in the standard Python only PIP package. We have\ncreated a custom binary pip package for protobuf that contains the binary\nextension. Follow these instructions to install the custom binary protobuf pip\npackage :\n\n```bash\n# Ubuntu/Linux 64-bit:\n$ pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/protobuf-3.0.0b2.post2-cp27-none-linux_x86_64.whl\n\n# Mac OS X:\n$ pip install --upgrade https://storage.googleapis.com/tensorflow/mac/protobuf-3.0.0b2.post2-cp27-none-any.whl\n```\n\nand for Python 3 :\n\n```bash\n# Ubuntu/Linux 64-bit:\n$ pip3 install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/protobuf-3.0.0b2.post2-cp34-none-linux_x86_64.whl\n\n# Mac OS X:\n$ pip3 install --upgrade https://storage.googleapis.com/tensorflow/mac/protobuf-3.0.0b2.post2-cp35-none-any.whl\n```\n\nInstall the above package _after_ you have installed TensorFlow via pip, as the\nstandard `pip install tensorflow` would install the python only pip package. The\nabove pip package will over-write the existing protobuf package.\nNote that the binary pip package already has support for protobuf larger than\n64MB, that should fix errors such as these :\n\n```bash\n[libprotobuf ERROR google/protobuf/src/google/protobuf/io/coded_stream.cc:207] A\nprotocol message was rejected because it was too big (more than 67108864 bytes).\nTo increase the limit (or to disable these warnings), see\nCodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.\n\n```\n\n\u003ca name=\"cannot-import-name-descriptor\"\u003e\u003c/a\u003e\n### Cannot import name 'descriptor'\n\n```python\nImportError: Traceback (most recent call last):\n  File \"/usr/local/lib/python3.4/dist-packages/tensorflow/core/framework/graph_pb2.py\", line 6, in \u003cmodule\u003e\n    from google.protobuf import descriptor as _descriptor\nImportError: cannot import name 'descriptor'\n```\n\nIf you the above error when upgrading to a newer version of TensorFlow, try\nuninstalling both TensorFlow and protobuf (if installed) and re-installing\nTensorFlow (which will also install the correct protobuf dependency).\n\n\u003ca name=\"cant-find-setuppy\"\u003e\u003c/a\u003e\n### Can't find setup.py\n\nIf, during `pip install`, you encounter an error like:\n\n```bash\n...\nIOError: [Errno 2] No such file or directory: '/tmp/pip-o6Tpui-build/setup.py'\n```\n\nSolution: upgrade your version of pip:\n\n```bash\npip install --upgrade pip\n```\n\nThis may require `sudo`, depending on how pip is installed.\n\n\u003ca name=\"sslerror-sslverifyfailed\"\u003e\u003c/a\u003e\n### SSLError: SSL_VERIFY_FAILED\n\nIf, during pip install from a URL, you encounter an error like:\n\n```bash\n...\nSSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed\n```\n\nSolution: Download the wheel manually via curl or wget, and pip install locally.\n\n\u003ca name=\"jupyter-notebook-kernel-is-always-busy-windows\"\u003e\u003c/a\u003e\n### Jupyter Notebook Kernel is always busy (Windows)\nIf your have installed Docker Toolbox on Windows but your jupyter notebook doesn't run properly (the notebook kernel keeps busy all the time when you open any file) then you might need to try different browsers (One guy tried Edge and it solved his problem after struggling for long time on Chrome/Firefox).\n\nAnd you should also enable port forwarding by:\n\n1. Open VirtualBox\n2. Click on your default docker image.\n3. Click Settings.\n4. Click Network.\n5. Click forward port.\n6. Add a new rule named jupyter with host ip=127.0.0.1 and host/guess port=8888\n7. Now you should be able to browse your notebook app via localhost:8888 (instead of having to browse 192.168.xx.xx:8888)\n\n\u003ca name=\"something-else\"\u003e\u003c/a\u003e\n### Something Else!\n\nPost on the [Forums](https://www.kadenze.com/courses/creative-applications-of-deep-learning-with-tensorflow-i/forums?sort=recent_activity) or check on the Tensorflow [README](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/get_started/os_setup.md#pip-installation)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpkmital%2Fcadl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpkmital%2Fcadl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpkmital%2Fcadl/lists"}