https://github.com/opencog/destin
Deep SpatioTemporal Inference Network (DeSTIN) for vision processing
https://github.com/opencog/destin
Last synced: 4 months ago
JSON representation
Deep SpatioTemporal Inference Network (DeSTIN) for vision processing
- Host: GitHub
- URL: https://github.com/opencog/destin
- Owner: opencog
- License: lgpl-3.0
- Created: 2013-09-12T00:05:02.000Z (almost 13 years ago)
- Default Branch: master
- Last Pushed: 2016-09-10T16:29:20.000Z (almost 10 years ago)
- Last Synced: 2023-10-20T22:12:02.825Z (over 2 years ago)
- Language: C++
- Homepage: http://wiki.opencog.org/w/DeSTIN
- Size: 10 MB
- Stars: 34
- Watchers: 24
- Forks: 28
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
DeSTIN
=============
Papers about DeSTIN
-------------------
* http://web.eecs.utk.edu/~itamar/Papers/BICA2009.pdf
* http://www.ece.utk.edu/~itamar/Papers/BICA2011T.pdf
* http://web.eecs.utk.edu/~itamar/Papers/AI_MAG_2011.pdf
* http://research.microsoft.com/en-us/um/people/dongyu/nips2009/papers/Arel-DeSTIN_NIPS%20tpk2.pdf
* http://goertzel.org/Goertzel_AAAI11.pdf
* http://goertzel.org/DeSTIN_OpenCog_paper_v1.pdf
* http://goertzel.org/papers/DeSTIN_OpenCog_paper_v2.pdf
* http://www.springerlink.com/content/264p486742666751/fulltext.pdf
* http://goertzel.org/VisualAttention_AGI_11.pdf
* http://goertzel.org/Uniform_DeSTIN_paper.pdf
* http://goertzel.org/papers/Uniform_DeSTIN_paper_v2.pdf
* http://goertzel.org/papers/CogPrime_Overview_Paper.pdf
Current Work
------------
Implementing uniform destin described here:
http://wiki.opencog.org/w/DestinOpenCog
Creating enhanced visualization capabilities.
For more detailed development progress see Diary.md
Building
--------
Instructions below are for Ubuntu.
See Windows build instructions at https://github.com/opencog/destin/blob/master/WindowsBuild.md
You may build docker image containing all dependencies apart from CUDA SDK by using https://github.com/opencog/destin/blob/master/Dockerfile
Dependencies:
* CMake (2.8.x, >= 3.2)
* OpenCV 2 (libopencv-dev)
Java Bindings: JDK, ant and SWIG 2.x
Python Bindings: python-dev (tested with python 2.7), python-opencv, and SWIG 2.x
To build DrentheDestin, CUDA SDK is required.
Individual parts can be skipped by commenting out their ADD_SUBDIRECTORY line in the CMakeLists.txt
Some examples:
* If you dont have CUDA for example, comment out the "ADD_SUBDIRECTORY(DrentheDestin)" in Destin/CMakeLists.txt
* If you dont want to install any language bindings comment out "ADD_SUBDIRECTORY(Bindings)" in Destin/CMakeLists.txt.
* If you want python bindings but not Java, then in Destin/Bindings/CMakeLists.txt comment out "add_subdirectory(Java)"
Building:
$ git clone http://github.com/opencog/destin.git
$ cd destin
$ git submodule init
$ git submodule update
$ cd Destin
$ cmake .
$ make
Rebuilding:
During development sometimes CMake is not able to detect dependencies between the different built libraries. So it is recommened
to do a "make clean" and "make -j4" (j4 lets you use multiple cores) from the Destin directory to make sure everyting is building fresh.
Building Java Bindings:
The native jni bindings lib is built with CMake, the actual java program which uses it is built with java_build.sh:
$ cd Destin/Bindings/Java
$ ./java_build.sh
$ ./java_run.sh
Show me something!
------------------
Currently the most interesting things to see are the scripts in the Destin/Bindings/Python directory.
These scripts are meant to be used interactivly so I recommend installing and using idle for python ( search for and install the idle package).
To see a self organizing map (SOM):
Manually download http://www.cs.toronto.edu/~kriz/cifar-10-binary.tar.gz
$ tar xf cifar-10-binary.tar.gz
$ cd cifar-10-batches-bin
$ pwd
Note the absolute path
$ cd Destin/Bindings/Python
Edit som.py and edit the cifar_dir variable near the top and change it to the absolute path of the cifar-10-batches-bin directory
$ idle som.py
Two windows appear, select the som.py window and click Run->Run Module (F5).
Training will begin, after less than 5 minutes the SOM will popup.
To see DeSTIN train on an video and see its output:
$ cd Destin/Bindings/Python
$ idle dostuff.py
Two windows should appear.
- select the dostuff.py window
- click Run -> Run Modedule ( F5 )
You should see a video of a hand and windows for 8 layers.
In the other "*Python Shell*" window you can interact with the code after its done with 500 frames ( about a minute).
To make it continue processing type go(100) to have it process 100 more frames. You can make it train on webcam input instead by
changing the line:
vs = pd.VideoSource(False, "hand.m4v")
to
vs = pd.VideoSource(True, "")