{"id":13652164,"url":"https://github.com/ANNetGPGPU/ANNetGPGPU","last_synced_at":"2025-04-23T03:30:36.501Z","repository":{"id":8892555,"uuid":"10612723","full_name":"ANNetGPGPU/ANNetGPGPU","owner":"ANNetGPGPU","description":"A GPU (CUDA) based Artificial Neural Network library","archived":false,"fork":false,"pushed_at":"2021-09-25T09:51:32.000Z","size":2089,"stargazers_count":107,"open_issues_count":5,"forks_count":20,"subscribers_count":11,"default_branch":"master","last_synced_at":"2024-11-10T03:33:06.149Z","etag":null,"topics":["c-plus-plus-11","cuda","propagation-network","self-organizing-map"],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ANNetGPGPU.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"license.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-06-11T01:16:22.000Z","updated_at":"2024-10-29T22:55:50.000Z","dependencies_parsed_at":"2022-08-25T08:02:45.191Z","dependency_job_id":null,"html_url":"https://github.com/ANNetGPGPU/ANNetGPGPU","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ANNetGPGPU%2FANNetGPGPU","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ANNetGPGPU%2FANNetGPGPU/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ANNetGPGPU%2FANNetGPGPU/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ANNetGPGPU%2FANNetGPGPU/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ANNetGPGPU","download_url":"https://codeload.github.com/ANNetGPGPU/ANNetGPGPU/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250365212,"owners_count":21418648,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c-plus-plus-11","cuda","propagation-network","self-organizing-map"],"created_at":"2024-08-02T02:00:56.293Z","updated_at":"2025-04-23T03:30:34.894Z","avatar_url":"https://github.com/ANNetGPGPU.png","language":"C++","funding_links":[],"categories":["Artificial Intelligence","Uncategorized"],"sub_categories":["Uncategorized"],"readme":"# Updates - 05/10/2017: Added a new example\n\nThe program \"image_generator\" is located in the \"/src/examples\" subdirectory and was submitted by Ben Bogart. It produces abstract looking images from ordinary input images. \nTo compile it, make sure to install OpenCV. If you want to give it a try, it takes two command line arguments and can be used like this:\n\n```\n./image_generator \u003cinput\u003e \u003coutput\u003e\n```\n\nHere is an example input image with the related output image.\n![artw_i](https://cloud.githubusercontent.com/assets/4668178/25889737/32f24db8-356b-11e7-96d7-ae1915225c76.png)\n__CUDA image generator example.__ The picture illustrates a some houses in vienna.\n![artw_i](https://cloud.githubusercontent.com/assets/4668178/25889736/32dfa848-356b-11e7-93cd-90ff58a3eba9.png)\n__CUDA image generator example.__ This image was calculated on a GTX 1080. It took apprx. 20 min and 500 MB VRAM.\n\n# Updates - 09/03/2016\n\n- Improvements of the implementation of the base classes\n- Added a Qt4 demo, illustrating how to implement a GUI on the example of a back-propagation network \n  - Note that the networks can be highly asymmetrical\n\n![artw_i](https://cloud.githubusercontent.com/assets/4668178/18225607/b8ddff64-71f6-11e6-872f-78c3b0450717.png)\n__GUI-example: Designer for back propagation networks.__ The layout of the underlying library is 1:1 represented as a QSceneGraph. After definition of the network topology, the in- and output can be defined by the user and the network trained accordingly. At the end, the error of each test-training cycle is plotted, which gives a handy representation of the network performance.\n\n\n# Updates - 07/31/2016\n\n- The project was rewritten into a template library\n- The CUDA device function pointers are poorly implemented, thus the function extensions work with template parameters now\n- As most of the code gets created just at compile time, the compile time increased in comparison to the old version\n- All classes but SOMNetGPU are now header only\n- A bug with SOMs on GPU was fixed, which led to wrong results of the radius decay function\n\nHere is an example of the new device pointer replacement. The distance function of the network is now a template argument, which will simplify library functionality extensions.\n\n```\nint main(int argc, char *argv[]) {\n\t// ..\n\tANNGPGPU::SOMNetGPU\u003cfloat, ANN::functor_gaussian\u003cfloat\u003e\u003e gpu;\n```\n\n# Updates - 04/12/2016\n\n- The support for CUDA \u003e version 4 was broken. Now it works again (tested with CUDA 7.5)\n\n# Projects\n## Artwork from Ben Bogart\n\n[![Image decomposition](https://cloud.githubusercontent.com/assets/4668178/18225709/9fe765e2-71f9-11e6-8846-1c47de4ad4f2.png)](https://player.vimeo.com/video/181111922?autoplay=1\u0026loop=1\u0026title=0\u0026byline=0\u0026portrait=0)\n__This video is from [ekran.org](http://www.ekran.org/ben/wp/2016/09/) and shows the decomposition of a high resolution panorama by a SOM on the GPU__\n\n![artw_i](http://www.ekran.org/ben/wp/wp-content/uploads/2016/07/good_result_24mm-final-detail-11.jpg)\n__Linewise growing neighborhood from [ekran.org](http://www.ekran.org/ben/wp/2016/03/)__\n\n# Introduction\n\nANNet is a small library to create neural nets. A hallmark of the project are implementations of several neural network models with usage of OpenMP and/or Thrust. \nSee quickstart guide to learn how to use them. \nEspecially self organizing maps (SOMs) benefit strongly from calculations on GPUs and speed-ups by a factor of 100 can easily be achieved for bigger networks (\u003e256x256 nodes). \nThe GPU implementation of SOMs is also supporting asynchronous calculation on more than one device. \n\n# Features\n- Implementation:\n  - Self organizing maps using CUDA\n  - Back propagation networks\n- Python interface for all classes\n- Multi core support using OpenMP\n- Plugin system based on template parameters\n- With the exception of the CUDA implementation, this project is a header only library\n\n# Build\n\nTo build the library with all features you need:\n\n- Qt5 (for some examples required)\n- SWIG for python bindings (just required for python bindings)\n- CUDA/Thrust (shipped with CUDA; just required for GPGPU implementation)\n- Doxygen (required for documentation generation)\n- OpenMP (required if multi CPU support is wished)\n- Lib bzip2 (required)\n- CMake (required if you want to use the CMake scripts)\n- A C++ compiler (GCC or MinGW; required)\n- How you build the library:\n\nClone the repository with git:\n\n```\ngit clone https://github.com/ANNetGPGPU/ANNetGPGPU.git\n```\n\nCreate a build directory, where your compiler stores the objects:\n\n```\ncd ANNetGPGPU\nmkdir build\ncd build\n```\n\nRun CMake and make to build. Dependent on the installed libraries, either all or just some example programs will be built:\n\n```\ncmake .. \u0026\u0026 make\n```\n\nor \n\n```\ncmake -DCMAKE_CXX_FLAGS=\"-O2\" -DCUDA_NVCC_FLAGS=\"-O2\" ..\nmake -j8\n```\n\n# Usage (Python interface)\n\nThere is a python interface for the library, which may have access to the GPU too. \nThe python demo here shows a k-means like clustering approach with the SOM implementation. \nIf the number of nodes is reduced you can calculate the \"centroids\" of given input vectors.\nIn this example a three-dimensional input was chosen for simplicity.\nNote: Currently, the template library could work with double precision. \nHowever, the python interface is currently implemented for float.\n\n```\nfrom ANPyNetCPU import *\nblack \t= vectorf([0,0,0])\nwhite \t= vectorf([1,1,1])\nred \t= vectorf([1,0,0])\ngreen \t= vectorf([0,1,0])\nblue \t= vectorf([0,0,1])\n\ntrainSet = TrainingSetF()\ntrainSet.AddInput(black)\ntrainSet.AddInput(white)\ntrainSet.AddInput(red)\ntrainSet.AddInput(green)\ntrainSet.AddInput(blue)\n\nwidthMap = 5\nheightMap = 1\n\ninpWidth = 3\ninpHeight = 1\n\nSOM = SOMNetGaussF(inpWidth,inpHeight,widthMap,heightMap)\nSOM.SetTrainingSet(trainSet)\nSOM.SetLearningRate(0.75)\nSOM.Training(100)\n\n# gets an ordered list of different centroids with the ID of the corresponding BMU\ncentroids = SOM.GetCentroidList()\n\n# output for fun\nfor i in centroids:\n\tprint (i)\n```\n  \nThe output of the demo would be:\n\n```\nWhite\n\u003e 1\n\u003e 1\n\u003e 1\n\nRed\n\u003e 1\n\u003e 7.2e-23\n\u003e 7.2e-23\n\nBlack\n\u003e 9.6e-18\n\u003e 1.4e-45\n\u003e 3.0e-18\n\nBlue\n\u003e 2.9e-22\n\u003e 1.4e-45\n\u003e 1\n```\n\n# Usage (C++)\n\nI prepared working examples for many typical use cases. These examples can be found in the \"/src/examples\" folder.\nIn the following I write a bit about these examples, to help you understand the layout of the library.\n\n\n## 1. Back propagation networks\n\nDefine the input, e.g. numbers from 0 to 9 could be declared.\n\n```\nfloat fInp1[56] = { 0,0,0,0,1,0,0,\n\t\t    0,0,0,1,1,0,0,\n\t\t    0,0,1,0,1,0,0,\n\t\t    0,1,0,0,1,0,0,\n\t\t    0,0,0,0,1,0,0,\n\t\t    0,0,0,0,1,0,0,\n\t\t    0,0,0,0,1,0,0,\n\t\t    0,0,0,0,1,0,0 };\n\t\t\t\t\t\nfloat fInp2[56] = { 0,0,0,1,1,0,0,\n\t\t    0,0,1,0,0,1,0,\n\t\t    0,1,0,0,0,1,0,\n\t\t    0,1,0,0,1,0,0,\n\t\t    0,0,0,1,0,0,0,\n\t\t    0,0,0,1,0,0,0,\n\t\t    0,0,1,0,0,0,0,\n\t\t    0,1,1,1,1,1,0 };\n\t\t\t\t\t\n// ..\n```\n\nAddLayer connects all neurons of both layers with each other.\nIt is also possible to create networks by defining each connection. \nThis can be done by a vector describing the graph.\nInternally the networks works as a linked list (cpu). \nFor the gpu implementation, a vector is created based on the connection graph.\n\n```\n#include \u003cANNet\u003e\n#include \u003cANContainers\u003e\n#include \u003cANMath\u003e\n\n#include \u003cSamples.h\u003e\n\n#include \u003cctime\u003e\n#include \u003ciostream\u003e\n\n\nint main(int argc, char *argv[]) {\n\tANN::BPNet\u003cfloat, ANN::fcn_log\u003cfloat\u003e\u003e cpu_one;\n        \n\tANN::BPLayer\u003cfloat, ANN::fcn_log\u003cfloat\u003e\u003e *layer1 = cpu_one.AddLayer(56, ANN::ANLayerInput);\n\tANN::BPLayer\u003cfloat, ANN::fcn_log\u003cfloat\u003e\u003e *layer2 = cpu_one.AddLayer(64, ANN::ANLayerHidden);\n\tANN::BPLayer\u003cfloat, ANN::fcn_log\u003cfloat\u003e\u003e *layer3 = cpu_one.AddLayer(9, ANN::ANLayerOutput);\n\n\tlayer1-\u003eConnectLayer(layer2);\n\tlayer2-\u003eConnectLayer(layer3);\n\t\n\tANN::TrainingSet\u003cfloat\u003e input;\n\tinput.AddInput(fInp1, 56);\n\tinput.AddOutput(fOut1, 9);\n\t// .. more input/output samples\n\t\n\tstd::vector\u003cfloat\u003e errors;\n\t\n\tANN::HebbianConf\u003cfloat\u003e conf = {0.5, 0, 0};\n\tcpu_one.Setup(conf);\n\tcpu_one.SetTrainingSet(input);\n\n\tbool b = false;\n\tfloat f;\n\terrors = cpu_one.TrainFromData(50, 0, b, f);\n\tstd::cout\u003c\u003c \u0026cpu_one \u003c\u003cstd::endl;\n\n\tcpu_one.ExpToFS(\"foo.bar\");\n\tANN::BPNet\u003cfloat, ANN::fcn_log\u003cfloat\u003e\u003e cpu_two;\n\tcpu_two.ImpFromFS(\"foo.bar\");\n\tcpu_two.SetTrainingSet(input);\n\t\n\tstd::cout\u003c\u003c \u0026cpu_two \u003c\u003cstd::endl;\n\treturn 0;\n}\n\n```\n\nHebbianConf is a small struct storing the learning rates and related constants of the network.\nA zero will automatically disable the related procedure like the momentum term, or weight decay during training.\n\n```\ntemplate \u003cclass T\u003e\nstruct HebbianConf {\n\tT learning_rate;\n\tT momentum_rate;\n\tT weight_decay;\n};\n```\n\n## 2. Self organizing maps (SOMs)\n\n### 1. CPU implementation\n\nSelf-organizing maps (SOM) are a type of network that is trained using unsupervised learning to produce a low-dimensional (typically two-dimensional), \nrepresentation of training samples. \nSOMs are different from other networks in the sense that they use a neighborhood function for learning. \nThe example shows a SOM 128x128 node network, each node can process a 3x1 input vector.\n\n```\nANN::SOMNet\u003cfloat, ANN::functor_gaussian\u003cfloat\u003e\u003e cpu;\ncpu.CreateSOM(3, 1, 128,128);\ncpu.SetTrainingSet(input);\ncpu.Training(100, ANN::ANSerialMode);\n```\n\n### 2. GPGPU implementation\n\nIt is possible to speed up the calculation of SOMs by processing them on GPU. \nOne can train, save and load the net with CPU as well and later continue with GPU and the other way round, \nsimply by calling the CTOR:\n\n```\nANN::SOMNet\u003cfloat, ANN::functor_gaussian\u003cfloat\u003e\u003e cpu;\ncpu.CreateSOM(3, 1, 128, 128); // input w, h, net w, h\nANN::SOMNetGPU\u003cfloat, ANN::functor_gaussian\u003cfloat\u003e\u003e gpu(\u0026cpu);      // use copy CTOR or create it like above\n\n// do stuff\n// ..\n\ncpu.ExpToFS(foo.bar);\ngpu.ImpFromFS(foo.bar);\n```\n\n### 3. GPGPU \u003c-\u003e CPU comparison \n![Figure 1](https://cloud.githubusercontent.com/assets/4668178/8379809/abfa53c2-1c22-11e5-8574-ebe873044d6d.png)\n\n__Figure 1: Training progress on quad core CPU (A) and GTX 570 (B) with three dimensional vectors as input.__\nBoth maps have the same size and the computation time was fixed. \nIn the same time the GPU implementation was able to process more iterations, \nthus the classification of the input vectors resulted in a lower error.\n\n# General class extensions\n\nYou certainly know that in C++ it is not possible to add methods to an existing class. So how that's possible? \nThe trick is to include in the declaration of a class a file defined by the preprocessor token e.g. \"\\_\\_ConTable_ADDON\".\nYou can extent any class and most files like this. The token always have a \"\\_\\_\" prefix followed by the class name and the \"\\_ADDON\" suffix.\nClasses/Structs have such injection points always in the \"public\" space. \n\n```\ntemplate \u003cclass T\u003e\nstruct ConTable {\n\t// ..\n\tstd::vector\u003cT\u003e m_vMisc;\n\t\n#ifdef __ConTable_ADDON\n\t#include __ConTable_ADDON\n#endif\n};\n```\n\nFor this we just need to create a new header file, e.g. \"foo.h\" with the example content:\n\n```\nstd::vector\u003cT\u003e m_vYourVector; // vector for your addon\nvoid your_new_function() { }  // your custom function ..\n```\n\nThen we just need to pass the path of this file to the build system.\nIn CMake, this would look like this.\n\n```\nadd_definitions(-D__ConTable_ADDON=\"${SOM_GPU_ADDON_SOURCE_DIR}/foo.h\")\n```\n\nHere, \"SOM_GPU_ADDON_SOURCE_DIR\" is the path of the file \"foo.h\"\n\n# Adding custom learning/distance functions\n\n## Adding new functions for the CPU based library\n\nAs long as you work with the CPU implementation it is super simple to define your own functions and pass them to the related functor. \n\n\n```\n/*\n * Here we declare just the basic learning functions \n */\ntemplate \u003cclass T\u003e\ninline T custom_learn(T fWeight, T fInfluence, T fInput) {\n\treturn fWeight + (fInfluence*(fInput-fWeight) );\n}\n\ntemplate \u003cclass T\u003e\ninline T custom_gaussian_nhood (T dist, T sigmaT) {\n\treturn exp(-pow(dist, 2.f)/(2.f*pow(sigmaT, 2.f)));\n}\n\ntemplate \u003cclass T\u003e\ninline T custom_rad_decay (T sigma0, T t, T lambda) {\n\treturn std::floor(sigma0*exp(-t/lambda) + 0.5f);\n}\n\ntemplate \u003cclass T\u003e\ninline T custom_lrate_decay (T sigma0, T t, T lambda) {\n\treturn sigma0*exp(-t/lambda);\n}\n\n/*\n * Here we define the functor for the network \n */\ntemplate\u003cclass T\u003e using custom_functor = ANN::DistFunction\u003cT, custom_learn\u003cT\u003e, custom_gaussian_nhood\u003cT\u003e, custom_rad_decay\u003cT\u003e, custom_lrate_decay\u003cT\u003e \u003e;\n\nint main(int argc, char *argv[]) {\n\t// ..\n\tANN::SOMNetCPU\u003cfloat, custom_functor\u003cfloat\u003e \u003e cpu;\n\t// ..\n```\n\n## Adding new functions for the GPU based library\n\nAdding new functions/functors for the GPU implementation is not as simple. The NVCC requires instantiation of the class \"SOMNetGPU\", \nbecause not all of the class implementation can be shifted into a regular header file (device code). \nThis means we need to create an instance of our \"extended\" class right when the library is build.\n\nFor this we can make use of the __general class extensions__. To achieve the same as illustrated in the previous example. \nWe define a file \"NewFunctions.h\":\n\n```\ntemplate \u003cclass T\u003e\ninline T __host__ __device__ custom_learn(T fWeight, T fInfluence, T fInput) {\n\treturn fWeight + (fInfluence*(fInput-fWeight) );\n}\n\ntemplate \u003cclass T\u003e\ninline T __host__ __device__ custom_gaussian_nhood (T dist, T sigmaT) {\n\treturn exp(-pow(dist, 2.f)/(2.f*pow(sigmaT, 2.f)));\n}\n\ntemplate \u003cclass T\u003e\ninline T __host__ __device__ custom_rad_decay (T sigma0, T t, T lambda) {\n\treturn std::floor(sigma0*exp(-t/lambda) + 0.5f);\n}\n\ntemplate \u003cclass T\u003e\ninline T __host__ __device__ custom_lrate_decay (T sigma0, T t, T lambda) {\n\treturn sigma0*exp(-t/lambda);\n}\n\ntemplate\u003cclass T\u003e using custom_functor = ANN::DistFunction\u003cT, custom_learn\u003cT\u003e, custom_gaussian_nhood\u003cT\u003e, custom_rad_decay\u003cT\u003e, custom_lrate_decay\u003cT\u003e \u003e;\n```\n.. and a file \"NewInstances.h\":\n\n```\ntemplate ANNGPGPU::SOMNetGPU\u003cfloat, custom_functor\u003cfloat\u003e \u003e::SOMNetGPU();\ntemplate ANNGPGPU::SOMNetGPU\u003cdouble, custom_functor\u003cdouble\u003e \u003e::SOMNetGPU();\n// ..\n```\nthen we adapt the CMake build file and add the following lines:\n\n```\n# here we pass the extension headers to the build system\nadd_definitions(-D__Functions_ADDONS=\"${SOM_GPU_ADDON_SOURCE_DIR}/NewFunctions.h\")\nadd_definitions(-D__SOMNetGPU_INSTANCES=\"${SOM_GPU_ADDON_SOURCE_DIR}/NewInstances.h\")\n```\n\n# Advanced implementation guide\n\nUsually a network consists of nodes and edges. Most implementations of neural networks store their data in a array. \nCalculation is simple with this approach, but asymmetric networks get hard to implement and new functionality is hard to add.\nTo create more complex variants of networks it makes sense to put the information of the network in the edges and the functionality to calculate parts of the network into the nodes. \nThis makes it easier to include new functionality and to re-use old code.\nBecause information flow in neuronal networks is often directed, container classes help to keep an order. \nThe network class on the other hand calls learning or training functions and implements the principal learning procedure, \ne.g. switching training patterns or break the learning procedure if a certain error limit was hit.\n\nTo make the long story short, the three important classes to derive from are:\n\n* AbsNeuron\n* AbsLayer\n* AbsNet\n\n## AbsNeuron\nIn the library some functions of the abstract base classes are meant to get implemented in the derived class. \nHere I give a small overview about them.\n\n```\nvirtual void AdaptEdges()       = 0;\nvirtual void CalcValue()        = 0;\n```\n\nIn CalcValue() you calculate the data you want to store in the neuron. \nEvery neuron (or node) in the network has a list of edges which direct to neurons of another (or the same) layer. \nThis example shows you how to run through this list to implement a neuron in a back propagation network.\n\n```\ntemplate \u003cclass Type, class Functor\u003e\nvoid BPNeuron\u003cType, Functor\u003e::CalcValue() {\n\tif(this-\u003eGetConsI().size() == 0) {\n\t\treturn;\n\t}\n\n\tType val = 0;\n\tfor(unsigned int i = 0; i \u003c this-\u003eGetConsI().size(); i++) {\n\t\tAbsNeuron\u003cType\u003e *from = this-\u003eGetConI(i)-\u003eGetDestination(this);\n\t\tval += from-\u003eGetValue() * this-\u003eGetConI(i)-\u003eGetValue();\n\t}\n\tthis-\u003eSetValue(val);\n\n\tval = Functor::transfer( this-\u003eGetValue(), 0.f );\n\tthis-\u003eSetValue(val);\n}\n```\n \nThe algorithm to adapt the edges is implemented in AdaptEdges(). \nAgain we use the internal list to run through all edges (outgoing ones) the neuron is connected with.\n\n```\ntemplate \u003cclass Type, class Functor\u003e\nvoid BPNeuron\u003cType, Functor\u003e::AdaptEdges() {\n\tif(this-\u003eGetConsO().size() == 0)\n\t\treturn;\n\n\tAbsNeuron\u003cType\u003e *pCurNeuron;\n\tEdge\u003cType\u003e \t*pCurEdge;\n\tType \t\tval;\n\n\t// calc error deltas\n\tval = this-\u003eGetErrorDelta();\n\tfor(unsigned int i = 0; i \u003c this-\u003eGetConsO().size(); i++) {\n\t\tpCurEdge \t= this-\u003eGetConO(i);\n\t\tpCurNeuron \t= pCurEdge-\u003eGetDestination(this);\n\t\tval += pCurNeuron-\u003eGetErrorDelta() * pCurEdge-\u003eGetValue();\n\t}\n\t\n\tval *= Functor::derivate( this-\u003eGetValue(), 0.f );\n\tthis-\u003eSetErrorDelta(val);\n\n\t// adapt weights\n\tfor(unsigned int i = 0; i \u003c this-\u003eGetConsO().size(); i++) {\n\t\tpCurEdge = this-\u003eGetConO(i);\n\t\tif(pCurEdge-\u003eGetAdaptationState() == true) {\n\t\t\tval = Functor::learn( \tthis-\u003eGetValue(), \n\t\t\t\t\t\tpCurEdge-\u003eGetValue(), \n\t\t\t\t\t\tpCurEdge-\u003eGetMomentum(),\n\t\t\t\t\t\tpCurEdge-\u003eGetDestination(this)-\u003eGetErrorDelta(),\n\t\t\t\t\t\tm_Setup );\n\t\t\t\n\t\t\tpCurEdge-\u003eSetMomentum( val );\n\t\t\tpCurEdge-\u003eSetValue( val+pCurEdge-\u003eGetValue() );\n\t\t}\n\t}\n}\n```\n\n## AbsLayer\n\nNeurons are stored in layers. If you decide to write your own layer class, then you have to implement the Resize() function. \nThis could be useful especially if you have strange layer topologies (e.g. 2-dimensional or 3-dimensional).\n\n```\ntemplate \u003cclass Type, class Functor\u003e\nvoid BPLayer\u003cType, Functor\u003e::Resize(const unsigned int \u0026iSize) {\n\tthis-\u003eEraseAll();\n\tthis-\u003eAddNeurons(iSize);\n}\n```\n\n## AbsNet\n\nThe last class you may want to derive from, is AbsNet. \nHere are three functions you have to overload: PropagateFW() and PropagateBW(). \n\n```\nvirtual void PropagateFW() = 0;\nvirtual void PropagateBW() = 0;\n```\n\nHere I show, how to implement these functions in a back propagation network.\n\n```\ntemplate \u003cclass Type, class Functor\u003e\nvoid BPNet\u003cType, Functor\u003e::PropagateFW() {\n\tfor(unsigned int i = 1; i \u003c this-\u003em_lLayers.size(); i++) {\n\t\tBPLayer\u003cType, Functor\u003e *curLayer = ((BPLayer\u003cType, Functor\u003e*)this-\u003eGetLayer(i) );\n\t\t//#pragma omp parallel for\n\t\tfor(int j = 0; j \u003c static_cast\u003cint\u003e(curLayer-\u003eGetNeurons().size() ); j++) {\n\t\t\tcurLayer-\u003eGetNeuron(j)-\u003eCalcValue();\n\t\t}\n\t}\n}\n\ntemplate \u003cclass Type, class Functor\u003e\nvoid BPNet\u003cType, Functor\u003e::PropagateBW() {\n\tfor(int i = this-\u003em_lLayers.size()-1; i \u003e= 0; i--) {\n\t\tBPLayer\u003cType, Functor\u003e *curLayer = ( (BPLayer\u003cType, Functor\u003e*)this-\u003eGetLayer(i) );\n\t\t//#pragma omp parallel for\n\t\tfor(int j = 0; j \u003c static_cast\u003cint\u003e( curLayer-\u003eGetNeurons().size() ); j++) {\n\t\t\tcurLayer-\u003eGetNeuron(j)-\u003eAdaptEdges();\n\t\t}\n\t}\n}\n```\n\nDifferent Implementations use different types of layers.\nThis is why you may want to overload AddLayer().\n\n```\ntemplate \u003cclass Type, class Functor\u003e\nvoid BPNet\u003cType, Functor\u003e::AddLayer(const unsigned int \u0026iSize, const LayerTypeFlag \u0026flType) {\n\tthis-\u003eAddLayer( new BPLayer\u003cType, Functor\u003e(iSize, flType, -1) );\n}\n```\n# Save and load your custom class data to file-system\n\n## Example: Back propagation networks\n\nIf you decide to create your own e.g. Layer class, then you probably want to add features which require to be stored on the hdd. \nFor this the ExpToFS() and ImpFromFS() functions are required to get modified. This works more or less like reimplementing virtual functions in Qt.\nCalling the virtual base class ensures to save the base class contents. \nThe following example shows how the freshly inserted integer storing the layer depth in a back propagation network can be saved and recovered from the hdd. \n\n```\ntemplate \u003cclass Type, class Functor\u003e\nvoid BPLayer\u003cType, Functor\u003e::ExpToFS(BZFILE* bz2out, int iBZ2Error) {\n\tstd::cout\u003c\u003c\"Save BPLayer to FS()\"\u003c\u003cstd::endl;\n\tAbsLayer\u003cType\u003e::ExpToFS(bz2out, iBZ2Error);\n\n\tint iZLayer = m_iZLayer;\n\tBZ2_bzWrite( \u0026iBZ2Error, bz2out, \u0026iZLayer, sizeof(int) );\n}\n```\n\nNow the other way round, we load the content from the file-system.\n\n```\ntemplate \u003cclass Type, class Functor\u003e\nint BPLayer\u003cType, Functor\u003e::ImpFromFS(BZFILE* bz2in, int iBZ2Error, ConTable\u003cType\u003e \u0026Table) {\n\tstd::cout\u003c\u003c\"Load BPLayer from FS()\"\u003c\u003cstd::endl;\n\tint iLayerID = AbsLayer\u003cType\u003e::ImpFromFS(bz2in, iBZ2Error, Table);\n\n\tint iZLayer = -1;\n\tBZ2_bzRead( \u0026iBZ2Error, bz2in, \u0026iZLayer, sizeof(int) );\n\tTable.ZValOfLayer.push_back(iZLayer);\n\n\treturn iLayerID;\n}\n```\n\nThe last function which has to be implemented is CreateNet(). \nHere the content loaded from the file system is used to create a copy of the net in the memory. \nThe base implementation creates the layers and the connections of the network, so we just have to implement the bias neuron.\n\n```\ntemplate \u003cclass Type, class Functor\u003e\nvoid BPNet\u003cType, Functor\u003e::CreateNet(const ConTable\u003cType\u003e \u0026Net) {\n\tstd::cout\u003c\u003c\"Create BPNet\"\u003c\u003cstd::endl;\n\n\t/*\n\t* Init\n\t*/\n\tunsigned int iDstNeurID = 0;\n\tunsigned int iDstLayerID = 0;\n\tunsigned int iSrcLayerID = 0;\n\n\tType fEdgeValue = 0.f;\n\n\tAbsLayer\u003cType\u003e *pDstLayer = NULL;\n\tAbsLayer\u003cType\u003e *pSrcLayer = NULL;\n\tAbsNeuron\u003cType\u003e *pDstNeur = NULL;\n\tAbsNeuron\u003cType\u003e *pSrcNeur = NULL;\n\n\t/*\n\t* For all nets necessary: Create Connections (Edges)\n\t*/\n\tAbsNet\u003cType\u003e::CreateNet(Net);\n\n\t/*\n\t* Support z-layers\n\t*/\n\tfor(unsigned int i = 0; i \u003c this-\u003em_lLayers.size(); i++) {\n\t\tBPLayer\u003cType, Functor\u003e *curLayer = ((BPLayer\u003cType, Functor\u003e*)this-\u003eGetLayer(i) );\n\t\tcurLayer-\u003eSetZLayer(Net.ZValOfLayer[i]);\n\t}\n}\n```\n\n## Example: Self organizing maps\n\nHere is another example for SOMs. Only the import of the positions has to be added.\n\n```\ntemplate\u003cclass Type, class Functor\u003e\nvoid SOMNet\u003cType, Functor\u003e::CreateNet(const ConTable\u003cType\u003e \u0026Net) {\n\tstd::cout\u003c\u003c\"Create SOMNet\"\u003c\u003cstd::endl;\n\n\t/*\n\t* For all nets necessary: Create Connections (Edges)\n\t*/\n\tAbsNet\u003cType\u003e::CreateNet(Net);\n\n\t/*\n\t* Set Positions\n\t*/\n\tfor(unsigned int i = 0; i \u003c Net.Neurons.size(); i++) {\n\t\tint iLayerID \t= Net.Neurons.at(i).m_iLayerID;\n\t\tint iNeurID \t= Net.Neurons.at(i).m_iNeurID;\n\t\t\n\t\t// Get position\n\t\tint iPosSize = Net.Neurons.at(i).m_vMisc.at(0);\n\t\tstd::vector\u003cType\u003e vPos(iPosSize);\n\t\tfor(int j = 0; j \u003c iPosSize; j++) {\n\t\t\tvPos[j] = Net.Neurons.at(i).m_vMisc[1+j];\n\t\t}\n\t\t\n\t\t// Save other information of the neuron\n\t\tANN::SOMNeuron\u003cType\u003e *pNeuron = (ANN::SOMNeuron\u003cType\u003e *)this-\u003eGetLayer(iLayerID)-\u003eGetNeuron(iNeurID);\n\t\tpNeuron-\u003eSetPosition(vPos);\n\t\tpNeuron-\u003eSetLearningRate(Net.Neurons.at(i).m_vMisc[iPosSize+1]);\n\t\tpNeuron-\u003eSetSigma0(Net.Neurons.at(i).m_vMisc[iPosSize+2]);\n\t}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FANNetGPGPU%2FANNetGPGPU","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FANNetGPGPU%2FANNetGPGPU","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FANNetGPGPU%2FANNetGPGPU/lists"}