{"id":13541490,"url":"https://github.com/ajtulloch/dnngraph","last_synced_at":"2025-12-11T23:28:22.684Z","repository":{"id":27341486,"uuid":"30816374","full_name":"ajtulloch/dnngraph","owner":"ajtulloch","description":"A DSL for deep neural networks, supporting Caffe and Torch","archived":false,"fork":false,"pushed_at":"2015-12-07T04:39:34.000Z","size":512,"stargazers_count":708,"open_issues_count":9,"forks_count":58,"subscribers_count":39,"default_branch":"master","last_synced_at":"2025-08-19T12:42:20.605Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://ajtulloch.github.io/dnngraph","language":"Haskell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ajtulloch.png","metadata":{"files":{"readme":"README.org","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-02-15T03:11:12.000Z","updated_at":"2025-08-04T23:37:11.000Z","dependencies_parsed_at":"2022-09-02T05:11:41.653Z","dependency_job_id":null,"html_url":"https://github.com/ajtulloch/dnngraph","commit_stats":null,"previous_names":["ajtulloch/caffegraph"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ajtulloch/dnngraph","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajtulloch%2Fdnngraph","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajtulloch%2Fdnngraph/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajtulloch%2Fdnngraph/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajtulloch%2Fdnngraph/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ajtulloch","download_url":"https://codeload.github.com/ajtulloch/dnngraph/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajtulloch%2Fdnngraph/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274402107,"owners_count":25278334,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-10T02:00:12.551Z","response_time":83,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T10:00:49.037Z","updated_at":"2025-10-21T14:07:14.631Z","avatar_url":"https://github.com/ajtulloch.png","language":"Haskell","funding_links":[],"categories":["Haskell Packages","Haskell"],"sub_categories":["Legacy Packages","Tools","[Tools](#tools-1)","Speech Recognition"],"readme":"* DNNGraph - A deep neural network model generation DSL in Haskell\nIt consists of several parts:\n\n- A DSL for specifying the model. This uses the [[http://lens.github.io/][lens]] library for\n  elegant, composable constructions, and the [[http://hackage.haskell.org/package/fgl-5.5.0.1][fgl]] graph library for\n  specifying the network layout.\n- A set of optimization passes that run over the graph representation\n  to improve the performance of the model. For example, we can take\n  advantage of the fact that several layers types (=ReLU=, =Dropout=)\n  can operate in-place.\n- A set of backends to generate code for the platform.  Currently, we\n  generate\n  - Caffe (by generating model =prototxt= files)\n  - Torch (by generating Lua scripts)\n- A set of useful CLI tools for exporting, visualizing and\n  understanding a model (visualization of network structure, parameter\n  density)\n\nFor a guided example, see a [[http://bit.ly/17kDYze][demonstration IHaskell Notebook]].\n** Building\nMake sure that you have Python 2 and =protoc= from [[https://developers.google.com/protocol-buffers/][Protocol Buffers]] installed. Then run\n#+BEGIN_SRC \n$ cabal install hprotoc\n$ ./lens_proto.sh # generate code from protocol buffers\n$ cabal install\n#+END_SRC\n\n\n\n** DSL Examples\nThe following script generates a replica of\nhttps://github.com/BVLC/caffe/blob/master/models/bvlc_alexnet/train_val.prototxt.\n\n*** AlexNet\n#+begin_src haskell\n  import           Control.Lens\n  import           Control.Monad\n\n  import           NN.DSL\n  import           NN.Examples.ImageNet\n  import           NN.Graph\n\n  alexTrain = train \u0026 cropSize' 227 \u0026 batchSize' 256 \u0026 mirror' True\n  alexTest = test \u0026 cropSize' 227 \u0026 batchSize' 50 \u0026 mirror' False\n\n  alexLrn = lrn \u0026 localSize' 5 \u0026 alphaLRN' 0.0001 \u0026 betaLRN' 0.75\n  alexConv = conv \u0026 param' alexMult \u0026 weightFillerC' (gaussian 0.01) \u0026 biasFillerC' zero\n  alexIP n = ip n \u0026 param' alexMult \u0026 weightFillerIP' (gaussian 0.005) \u0026 biasFillerIP' (constant 0.1)\n  alexPool = maxPool \u0026 sizeP' 3\n\n  alexMult = [def \u0026 lrMult' 1 \u0026 decayMult' 1, -- weights\n              def \u0026 lrMult' 2 \u0026 decayMult' 0] -- biases\n\n  -- |Model\n  conv1 = alexConv \u0026 numOutputC' 96 \u0026 kernelSizeC' 11 \u0026 strideC' 4\n  conv2 = alexConv \u0026 numOutputC' 256 \u0026 padC' 2 \u0026 kernelSizeC' 5 \u0026 groupC' 2\n  conv3 = alexConv \u0026 numOutputC' 384 \u0026 padC' 1 \u0026 kernelSizeC' 3\n  conv4 = alexConv \u0026 numOutputC' 384 \u0026 padC' 1 \u0026 kernelSizeC' 3 \u0026 groupC' 2 \u0026 biasFillerC' (constant 0.1)\n  conv5 = alexConv \u0026 numOutputC' 256 \u0026 padC' 1 \u0026 kernelSizeC' 3 \u0026 groupC' 2 \u0026 biasFillerC' (constant 0.1)\n\n  alexNet = do\n    -- Set up the model\n    (input', representation) \u003c-\n        sequential [\n             -- Convolutional Layers\n             conv1, relu, alexLrn, alexPool \u0026 strideP' 3,\n             conv2, relu, alexLrn, alexPool \u0026 strideP' 2,\n             conv3, relu,\n             conv4, relu,\n             conv5, relu, alexPool \u0026 strideP' 2,\n             -- FC Layers\n             alexIP 4096, relu, dropout 0.5,\n             alexIP 4096, relu, dropout 0.5,\n             alexIP 1000 \u0026 weightFillerIP' (gaussian 0.01) \u0026 biasFillerIP' zero]\n\n    forM_ [alexTrain, alexTest] $ attach (To input')\n    forM_ [accuracy 1, accuracy 5, softmax] $ attach (From representation)\n#+end_src\n\nor visually, using =NN.Visualize=,\n\n#+ATTR_HTML: :height 600px\n[[http://i.imgur.com/1hKlPdA.png]]\n\n*** GoogLeNet\nThe following script generates a replica of\nhttps://github.com/BVLC/caffe/blob/master/models/bvlc_googlenet/train_val.prototxt\n\n#+begin_src haskell\n  module NN.Examples.GoogLeNet where\n\n  import           Gen.Caffe.FillerParameter       as FP\n  import           Gen.Caffe.InnerProductParameter as IP\n  import           Gen.Caffe.LayerParameter        as LP\n\n  import           Control.Lens\n  import           Control.Monad\n  import           Data.Sequence                   (singleton)\n  import           Data.Word\n\n  import           NN\n  import           NN.Examples.ImageNet\n\n\n  googleTrain = train \u0026 mirror' True \u0026 batchSize' 32 \u0026 cropSize' 224\n  googleTest = test \u0026 mirror' False \u0026 batchSize' 50 \u0026 cropSize' 224\n\n  googleMult = [def \u0026 lrMult' 1 \u0026 decayMult' 1, -- weights\n                def \u0026 lrMult' 2 \u0026 decayMult' 0] -- biases\n  googleConv = conv \u0026 param' googleMult \u0026 biasFillerC' (constant 0.2)\n  googleLRN = lrn \u0026 localSize' 5 \u0026 alphaLRN' 0.0001 \u0026 betaLRN' 0.75\n  googlePool = maxPool \u0026 sizeP' 3 \u0026 strideP' 2\n  googleIP n = ip n \u0026 param' googleMult\n\n  conv1 = googleConv \u0026 numOutputC' 64 \u0026 padC' 3 \u0026 kernelSizeC' 7 \u0026 strideC' 2 \u0026 weightFillerC' (xavier 0.1)\n  conv2 = googleConv \u0026 numOutputC' 192 \u0026 padC' 1 \u0026 kernelSizeC' 3 \u0026 weightFillerC' (xavier 0.03)\n\n  topPool = avgPool \u0026 sizeP' 7 \u0026 strideP' 1\n  topFc = googleIP 1000 \u0026 biasFillerIP' (constant 0) \u0026 weightFillerIP' (xavier 0.0)\n          -- Weird, but in Caffe replication\n          \u0026 _inner_product_param._Just.IP._weight_filler._Just._std .~ Nothing\n\n  data Inception = Inception {_1x1, _3x3reduce, _3x3, _5x5reduce, _5x5, _poolProj :: Word32}\n\n  inception :: Node -\u003e Inception -\u003e NetBuilder Node\n  inception input Inception{..} = do\n    columns' \u003c- mapM sequential columns\n    concat'' \u003c- layer' concat'\n    forM_ columns' $ \\(bottom, top) -\u003e do\n                                    input \u003e-\u003e bottom\n                                    top \u003e-\u003e concat''\n    return concat''\n      where\n        columns = [\n         [googleConv \u0026 numOutputC' _1x1  \u0026 kernelSizeC' 1 \u0026 weightFillerC' (xavier 0.03), relu],\n         [googleConv \u0026 numOutputC' _3x3reduce \u0026 kernelSizeC' 1 \u0026 weightFillerC' (xavier 0.09), relu, googleConv \u0026 numOutputC' _3x3 \u0026 kernelSizeC' 3 \u0026 weightFillerC' (xavier 0.03) \u0026 padC' 1, relu],\n         [googleConv \u0026 numOutputC' _5x5reduce \u0026 kernelSizeC' 1 \u0026 weightFillerC' (xavier 0.2), relu, googleConv \u0026 numOutputC' _5x5 \u0026 kernelSizeC' 5 \u0026 weightFillerC' (xavier 0.03) \u0026 padC' 2, relu],\n         [maxPool\u0026 sizeP' 3 \u0026 strideP' 3 \u0026 padP' 1, googleConv \u0026 numOutputC' _poolProj \u0026 kernelSizeC' 1 \u0026 weightFillerC' (xavier 0.1), relu]]\n\n  intermediateClassifier :: Node -\u003e NetBuilder ()\n  intermediateClassifier source = do\n    (input, representation) \u003c- sequential [pool1, conv1', relu, fc1, relu, dropout 0.7, fc2]\n    source \u003e-\u003e input\n\n    forM_ [accuracy 1, accuracy 5, softmax \u0026 _loss_weight \u003c\u003e~ singleton 0.3] $ attach (From representation)\n      where\n        pool1 = avgPool \u0026 sizeP' 5 \u0026 strideP' 3\n        conv1' = googleConv \u0026 numOutputC' 128 \u0026 kernelSizeC' 1 \u0026 weightFillerC' (xavier 0.08)\n        fc1 = googleIP 1024 \u0026 weightFillerIP' (xavier 0.02) \u0026 biasFillerIP' (constant 0.2)\n        fc2 = googleIP 1000 \u0026 weightFillerIP' (xavier 0.0009765625) \u0026 biasFillerIP' (constant 0)\n\n  -- What to do at each row in the inner column?\n  data Row = I Inception | Classifier | MaxPool\n\n  insertRow :: Node -\u003e Row -\u003e NetBuilder Node\n  insertRow input (I inceptor) = inception input inceptor\n  insertRow input Classifier = do\n    intermediateClassifier input\n    return input\n  insertRow input MaxPool = do\n    node \u003c- layer' googlePool\n    input \u003e-\u003e node\n    return node\n\n  googLeNet :: NetBuilder ()\n  googLeNet = do\n    (input, initial) \u003c- sequential [conv1, relu, googlePool, googleLRN, conv2, relu, googleLRN, googlePool]\n\n    top \u003c- foldM insertRow initial [\n               I $ Inception 64 96 128 16 32 32,\n               I $ Inception 128 128 192 32 96 64,\n               MaxPool,\n               I $ Inception 192 96 208 16 48 64,\n               Classifier,\n               I $ Inception 150 112 224 24 64 64,\n               I $ Inception 128 128 256 24 64 64,\n               I $ Inception 112 144 288 32 64 64,\n               Classifier,\n               I $ Inception 256 160 320 32 128 128,\n               MaxPool,\n               I $ Inception 256 160 320 32 128 128,\n               I $ Inception 384 192 384 48 128 128]\n\n    (_, representation) \u003c- with top \u003e- sequential [topPool, dropout 0.4, topFc]\n\n    forM_ [accuracy 1, accuracy 5, softmax] $ attach (From representation)\n    forM_ [googleTrain, googleTest] $ attach (To input)\n\n  main :: IO ()\n  main = cli googLeNet\n#+end_src\n\n** CLI Usage\nIn the GoogLeNet example, above, we included the line =main = cli\ngoogLeNet=. This generates a CLI for our model that can be accessed\nwith =runhaskell /path/to/our/model.hs=.  Currently, we can\n\n- export to Caffe\n- export to Torch\n- visualize the network structure.\n\nFor example:\n#+BEGIN_SRC \n$ runhaskell NN/Examples/GoogLeNet.hs --help\nUsage: GoogLeNet.hs COMMAND\n\nAvailable options:\n  -h,--help                Show this help text\n\nAvailable commands:\n  caffe                    Generate a Caffe .prototxt to run with `caffe train\n                           --model=\u003c\u003e\n  torch                    Generate Lua code to be `require`'d into an existing\n                           Torch script\n  visualize                Generate an image visualizing the model's connectivity\n\n$ runhaskell NN/Examples/GoogLeNet.hs caffe --output /tmp/x.prototxt\n$ runhaskell NN/Examples/GoogLeNet.hs visualize --format pdf --output /tmp/x.pdf\n#+END_SRC\n\n** Caffe Backend\nThe Caffe backend generates a Caffe =.prototxt= that can be run with\n=caffe train --model=\u003c\u003e=, without any modification necessary.\n\n** Torch Backend\nThe Torch backend generates Lua code that can be imported directly\ninto an existing Torch script.\n\nAnything network that can be expressed as a nested combination of\ncomputational layers, combined with =nn.Sequential=, =nn.Concat=,\n=nn.ModelParallel=, =nn.DataParallel= etc can be generated under this framework.\n\nFor an example output, the model specified as\n\n#+begin_src haskell\n  alexTrain = train \u0026 cropSize' 227 \u0026 batchSize' 256 \u0026 mirror' True\n  alexTest = test \u0026 cropSize' 227 \u0026 batchSize' 50 \u0026 mirror' False\n\n  alexConv = conv \u0026 param' alexMult \u0026 weightFillerC' (gaussian 0.01) \u0026 biasFillerC' zero\n  alexPool = maxPool \u0026 sizeP' 3\n\n  conv1 = alexConv \u0026 numOutputC' 96 \u0026 kernelSizeC' 11 \u0026 strideC' 4\n  pool1 = alexPool \u0026 strideP' 3\n\n  model = do\n    (input', representation) \u003c- sequential [conv1, relu, pool1]\n    forM_ [alexTrain, alexTest] $ attach (To input')\n    forM_ [accuracy 1, accuracy 5, softmax] $ attach (From representation)\n#+end_src\n\ngenerates the following code:\n\n#+begin_src lua\n  require(\"nn\")\n  require(\"cunn\")\n  local seq0 = nn.Sequential()\n  seq0:add(nn.SpatialConvolutionMM(nil, 96, 11, 11, 4, 4, 0))\n  seq0:add(nn.Threshold())\n  seq0:add(nn.SpatialMaxPooling(3, 3, 3, 3))\n  seq0:add(nn.LogSoftMax())\n  local criterion1 = nn.ClassNLLCriterion()\n  return seq0, criterion1\n#+end_src\n\nFor a more complicated example, the network specified as\n\n#+begin_src haskell\n  do\n    x \u003c- layer' relu\n    (_, y) \u003c- with x \u003e- sequential [conv, relu, maxPool, conv, relu]\n    (_, z) \u003c- with x \u003e- sequential [conv, relu, maxPool, conv, relu]\n    concat'' \u003c- layer' concat'\n\n    y \u003e-\u003e concat''\n    z \u003e-\u003e concat''\n    _ \u003c- with concat'' \u003e- sequential [ip 4096, relu, dropout 0.5, ip 1000, softmax]\n    return ()\n#+end_src\n\nthat looks like\n\n#+ATTR_HTML: :height 600px\n[[http://i.imgur.com/dsqgYna.png][http://i.imgur.com/dsqgYna.png]]\n\nwill generate\n#+begin_src lua\nrequire(\"nn\")\nlocal seq0 = nn.Sequential()\nlocal mod1 = nn.Threshold()\nseq0:add(mod1)\nlocal concat2 = nn.DepthConcat()\nlocal seq3 = nn.Sequential()\nlocal mod4 = nn.SpatialConvolutionMM(nil, nil, nil, nil, 1, 1, 0)\nseq3:add(mod4)\nlocal mod5 = nn.Threshold()\nseq3:add(mod5)\nlocal mod6 = nn.SpatialMaxPooling(nil, nil, 1, 1)\nseq3:add(mod6)\nlocal mod7 = nn.SpatialConvolutionMM(nil, nil, nil, nil, 1, 1, 0)\nseq3:add(mod7)\nlocal mod8 = nn.Threshold()\nseq3:add(mod8)\nconcat2:add(seq3)\nlocal seq9 = nn.Sequential()\nlocal mod10 = nn.SpatialConvolutionMM(nil, nil, nil, nil, 1, 1, 0)\nseq9:add(mod10)\nlocal mod11 = nn.Threshold()\nseq9:add(mod11)\nlocal mod12 = nn.SpatialMaxPooling(nil, nil, 1, 1)\nseq9:add(mod12)\nlocal mod13 = nn.SpatialConvolutionMM(nil, nil, nil, nil, 1, 1, 0)\nseq9:add(mod13)\nlocal mod14 = nn.Threshold()\nseq9:add(mod14)\nconcat2:add(seq9)\nseq0:add(concat2)\nlocal mod15 = nn.Linear(nil, 4096)\nseq0:add(mod15)\nlocal mod16 = nn.Threshold()\nseq0:add(mod16)\nlocal mod17 = nn.Dropout(0.5)\nseq0:add(mod17)\nlocal mod18 = nn.Linear(nil, 1000)\nseq0:add(mod18)\nlocal mod19 = nn.LogSoftMax()\nseq0:add(mod19)\nlocal criteria20 = nn.ClassNLLCriterion()\nreturn seq0, criteria20\n#+end_src\n\n** Visualization Examples\nThe =NN.Visualize= module provides some plotting tools. To use these,\n\n#+begin_src haskell\n  import NN.Visualize\n\n  visualize :: Net -\u003e DotGraph Node\n  png :: FilePath -\u003e DotGraph Node -\u003e IO FilePath\n\n  -- For example, to visualize GoogLeNet to a file\n  file :: FilePath\n  (frontend googLeNet \u0026 visualize \u0026 png file) :: IO FilePath\n#+end_src\n\nAn example output is (click for higher resolution):\n#+ATTR_HTML: :height 600px\n[[http://i.imgur.com/ScvjNmT.jpg]]\n** Parameter Sweeps\nTo use this, write your model generation script as a Haskell file, and\nthen (for example)\n#+begin_src sh\n  caffe train --model \u003c(runhaskell Model.hs) --solver=solver.prototxt\n#+end_src\n\nTo perform a parameter sweep, use the parameterizing\n#+begin_src sh\n  for model in $(runhaskell Model.hs); do\n      caffe train --model=$model --solver=solver.prototxt\n  done\n#+end_src\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fajtulloch%2Fdnngraph","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fajtulloch%2Fdnngraph","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fajtulloch%2Fdnngraph/lists"}