{"id":20599170,"url":"https://github.com/dusenberrymw/systemml-nn","last_synced_at":"2025-09-02T23:43:40.563Z","repository":{"id":73171534,"uuid":"55621849","full_name":"dusenberrymw/systemml-nn","owner":"dusenberrymw","description":"A deep learning library for Apache SystemML.","archived":false,"fork":false,"pushed_at":"2018-01-03T04:28:44.000Z","size":163,"stargazers_count":9,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-28T12:21:18.648Z","etag":null,"topics":["data-science","deep-learning","machine-learning","neural-networks","systemml"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dusenberrymw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-04-06T16:22:26.000Z","updated_at":"2023-09-23T02:21:22.000Z","dependencies_parsed_at":null,"dependency_job_id":"452f45c4-33f4-439a-88c1-b47e4cd604cb","html_url":"https://github.com/dusenberrymw/systemml-nn","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dusenberrymw%2Fsystemml-nn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dusenberrymw%2Fsystemml-nn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dusenberrymw%2Fsystemml-nn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dusenberrymw%2Fsystemml-nn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dusenberrymw","download_url":"https://codeload.github.com/dusenberrymw/systemml-nn/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248983684,"owners_count":21193621,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-science","deep-learning","machine-learning","neural-networks","systemml"],"created_at":"2024-11-16T08:31:33.115Z","updated_at":"2025-04-15T00:29:51.496Z","avatar_url":"https://github.com/dusenberrymw.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SystemML-NN\n\n### A deep learning library for [Apache SystemML](https://github.com/apache/incubator-systemml).\n\n### \\*\\*Update\\*\\*: The SystemML-NN library has been merged into the Apache SystemML project, and can be found at https://github.com/apache/systemml/tree/master/scripts/nn.\n\n## Examples:\n#### Please see the [`examples`](nn/examples) folder for more detailed examples, or view the following two quick examples.\n### Neural net for regression with vanilla SGD:\n```python\n# Imports\nsource(\"nn/layers/affine.dml\") as affine\nsource(\"nn/layers/l2_loss.dml\") as l2_loss\nsource(\"nn/layers/relu.dml\") as relu\nsource(\"nn/optim/sgd.dml\") as sgd\n\n# Generate input data\nN = 1024 # num examples\nD = 100 # num features\nt = 1 # num targets\nX = rand(rows=N, cols=D, pdf=\"normal\")\ny = rand(rows=N, cols=t)\n\n# Create 2-layer network:\n## affine1 -\u003e relu1 -\u003e affine2\nM = 64 # number of neurons\n[W1, b1] = affine::init(D, M)\n[W2, b2] = affine::init(M, t)\n\n# Initialize optimizer\nlr = 0.05  # learning rate\nmu = 0.9  # momentum\ndecay = 0.99  # learning rate decay constant\n\n# Optimize\nprint(\"Starting optimization\")\nbatch_size = 32\nepochs = 5\niters = 1024 / batch_size\nfor (e in 1:epochs) {\n  for(i in 1:iters) {\n    # Get next batch\n    X_batch = X[i:i+batch_size-1,]\n    y_batch = y[i:i+batch_size-1,]\n\n    # Compute forward pass\n    out1 = affine::forward(X_batch, W1, b1)\n    outr1 = relu::forward(out1)\n    out2 = affine::forward(outr1, W2, b2)\n\n    # Compute loss\n    loss = l2_loss::forward(out2, y_batch)\n    print(\"L2 loss: \" + loss)\n\n    # Compute backward pass\n    dout2 = l2_loss::backward(out2, y_batch)\n    [doutr1, dW2, db2] = affine::backward(dout2, outr1, W2, b2)\n    dout1 = relu::backward(doutr1, out1)\n    [dX_batch, dW1, db1] = affine::backward(dout1, X_batch, W1, b1)\n\n    # Optimize with vanilla SGD\n    W1 = sgd::update(W1, dW1, lr)\n    b1 = sgd::update(b1, db1, lr)\n    W2 = sgd::update(W2, dW2, lr)\n    b2 = sgd::update(b2, db2, lr)\n  }\n  # Decay learning rate\n  lr = lr * decay\n}\n```\n\n### Neural net for multi-class classification with dropout and SGD w/ Nesterov momentum:\n```python\n# Imports\nsource(\"nn/layers/affine.dml\") as affine\nsource(\"nn/layers/cross_entropy_loss.dml\") as cross_entropy_loss\nsource(\"nn/layers/dropout.dml\") as dropout\nsource(\"nn/layers/relu.dml\") as relu\nsource(\"nn/layers/softmax.dml\") as softmax\nsource(\"nn/optim/sgd_nesterov.dml\") as sgd_nesterov\n\n# Generate input data\nN = 1024 # num examples\nD = 100 # num features\nt = 5 # num targets\nX = rand(rows=N, cols=D, pdf=\"normal\")\nclasses = round(rand(rows=N, cols=1, min=1, max=t, pdf=\"uniform\"))\ny = matrix(0, rows=N, cols=t)\nparfor (i in 1:N) {\n  y[i, as.scalar(classes[i,1])] = 1  # one-hot encoding\n}\n\n# Create network:\n# affine1 -\u003e relu1 -\u003e dropout1 -\u003e affine2 -\u003e relu2 -\u003e dropout2 -\u003e affine3 -\u003e softmax\nH1 = 64 # number of neurons in 1st hidden layer\nH2 = 64 # number of neurons in 2nd hidden layer\np = 0.5  # dropout probability\n[W1, b1] = affine::init(D, H1)\n[W2, b2] = affine::init(H1, H2)\n[W3, b3] = affine::init(H2, t)\n\n# Initialize SGD w/ Nesterov momentum optimizer\nlr = 0.05  # learning rate\nmu = 0.5  # momentum\ndecay = 0.99  # learning rate decay constant\nvW1 = sgd_nesterov::init(W1); vb1 = sgd_nesterov::init(b1)\nvW2 = sgd_nesterov::init(W2); vb2 = sgd_nesterov::init(b2)\nvW3 = sgd_nesterov::init(W3); vb3 = sgd_nesterov::init(b3)\n\n# Optimize\nprint(\"Starting optimization\")\nbatch_size = 64\nepochs = 10\niters = 1024 / batch_size\nfor (e in 1:epochs) {\n  for(i in 1:iters) {\n    # Get next batch\n    X_batch = X[i:i+batch_size-1,]\n    y_batch = y[i:i+batch_size-1,]\n\n    # Compute forward pass\n    ## layer 1:\n    out1 = affine::forward(X_batch, W1, b1)\n    outr1 = relu::forward(out1)\n    [outd1, maskd1] = dropout::forward(outr1, p, -1)\n    ## layer 2:\n    out2 = affine::forward(outd1, W2, b2)\n    outr2 = relu::forward(out2)\n    [outd2, maskd2] = dropout::forward(outr2, p, -1)\n    ## layer 3:\n    out3 = affine::forward(outd2, W3, b3)\n    probs = softmax::forward(out3)\n\n    # Compute loss\n    loss = cross_entropy_loss::forward(probs, y_batch)\n    print(\"Cross entropy loss: \" + loss)\n\n    # Compute backward pass\n    ## loss:\n    dprobs = cross_entropy_loss::backward(probs, y_batch)\n    ## layer 3:\n    dout3 = softmax::backward(dprobs, out3)\n    [doutd2, dW3, db3] = affine::backward(dout3, outd2, W3, b3)\n    ## layer 2:\n    doutr2 = dropout::backward(doutd2, outr2, p, maskd2)\n    dout2 = relu::backward(doutr2, out2)\n    [doutd1, dW2, db2] = affine::backward(dout2, outd1, W2, b2)\n    ## layer 1:\n    doutr1 = dropout::backward(doutd1, outr1, p, maskd1)\n    dout1 = relu::backward(doutr1, out1)\n    [dX_batch, dW1, db1] = affine::backward(dout1, X_batch, W1, b1)\n\n    # Optimize with SGD w/ Nesterov momentum\n    [W1, vW1] = sgd_nesterov::update(W1, dW1, lr, mu, vW1)\n    [b1, vb1] = sgd_nesterov::update(b1, db1, lr, mu, vb1)\n    [W2, vW2] = sgd_nesterov::update(W2, dW2, lr, mu, vW2)\n    [b2, vb2] = sgd_nesterov::update(b2, db2, lr, mu, vb2)\n    [W3, vW3] = sgd_nesterov::update(W3, dW3, lr, mu, vW3)\n    [b3, vb3] = sgd_nesterov::update(b3, db3, lr, mu, vb3)\n  }\n  # Anneal momentum towards 0.999\n  mu = mu + (0.999 - mu)/(1+epochs-e)\n  # Decay learning rate\n  lr = lr * decay\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdusenberrymw%2Fsystemml-nn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdusenberrymw%2Fsystemml-nn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdusenberrymw%2Fsystemml-nn/lists"}