{"id":15961551,"url":"https://github.com/cuuupid/better-tensorflow","last_synced_at":"2026-04-29T06:38:01.454Z","repository":{"id":80644363,"uuid":"125315437","full_name":"cuuupid/better-tensorflow","owner":"cuuupid","description":"MNIST classifier api","archived":false,"fork":false,"pushed_at":"2018-03-15T15:18:39.000Z","size":1614,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-27T22:40:38.345Z","etag":null,"topics":["graph-model","machine-learning","neural-network","numpy","pickle","sigmoid"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cuuupid.png","metadata":{"files":{"readme":"README.html","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-15T05:12:49.000Z","updated_at":"2019-01-13T21:37:22.000Z","dependencies_parsed_at":"2023-03-26T12:52:34.855Z","dependency_job_id":null,"html_url":"https://github.com/cuuupid/better-tensorflow","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/cuuupid/better-tensorflow","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuuupid%2Fbetter-tensorflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuuupid%2Fbetter-tensorflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuuupid%2Fbetter-tensorflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuuupid%2Fbetter-tensorflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cuuupid","download_url":"https://codeload.github.com/cuuupid/better-tensorflow/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuuupid%2Fbetter-tensorflow/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32414422,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-29T06:29:02.080Z","status":"ssl_error","status_checked_at":"2026-04-29T06:29:00.631Z","response_time":110,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["graph-model","machine-learning","neural-network","numpy","pickle","sigmoid"],"created_at":"2024-10-07T15:42:47.453Z","updated_at":"2026-04-29T06:38:01.440Z","avatar_url":"https://github.com/cuuupid.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 id=\"the-graph\"\u003eThe Graph\u003c/h1\u003e\n\u003ch2 id=\"preprocessing\"\u003ePreprocessing\u003c/h2\u003e\n\u003cp\u003eTo preprocess, we downloaded the MNIST pickle data (gzipped) and preprocessed the images with the following methods:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\u003cp\u003eReshape and standardize the image (\u003ccode\u003enp.reshape(image / 255, (29 * 29, 1))\u003c/code\u003e)\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003edividing by 255 standardizes as pixel values are grayscale 0-255\u003c/li\u003e\n\u003cli\u003emnist pictures are supposed to be 29 \u003cem\u003e 29, so instead of using -1 here which would flatten, we use 29 \u003c/em\u003e 29 to also assert that the image is a valid MNIST image\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cp\u003eOne-hot encode results (make the vector using \u003ccode\u003enp.zeros(10)\u003c/code\u003e and then encode using \u003ccode\u003evector[label_index] = 1\u003c/code\u003e)\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003emaking an array of zeroes creates a vector that we can one-hot encode (i.e. every index except the label\u0026#39;s index is 0)\u003c/li\u003e\n\u003cli\u003enext we can get the label index as the label int, as labels are 0-9, so \u003ccode\u003eint(label)\u003c/code\u003e works out the index\u003c/li\u003e\n\u003cli\u003eone-hot encoding is important since this is a classification problem, so by one-hot encoding instead of using a rounding method, we can access the raw value assigned/activated for each class, which we can then consider the \u0026quot;confidence\u0026quot;\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cp\u003eCombine features and labels (\u003ccode\u003enp.array(list(zip(X, y))))\u003c/code\u003e)\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cp\u003eSplit into train, val, test (70%, 20%, and 10% is what we chose in the end for ratios)\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cp\u003eSave as pickle (\u003ccode\u003epickle.dump((train, val, test), open(\u0026#39;./data/mnist_preprocessed.pkl\u0026#39;, \u0026#39;wb\u0026#39;))\u003c/code\u003e)\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003ewe create a tuple (train, test, val) to save everything all at once but also split into ratios\u003c/li\u003e\n\u003cli\u003ewe use the Python pickle interface to pickle objects together and write them to a file\u003c/li\u003e\n\u003cli\u003ethis is more memory efficient that preprocessing each time\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eData can then be loaded using:\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"lang-python\"\u003etrain, val, test = pickle.\u003cspan class=\"hljs-keyword\"\u003eload\u003c/span\u003e(\n    \u003cspan class=\"hljs-keyword\"\u003eopen\u003c/span\u003e(\u003cspan class=\"hljs-string\"\u003e'./data/mnist_preprocessed.pkl'\u003c/span\u003e, \u003cspan class=\"hljs-string\"\u003e'rb'\u003c/span\u003e), \u003cspan class=\"hljs-keyword\"\u003eencoding\u003c/span\u003e=\u003cspan class=\"hljs-string\"\u003e'latin1'\u003c/span\u003e\n)\n\u003c/code\u003e\u003c/pre\u003e\n\u003ch2 id=\"inspirations\"\u003eInspirations\u003c/h2\u003e\n\u003cp\u003eRather than arbitrarily choose a structure, we researched existing frameworks and common guidelines for defining ML frameworks. We took inspiration from the Session-Graph model that TensorFlow uses, as well as the Model system in Keras. We combined these into a Graph model, which borrows what we believe are the most convenient features from Keras and Tensorflow, and then implemented each method in Numpy using Gradient Descent. We also noted that for saving and loading models, HD5 and Protobuf were very powerful and popular. However, putting this into prod with an API, we discovered these libraries, although efficient, used up too many resources. We instead opted for NPZ (compressed numpy archives), which is convenient as the model is written in pure numpy, and is also popular in implementations. For the architecture, we tried many different hyperparameters, and were able to get steady results on with hidden units of 42 and 16 respectively without the use of convolutional, batch normalization, or pooling layers, and a batch size of 64. With further research, we discovered another numpy implementation by Michael Nielsen in his book \u0026quot;Neural Networks and Deep Learning.\u0026quot; This implementation used a 30-10 network with a batch size of 32; as this proved to be more accurate, we chose this in training. In production, as we recognize the tradeoff between speed and accuracy, we compromised at a batch size of 128, which proved to be the point of diminishing returns in terms of speed gained from increasing batch size. This has worked steadily and we can easily reach accuracies of over 92% using this model with 20 epochs.\u003c/p\u003e\n\u003ch2 id=\"activation-functions\"\u003eActivation Functions\u003c/h2\u003e\n\u003cp\u003eWe chose to use sigmoid as this is a classification problem and sigmoid is standard/excels at these discrete, class-based approaches. Sigmoid also scales between 0 and 1, so we can consider the output the confidence.\u003c/p\u003e\n\u003cp\u003eWe defined the Sigmoid function, which is 1/(1 + e ^ (-x)), using the Python math library to calculate \u003cem\u003ee\u003c/em\u003e as such:\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"lang-python\"\u003e\u003cspan class=\"hljs-function\"\u003e\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-title\"\u003esigmoid\u003c/span\u003e\u003cspan class=\"hljs-params\"\u003e(x)\u003c/span\u003e\u003c/span\u003e: \u003cspan class=\"hljs-keyword\"\u003ereturn\u003c/span\u003e \u003cspan class=\"hljs-number\"\u003e1.0\u003c/span\u003e / (\u003cspan class=\"hljs-number\"\u003e1.0\u003c/span\u003e + math.exp(-x))\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eFor speed, we switched to Numpy\u0026#39;s \u003ccode\u003eexp\u003c/code\u003e function instead:\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"lang-python\"\u003e\u003cspan class=\"hljs-function\"\u003e\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-title\"\u003esigmoid\u003c/span\u003e\u003cspan class=\"hljs-params\"\u003e(x)\u003c/span\u003e\u003c/span\u003e: \u003cspan class=\"hljs-keyword\"\u003ereturn\u003c/span\u003e \u003cspan class=\"hljs-number\"\u003e1.0\u003c/span\u003e / (\u003cspan class=\"hljs-number\"\u003e1.0\u003c/span\u003e + np.exp(-x))\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eThe derivative of sigmoid is simply itself multiplied by the complement to 1, e.g. f * (1 - f), and can be represented as such:\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"lang-python\"\u003e\u003cspan class=\"hljs-function\"\u003e\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-title\"\u003edsigmoid\u003c/span\u003e\u003cspan class=\"hljs-params\"\u003e(x)\u003c/span\u003e\u003c/span\u003e: \u003cspan class=\"hljs-keyword\"\u003ereturn\u003c/span\u003e sigmoid(x) * (\u003cspan class=\"hljs-number\"\u003e1\u003c/span\u003e - sigmoid(x))\n\u003c/code\u003e\u003c/pre\u003e\n\u003ch2 id=\"feeding-data\"\u003eFeeding data\u003c/h2\u003e\n\u003cp\u003eFor feeding data, we took inspiration from Tensorflow\u0026#39;s \u003ccode\u003efeed dict\u003c/code\u003e, which can be provided to a session in the format:\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"lang-python\"\u003e{\u003cspan class=\"hljs-string\"\u003e'X'\u003c/span\u003e: x, \u003cspan class=\"hljs-string\"\u003e'y'\u003c/span\u003e: y}\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eHowever, a more convenient approach is to just zip (X, y) for each training point. Thus, we defined the \u003ccode\u003efeed\u003c/code\u003e method and \u003ccode\u003ebatchify\u003c/code\u003e (to get batches) methods as follows:\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"lang-python\"\u003e\u003cspan class=\"hljs-comment\"\u003e# feed just stores the training and validation data\u003c/span\u003e\n\u003cspan class=\"hljs-function\"\u003e\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-title\"\u003efeed\u003c/span\u003e\u003cspan class=\"hljs-params\"\u003e(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e, train_data, val_data)\u003c/span\u003e\u003c/span\u003e:\n    \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.train_data = train_data\n    \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.val_data = val_data\n\n\u003cspan class=\"hljs-comment\"\u003e# Helper method to get batches\u003c/span\u003e\n\u003cspan class=\"hljs-function\"\u003e\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-title\"\u003ebatchify\u003c/span\u003e\u003cspan class=\"hljs-params\"\u003e(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e)\u003c/span\u003e\u003c/span\u003e:\n    random.shuffle(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.train_data) \u003cspan class=\"hljs-comment\"\u003e# shuffle the data around\u003c/span\u003e\n    batches = []\n    \u003cspan class=\"hljs-comment\"\u003e# we use the third parameter of range to chunk batch starting positions\u003c/span\u003e\n    \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e start \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e range(\u003cspan class=\"hljs-number\"\u003e0\u003c/span\u003e, len(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.train_data), \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.batch_size):\n        \u003cspan class=\"hljs-comment\"\u003e# append the data from start pos of length batch size\u003c/span\u003e\n        batches.append(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.train_data[\u003cspan class=\"hljs-symbol\"\u003estart:\u003c/span\u003estart + \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.batch_size])\n    console.info(\u003cspan class=\"hljs-string\"\u003e\"Created %d batches.\"\u003c/span\u003e % len(batches))\n    \u003cspan class=\"hljs-keyword\"\u003ereturn\u003c/span\u003e batches\n\u003c/code\u003e\u003c/pre\u003e\n\u003ch2 id=\"moving-forward\"\u003eMoving forward\u003c/h2\u003e\n\u003cp\u003eTo handle moving forward through the network, we first set the first layer to the input. Next, for each succeeding layer, we calculate the raw output as the dot product of the weights between the previous and current layer and the value of the previous layer, summed with the bias values for the current layer. We then activate this raw output to get the layer\u0026#39;s final value by applying sigmoid.\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"lang-python\"\u003e\u003cspan class=\"hljs-function\"\u003e\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-title\"\u003eforward\u003c/span\u003e\u003cspan class=\"hljs-params\"\u003e(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e, X)\u003c/span\u003e\u003c/span\u003e:\n    \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.activations[\u003cspan class=\"hljs-number\"\u003e0\u003c/span\u003e] = X\n    \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e layer \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e range(\u003cspan class=\"hljs-number\"\u003e1\u003c/span\u003e, len(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.sizes)):\n        \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.bias_values[layer] = (\n            \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.weights[layer].dot(\n                \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.activations[layer - \u003cspan class=\"hljs-number\"\u003e1\u003c/span\u003e]) + \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.biases[layer]\n        )\n        \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.activations[layer] = sigmoid(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.bias_values[layer])\n\u003c/code\u003e\u003c/pre\u003e\n\u003ch2 id=\"taking-a-step-back\"\u003eTaking a step back\u003c/h2\u003e\n\u003cp\u003eWe then translated the mathematical approach discussed earlier into code, using the helper functions alongside numpy\u0026#39;s excellent mathematical library to perform the necessary calculations.\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"lang-python\"\u003edef back(self, X, y, \u003cspan class=\"hljs-keyword\"\u003edB\u003c/span\u003e, dW):\n    ndB = np.array([np.zeros(bias.shape) \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e bias \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e self.biases])\n    ndW = np.array([np.zeros(weight.shape) \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e weight \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e self.weights])\n\n    \u003cspan class=\"hljs-keyword\"\u003eerr\u003c/span\u003e = (self.activations[-1] - y) * dsigmoid(self.bias_values[-1])\n    ndB[-1] = \u003cspan class=\"hljs-keyword\"\u003eerr\u003c/span\u003e\n    ndW[-1] = \u003cspan class=\"hljs-keyword\"\u003eerr\u003c/span\u003e.dot(self.activations[-2].T)\n\n    \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e \u003cspan class=\"hljs-keyword\"\u003el\u003c/span\u003e \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e \u003cspan class=\"hljs-keyword\"\u003elist\u003c/span\u003e(\u003cspan class=\"hljs-keyword\"\u003erange\u003c/span\u003e(len(self.sizes) - 1))[::-1]:\n        \u003cspan class=\"hljs-keyword\"\u003eerr\u003c/span\u003e = np.multiply(\n            self.weights[\u003cspan class=\"hljs-keyword\"\u003el\u003c/span\u003e + 1].T.dot(\u003cspan class=\"hljs-keyword\"\u003eerr\u003c/span\u003e),\n            dsigmoid(self.bias_values[\u003cspan class=\"hljs-keyword\"\u003el\u003c/span\u003e])\n        )\n        ndB[\u003cspan class=\"hljs-keyword\"\u003el\u003c/span\u003e] = \u003cspan class=\"hljs-keyword\"\u003eerr\u003c/span\u003e\n        ndW[\u003cspan class=\"hljs-keyword\"\u003el\u003c/span\u003e] = \u003cspan class=\"hljs-keyword\"\u003eerr\u003c/span\u003e.dot(self.activations[\u003cspan class=\"hljs-keyword\"\u003el\u003c/span\u003e - 1].transpose())\n    \u003cspan class=\"hljs-keyword\"\u003edB\u003c/span\u003e = \u003cspan class=\"hljs-keyword\"\u003edB\u003c/span\u003e + ndB  # \u003cspan class=\"hljs-keyword\"\u003edB\u003c/span\u003e = [nb + dnb \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e nb, dnb \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e \u003cspan class=\"hljs-keyword\"\u003ezip\u003c/span\u003e(\u003cspan class=\"hljs-keyword\"\u003edB\u003c/span\u003e, ndB)]\n    dW = dW + ndW  # dW = [nw + dnw \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e nw, dnw \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e \u003cspan class=\"hljs-keyword\"\u003ezip\u003c/span\u003e(dW, ndW)]\n    \u003cspan class=\"hljs-keyword\"\u003ereturn\u003c/span\u003e \u003cspan class=\"hljs-keyword\"\u003edB\u003c/span\u003e, dW\n\u003c/code\u003e\u003c/pre\u003e\n\u003ch2 id=\"put-it-all-together\"\u003ePut it all together\u003c/h2\u003e\n\u003cp\u003eFinally, we can combine these pieces into one \u003ccode\u003erun\u003c/code\u003e method (named after Tensorflow\u0026#39;s \u003ccode\u003esession.run\u003c/code\u003e) like so:\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"lang-python\"\u003e\u003cspan class=\"hljs-function\"\u003e\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-title\"\u003erun\u003c/span\u003e\u003cspan class=\"hljs-params\"\u003e(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e, epochs=\u003cspan class=\"hljs-number\"\u003e10\u003c/span\u003e)\u003c/span\u003e\u003c/span\u003e:\n    batches = \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.batchify()\n    \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e epoch \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e range(epochs):\n        \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e batch \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e \u003cspan class=\"hljs-symbol\"\u003ebatches:\u003c/span\u003e\n            \u003cspan class=\"hljs-comment\"\u003e#####\u003c/span\u003e\n            \u003cspan class=\"hljs-comment\"\u003e# We now perform gradient descent\u003c/span\u003e\n            dB = np.array([np.zeros(bias.shape) \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e bias \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.biases])\n            dW = np.array([np.zeros(weight.shape)\n                            \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e weight \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.weights])\n            \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e X, y \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e \u003cspan class=\"hljs-symbol\"\u003ebatch:\u003c/span\u003e\n                \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.forward(X)\n                dB, dW = \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.back(X, y, dB, dW)\n            \u003cspan class=\"hljs-comment\"\u003e# Adjust the weights\u003c/span\u003e\n            \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.weights = np.array([\n                w - (\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.learning_rate / \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.batch_size) * dw\n                \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e w, dw \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e zip(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.weights, dW)\n            ])\n            \u003cspan class=\"hljs-comment\"\u003e# Adjust the biases\u003c/span\u003e\n            \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.biases = np.array([\n                b - (\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.learning_rate / \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.batch_size) * db\n                \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e b, db \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e zip(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.biases, dB)\n            ])\n            \u003cspan class=\"hljs-comment\"\u003e#####\u003c/span\u003e\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eWe can decorate this method to be more resourceful in several ways. Using TQDM, we can provide a progress bar for the batches similar to how Keras has it implemented in \u003ccode\u003emodel.fit\u003c/code\u003e, and we can also log the current epoch (further using \u003ccode\u003eself.epochs\u003c/code\u003e to keep count of how many epochs the model has been trained for). We also keep track of time, and cross-validate if validation data is present (this is again similar to Keras\u0026#39;s \u003ccode\u003emodel.fit\u003c/code\u003e, which uses the \u003ccode\u003ecross_validate\u003c/code\u003e parameter to validate each epoch).\u003c/p\u003e\n\u003cp\u003eThis brings us to the decorated \u003ccode\u003erun\u003c/code\u003e method:\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"lang-python\"\u003edef run(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e, epochs=\u003cspan class=\"hljs-number\"\u003e10\u003c/span\u003e):\n    batches = \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.batchify()\n    \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e epoch \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e range(epochs):\n        t0 = t()\n        \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e batch \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e tqdm(batches):\n            dB = np.array([np.zeros(bias.shape) \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e bias \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.biases])\n            dW = np.array([np.zeros(weight.shape)\n                            \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e weight \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.weights])\n            \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e \u003cspan class=\"hljs-type\"\u003eX\u003c/span\u003e, y \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e batch:\n                \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.forward(\u003cspan class=\"hljs-type\"\u003eX\u003c/span\u003e)\n                dB, dW = \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.back(\u003cspan class=\"hljs-type\"\u003eX\u003c/span\u003e, y, dB, dW)\n            \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.weights = np.array([\n                w - (\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.learning_rate / \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.batch_size) * dw\n                \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e w, dw \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e \u003cspan class=\"hljs-built_in\"\u003ezip\u003c/span\u003e(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.weights, dW)\n            ])\n            \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.biases = np.array([\n                b - (\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.learning_rate / \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.batch_size) * db\n                \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e b, db \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e \u003cspan class=\"hljs-built_in\"\u003ezip\u003c/span\u003e(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.biases, dB)\n            ])\n        console.log(\u003cspan class=\"hljs-string\"\u003e\"Processed %d batches in %.02f seconds.\"\u003c/span\u003e %\n                    (len(batches), t() - t0))\n        \u003cspan class=\"hljs-keyword\"\u003eif\u003c/span\u003e \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.val_data \u003cspan class=\"hljs-keyword\"\u003eis\u003c/span\u003e not \u003cspan class=\"hljs-type\"\u003eNone\u003c/span\u003e:  # cannot use \u003cspan class=\"hljs-keyword\"\u003eif\u003c/span\u003e \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.val_data bc numpy\n            console.info(\n                \u003cspan class=\"hljs-string\"\u003e\"Accuracy: %02f\"\u003c/span\u003e %\n                (\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.validate(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.val_data) / \u003cspan class=\"hljs-number\"\u003e100.0\u003c/span\u003e)\n            )\n        console.success(\u003cspan class=\"hljs-string\"\u003e\"Processed epoch %d\"\u003c/span\u003e % epoch)\n        \u003cspan class=\"hljs-built_in\"\u003eprint\u003c/span\u003e(\u003cspan class=\"hljs-string\"\u003e\"Processed epoch {0}.\"\u003c/span\u003e.format(epoch))\n        \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.epochs += \u003cspan class=\"hljs-number\"\u003e1\u003c/span\u003e\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eNote the use of \u003ccode\u003eself.validate\u003c/code\u003e, which is a simple method inspired by Michael Nielsen\u0026#39;s take on classification problems; it uses Python truthiness alongside list comprehension to simply count the number of correct predictions. \u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e\u003cspan class=\"hljs-function\"\u003e\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-title\"\u003evalidate\u003c/span\u003e\u003cspan class=\"hljs-params\"\u003e(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e, val_data)\u003c/span\u003e\u003c/span\u003e:\n    \u003cspan class=\"hljs-keyword\"\u003ereturn\u003c/span\u003e sum(\n        [(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.predict(X)[\u003cspan class=\"hljs-number\"\u003e0\u003c/span\u003e] == y) \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e X, y \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e val_data])\n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eNote here the use of \u003ccode\u003eself.predict\u003c/code\u003e, which is inspired by Keras\u0026#39;s \u003ccode\u003epredict\u003c/code\u003e function. We simply move forward with an input, and then return the final activation layer.\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"lang-python\"\u003e\u003cspan class=\"hljs-function\"\u003e\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-title\"\u003epredict\u003c/span\u003e\u003cspan class=\"hljs-params\"\u003e(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e, X)\u003c/span\u003e\u003c/span\u003e:\n    \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.forward(X)\n    \u003cspan class=\"hljs-keyword\"\u003ereturn\u003c/span\u003e \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.activations[-\u003cspan class=\"hljs-number\"\u003e1\u003c/span\u003e]\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eSince we need to know the label with highest confidence, we can apply numpy\u0026#39;s \u003ccode\u003eargmax\u003c/code\u003e function to find the index (which is also the label since our labels are 0-9 and indices are 0-9 --\u0026gt; our labels are our indices) of the highest value.\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"lang-python\"\u003e\u003cspan class=\"hljs-function\"\u003e\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-title\"\u003epredict\u003c/span\u003e\u003cspan class=\"hljs-params\"\u003e(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e, X)\u003c/span\u003e\u003c/span\u003e:\n    \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.forward(X)\n    yhat = \u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e.activations[-\u003cspan class=\"hljs-number\"\u003e1\u003c/span\u003e]\n    \u003cspan class=\"hljs-keyword\"\u003ereturn\u003c/span\u003e np.argmax(yhat), yhat\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eAnother improvement is to initialize random weights instead of zero-ed weights. This should bring us closer to the minima. To achieve this, we can use numpy\u0026#39;s \u003ccode\u003erandn\u003c/code\u003e function with the shape of the weights (each node in the next layer must have a weight set, and there must be a weight in the weight set for each node in the previous layer).\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"lang-python\"\u003eself.weights = \u003cspan class=\"hljs-built_in\"\u003enp\u003c/span\u003e.\u003cspan class=\"hljs-built_in\"\u003earray\u003c/span\u003e([\u003cspan class=\"hljs-built_in\"\u003enp\u003c/span\u003e.zeros(\u003cspan class=\"hljs-number\"\u003e1\u003c/span\u003e)] + [\n            \u003cspan class=\"hljs-built_in\"\u003enp\u003c/span\u003e.\u003cspan class=\"hljs-built_in\"\u003erandom\u003c/span\u003e.randn(next_layer_size, previous_layer_size)\n            \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e next_layer_size, previous_layer_size\n            \u003cspan class=\"hljs-keyword\"\u003ein\u003c/span\u003e zip(sizes[\u003cspan class=\"hljs-number\"\u003e1\u003c/span\u003e:], sizes[:-\u003cspan class=\"hljs-number\"\u003e1\u003c/span\u003e])\n        ])\n\u003c/code\u003e\u003c/pre\u003e\n\u003ch2 id=\"checkpoints\"\u003eCheckpoints\u003c/h2\u003e\n\u003cp\u003eInstead of implementing Tensorflow\u0026#39;s \u003cem\u003every\u003c/em\u003e bulky checkpointer, we opted for Keras\u0026#39;s very lightweight \u003ccode\u003esave\u003c/code\u003e and \u003ccode\u003eload\u003c/code\u003e methods. As discussed in the Inspirations section, we use NPZ archives with inspiration from Michael Nielsen.\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"lang-python\"\u003edef \u003cspan class=\"hljs-keyword\"\u003eload\u003c/span\u003e(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e, \u003cspan class=\"hljs-keyword\"\u003efile\u003c/span\u003e=\u003cspan class=\"hljs-string\"\u003e'model.npz'\u003c/span\u003e):\n    \u003cspan class=\"hljs-keyword\"\u003emodel\u003c/span\u003e = np.load(\u003cspan class=\"hljs-string\"\u003e'./models/%s'\u003c/span\u003e % \u003cspan class=\"hljs-keyword\"\u003efile\u003c/span\u003e)\n    self.epochs = \u003cspan class=\"hljs-built_in\"\u003eint\u003c/span\u003e(\u003cspan class=\"hljs-keyword\"\u003emodel\u003c/span\u003e[\u003cspan class=\"hljs-string\"\u003e'epochs'\u003c/span\u003e])\n    self.learning_rate = \u003cspan class=\"hljs-built_in\"\u003efloat\u003c/span\u003e(\u003cspan class=\"hljs-keyword\"\u003emodel\u003c/span\u003e[\u003cspan class=\"hljs-string\"\u003e'learning_rate'\u003c/span\u003e])\n    self.weights = np.array(\u003cspan class=\"hljs-keyword\"\u003emodel\u003c/span\u003e[\u003cspan class=\"hljs-string\"\u003e'weights'\u003c/span\u003e])\n    self.biases = np.array(\u003cspan class=\"hljs-keyword\"\u003emodel\u003c/span\u003e[\u003cspan class=\"hljs-string\"\u003e'biases'\u003c/span\u003e])\n    self.batch_size = \u003cspan class=\"hljs-built_in\"\u003eint\u003c/span\u003e(\u003cspan class=\"hljs-keyword\"\u003emodel\u003c/span\u003e[\u003cspan class=\"hljs-string\"\u003e'batch_size'\u003c/span\u003e])\n\n\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-keyword\"\u003esave\u003c/span\u003e(\u003cspan class=\"hljs-keyword\"\u003eself\u003c/span\u003e, \u003cspan class=\"hljs-keyword\"\u003efile\u003c/span\u003e=\u003cspan class=\"hljs-string\"\u003e'model.npz'\u003c/span\u003e):\n    np.savez_compressed(\n        \u003cspan class=\"hljs-keyword\"\u003efile\u003c/span\u003e=\u003cspan class=\"hljs-string\"\u003e'./models/%s'\u003c/span\u003e % \u003cspan class=\"hljs-keyword\"\u003efile\u003c/span\u003e,\n        epochs=self.epochs,\n        learning_rate=self.learning_rate,\n        weights=self.weights,\n        biases=self.biases,\n        batch_size=self.batch_size\n    )\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eFor the API, we pretrained models from less than 50% to more than 90% accuracy.\u003c/p\u003e\n\u003cp\u003eThe entire module is put under the \u0026quot;Graph\u0026quot; class in \u003ccode\u003ebetter_tensorflow.py\u003c/code\u003e. An example would be as follows:\u003c/p\u003e\n\u003cpre\u003e\u003ccode class=\"lang-python\"\u003eimport better_tensorflow as btf\nnet = btf.Graph((\u003cspan class=\"hljs-number\"\u003e784\u003c/span\u003e, \u003cspan class=\"hljs-number\"\u003e30\u003c/span\u003e, \u003cspan class=\"hljs-number\"\u003e10\u003c/span\u003e), \u003cspan class=\"hljs-number\"\u003e0.004\u003c/span\u003e, \u003cspan class=\"hljs-number\"\u003e128\u003c/span\u003e)\n\u003c/code\u003e\u003c/pre\u003e\n\u003ch1 id=\"the-api\"\u003eThe API\u003c/h1\u003e\n\u003cp\u003eThe API still requires the \u003ccode\u003edata\u003c/code\u003e folder with the MNIST pickle file, which you can find under \u003ca href=\"https://github.com/pshah123/better-tensorflow/releases\"\u003eReleases on GitHub\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eTo build the API server, we needed to make sure I/O and other operations were non-blocking so that the training portion (which is an enormous synchronous call) is the only synchronous portion. To this end, Sanic was used. Sanic is an asyncio implementation of a web server in Python 3, and is extremely similar (if not identical) to Flask. It is regardedly much more stable than Flask, although Flask has more features. While deploying a Flask app requires infrastructure to support uptime such as a Node-based task queue and gunicorn, Sanic\u0026#39;s use of async means it maintains uptime regardless of failure (it handles failure with a 500), and cutoff threads/timeouts are killed separately to the main process. Core metrics in the past have shown Sanic remains reliable for weeks under heavy load, even when used with AI models running on GPU threads. To this end, we believe Sanic is the perfect choice to deploy an API.\u003c/p\u003e\n\u003cp\u003eThe basic Sanic setup requires route decorators. To avoid reiterating a basic tutorial of Sanic, we will not go into the specifics of setup, and instead discuss higher-level implementations. We use a CORS handler from the \u003ccode\u003esanic_cors\u003c/code\u003e package to setup cross origin headers, which sits as a middleware. Extra methods are used to handle preflight requests so as not to clutter the main endpoints.\u003c/p\u003e\n\u003cp\u003eFurthermore, Sanic has special streaming responses that must be implemented; in this scenario, sanic_json and text are used as the main responses, and the \u003ccode\u003ejson\u003c/code\u003e module is independently used so as to properly hash the NumPy floats (which are float32 or float64, not Python floats).\u003c/p\u003e\n\u003cp\u003eLogistically, the system-wide representation of the current file/module is referred to as \u003ccode\u003e_this\u003c/code\u003e (\u003ccode\u003e_this = sys.modules[__name__]\u003c/code\u003e); this is used due to the async nature of Sanic. To provide all threads with the same variables, we store variables in the module under \u003ccode\u003e_this\u003c/code\u003e (ie. \u003ccode\u003e_this.net = btf.Graph(...)\u003c/code\u003e), we do not implement locking, \u003ccode\u003emultiprocessing.Variable\u003c/code\u003es, or data stores due to the simple fact that the only modifying operation is synchronous and thread-locking. As the API is being hosted on a DoC server, we also contacted CSG to open up an obfuscated port which is stored in an environmental variable (and not shown here).\u003c/p\u003e\n\u003cp\u003eThe Sanic app layout is as follows:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eimport os\nfrom sanic import Sanic\nfrom sanic.response import text, json as sanic_json\nimport json\nfrom sanic_cors import CORS, cross_origin\napp = Sanic()\nCORS(app)\n\nimport random\n\n\n@app.route(\u003cspan class=\"hljs-string\"\u003e'/predict'\u003c/span\u003e, methods=[\u003cspan class=\"hljs-string\"\u003e'OPTIONS'\u003c/span\u003e])\nasync \u003cspan class=\"hljs-function\"\u003e\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-title\"\u003epreflight\u003c/span\u003e\u003cspan class=\"hljs-params\"\u003e(q)\u003c/span\u003e\u003c/span\u003e:\n    \u003cspan class=\"hljs-keyword\"\u003ereturn\u003c/span\u003e text(\u003cspan class=\"hljs-string\"\u003e'OK'\u003c/span\u003e, status=\u003cspan class=\"hljs-number\"\u003e200\u003c/span\u003e)\n\n\n@app.route(\u003cspan class=\"hljs-string\"\u003e'/reset'\u003c/span\u003e, methods=[\u003cspan class=\"hljs-string\"\u003e'GET'\u003c/span\u003e, \u003cspan class=\"hljs-string\"\u003e'POST'\u003c/span\u003e, \u003cspan class=\"hljs-string\"\u003e'OPTIONS'\u003c/span\u003e])\nasync \u003cspan class=\"hljs-function\"\u003e\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-title\"\u003ereset\u003c/span\u003e\u003cspan class=\"hljs-params\"\u003e(q)\u003c/span\u003e\u003c/span\u003e:\n    _this.net = btf.Graph([\u003cspan class=\"hljs-number\"\u003e784\u003c/span\u003e, \u003cspan class=\"hljs-number\"\u003e30\u003c/span\u003e, \u003cspan class=\"hljs-number\"\u003e10\u003c/span\u003e], \u003cspan class=\"hljs-number\"\u003e0\u003c/span\u003e.\u003cspan class=\"hljs-number\"\u003e5\u003c/span\u003e)\n    \u003cspan class=\"hljs-keyword\"\u003ereturn\u003c/span\u003e text(\u003cspan class=\"hljs-string\"\u003e'OK'\u003c/span\u003e, status=\u003cspan class=\"hljs-number\"\u003e200\u003c/span\u003e)\n\n\n@app.route(\u003cspan class=\"hljs-string\"\u003e'/pretrain'\u003c/span\u003e, methods=[\u003cspan class=\"hljs-string\"\u003e'GET'\u003c/span\u003e, \u003cspan class=\"hljs-string\"\u003e'POST'\u003c/span\u003e, \u003cspan class=\"hljs-string\"\u003e'OPTIONS'\u003c/span\u003e])\nasync \u003cspan class=\"hljs-function\"\u003e\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-title\"\u003epretrain\u003c/span\u003e\u003cspan class=\"hljs-params\"\u003e(q)\u003c/span\u003e\u003c/span\u003e:\n    _this.net = _this.pretrained\n    \u003cspan class=\"hljs-keyword\"\u003ereturn\u003c/span\u003e text(\u003cspan class=\"hljs-string\"\u003e'OK'\u003c/span\u003e, status=\u003cspan class=\"hljs-number\"\u003e200\u003c/span\u003e)\n\n\n@app.route(\u003cspan class=\"hljs-string\"\u003e'/train'\u003c/span\u003e, methods=[\u003cspan class=\"hljs-string\"\u003e'OPTIONS'\u003c/span\u003e])\nasync \u003cspan class=\"hljs-function\"\u003e\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-title\"\u003epreflight2\u003c/span\u003e\u003cspan class=\"hljs-params\"\u003e(q)\u003c/span\u003e\u003c/span\u003e:\n    \u003cspan class=\"hljs-keyword\"\u003ereturn\u003c/span\u003e text(\u003cspan class=\"hljs-string\"\u003e'OK'\u003c/span\u003e, status=\u003cspan class=\"hljs-number\"\u003e200\u003c/span\u003e)\n\n\n@app.route(\u003cspan class=\"hljs-string\"\u003e'/train'\u003c/span\u003e, methods=[\u003cspan class=\"hljs-string\"\u003e'POST'\u003c/span\u003e, \u003cspan class=\"hljs-string\"\u003e'GET'\u003c/span\u003e])\nasync \u003cspan class=\"hljs-function\"\u003e\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-title\"\u003etrain\u003c/span\u003e\u003cspan class=\"hljs-params\"\u003e(q)\u003c/span\u003e\u003c/span\u003e:\n    epochs = q.json.get(\u003cspan class=\"hljs-string\"\u003e'epochs'\u003c/span\u003e) \u003cspan class=\"hljs-keyword\"\u003eif\u003c/span\u003e q.json \u003cspan class=\"hljs-keyword\"\u003eelse\u003c/span\u003e \u003cspan class=\"hljs-number\"\u003e1\u003c/span\u003e\n    random.shuffle(_this.data)\n    _this.net.feed(_this.data, None)\n    _this.net.run(epochs)\n    random.shuffle(_this.val_data)\n    acc = _this.net.validate(_this.val_data[\u003cspan class=\"hljs-symbol\"\u003e:\u003c/span\u003e\u003cspan class=\"hljs-number\"\u003e100\u003c/span\u003e])\n    d = {\n        \u003cspan class=\"hljs-string\"\u003e'accuracy'\u003c/span\u003e: int(acc)\n    }\n    \u003cspan class=\"hljs-keyword\"\u003ereturn\u003c/span\u003e sanic_json(d, dumps=json.dumps)\n\n\n@app.route(\u003cspan class=\"hljs-string\"\u003e'/predict'\u003c/span\u003e, methods=[\u003cspan class=\"hljs-string\"\u003e'POST'\u003c/span\u003e])\nasync \u003cspan class=\"hljs-function\"\u003e\u003cspan class=\"hljs-keyword\"\u003edef\u003c/span\u003e \u003cspan class=\"hljs-title\"\u003epredict\u003c/span\u003e\u003cspan class=\"hljs-params\"\u003e(q)\u003c/span\u003e\u003c/span\u003e:\n    ...\n\n\u003cspan class=\"hljs-keyword\"\u003eif\u003c/span\u003e __name_\u003cspan class=\"hljs-number\"\u003e_\u003c/span\u003e == \u003cspan class=\"hljs-string\"\u003e'__main__'\u003c/span\u003e:\n    app.run(host=\u003cspan class=\"hljs-string\"\u003e'0.0.0.0'\u003c/span\u003e, port=os.getenv(\u003cspan class=\"hljs-string\"\u003e'DOC_PORT'\u003c/span\u003e))\n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eNote that the \u003ccode\u003etrain\u003c/code\u003e endpoint supports GET, and simply trains, validates, and returns accuracy, so it can be called from a browser as well.\u003c/p\u003e\n\u003cp\u003eThe data loading is not shown as it was discussed earlier in the Preprocessing section. The predict method is omitted and is instead shown below:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003e@app.route(\u003cspan class=\"hljs-string\"\u003e'/predict'\u003c/span\u003e, methods=[\u003cspan class=\"hljs-string\"\u003e'POST'\u003c/span\u003e])\nasync def predict(q):\n    imageURI = base64.urlsafe_b64decode(\n        re.sub(\u003cspan class=\"hljs-string\"\u003e'^data:image/.+;base64,'\u003c/span\u003e, \u003cspan class=\"hljs-string\"\u003e''\u003c/span\u003e, q.json.\u003cspan class=\"hljs-built_in\"\u003eget\u003c/span\u003e(\u003cspan class=\"hljs-string\"\u003e'image'\u003c/span\u003e)))\n    \u003cspan class=\"hljs-built_in\"\u003eimage\u003c/span\u003e = cv2.imdecode(np.fromstring(imageURI, dtype=np.uint8), \u003cspan class=\"hljs-number\"\u003e0\u003c/span\u003e)\n    \u003cspan class=\"hljs-built_in\"\u003eimage\u003c/span\u003e = \u003cspan class=\"hljs-built_in\"\u003eimage\u003c/span\u003e / \u003cspan class=\"hljs-number\"\u003e255\u003c/span\u003e\n    \u003cspan class=\"hljs-built_in\"\u003eprint\u003c/span\u003e(\u003cspan class=\"hljs-built_in\"\u003eimage\u003c/span\u003e.\u003cspan class=\"hljs-built_in\"\u003eshape\u003c/span\u003e)\n    _in = np.reshape(\u003cspan class=\"hljs-built_in\"\u003eimage\u003c/span\u003e, (\u003cspan class=\"hljs-number\"\u003e784\u003c/span\u003e, \u003cspan class=\"hljs-number\"\u003e1\u003c/span\u003e))\n    \u003cspan class=\"hljs-keyword\"\u003etry\u003c/span\u003e:\n        \u003cspan class=\"hljs-keyword\"\u003eif\u003c/span\u003e q.json.\u003cspan class=\"hljs-built_in\"\u003eget\u003c/span\u003e(\u003cspan class=\"hljs-string\"\u003e'pretrained'\u003c/span\u003e) \u0026gt; \u003cspan class=\"hljs-number\"\u003e0\u003c/span\u003e:\n            console.\u003cspan class=\"hljs-built_in\"\u003elog\u003c/span\u003e(\u003cspan class=\"hljs-string\"\u003e\"Using model-v%d.npz\"\u003c/span\u003e % q.json.\u003cspan class=\"hljs-built_in\"\u003eget\u003c/span\u003e(\u003cspan class=\"hljs-string\"\u003e'pretrained'\u003c/span\u003e))\n            _this.pretrained.load(file=\u003cspan class=\"hljs-string\"\u003e'model-v%d.npz'\u003c/span\u003e %\n                                  q.json.\u003cspan class=\"hljs-built_in\"\u003eget\u003c/span\u003e(\u003cspan class=\"hljs-string\"\u003e'pretrained'\u003c/span\u003e))\n            prediction, predictions = _this.pretrained.predict(_in)\n    except Exception as e:\n        prediction, predictions = _this.net.predict(_in)\n    d = {\n        \u003cspan class=\"hljs-string\"\u003e'label'\u003c/span\u003e: \u003cspan class=\"hljs-built_in\"\u003eint\u003c/span\u003e(prediction),\n        \u003cspan class=\"hljs-string\"\u003e'predictions'\u003c/span\u003e: [\n            {\u003cspan class=\"hljs-string\"\u003e'label'\u003c/span\u003e: \u003cspan class=\"hljs-built_in\"\u003eint\u003c/span\u003e(i), \u003cspan class=\"hljs-string\"\u003e'confidence'\u003c/span\u003e: \u003cspan class=\"hljs-built_in\"\u003efloat\u003c/span\u003e(\u003cspan class=\"hljs-string\"\u003e'%02f'\u003c/span\u003e % c[\u003cspan class=\"hljs-number\"\u003e0\u003c/span\u003e])}\n            \u003cspan class=\"hljs-keyword\"\u003efor\u003c/span\u003e i, c in enumerate(predictions)\n        ]\n    }\n    j = json.dumps(d)\n    \u003cspan class=\"hljs-built_in\"\u003eprint\u003c/span\u003e(prediction)\n    \u003cspan class=\"hljs-built_in\"\u003eprint\u003c/span\u003e(j)\n    \u003cspan class=\"hljs-built_in\"\u003eprint\u003c/span\u003e(d)\n    \u003cspan class=\"hljs-keyword\"\u003ereturn\u003c/span\u003e sanic_json(d, dumps=json.dumps)\n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eWe read the data as base64, as this is a convenient and efficient format for moving images without storing them. Using the Python OpenCV2 module which is standard for image pipelines, we then decode the base64 string and read it as grayscale (the \u003ccode\u003e0\u003c/code\u003e in \u003ccode\u003eimdecode\u003c/code\u003e is the flag for grayscale). Normalization and reshaping as discussed in preprocessing is performed, and if the \u003ccode\u003epretrained\u003c/code\u003e key is present then a pretrained model is selected according to specification. Then, a prediction is performed. We round the floats using string templating and then pass the result through a float constructor; although this seems \u0026quot;hacky\u0026quot;, the float constructor is necessary so that Sanic can respond, as \u003ccode\u003efloat32\u003c/code\u003e and \u003ccode\u003efloat64\u003c/code\u003e (NumPy\u0026#39;s defaults) are not hashable in JSON. We also use the int constructor as NumPy likely used uint8 to store the label.\u003c/p\u003e\n\u003cp\u003eAs such, the endpoints are then available to be referenced by a web application.\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcuuupid%2Fbetter-tensorflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcuuupid%2Fbetter-tensorflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcuuupid%2Fbetter-tensorflow/lists"}