{"id":16423224,"url":"https://github.com/hailiang-wang/nnetcpp","last_synced_at":"2026-06-14T16:33:00.716Z","repository":{"id":72218544,"uuid":"108720965","full_name":"hailiang-wang/nnetcpp","owner":"hailiang-wang","description":"Simple, Fast and Powerful RNN for textsum","archived":false,"fork":false,"pushed_at":"2017-10-29T12:11:42.000Z","size":82,"stargazers_count":1,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-03T23:05:04.258Z","etag":null,"topics":["cpp","deep-learning","lstm","rnn"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hailiang-wang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-10-29T09:27:59.000Z","updated_at":"2022-03-26T11:49:45.000Z","dependencies_parsed_at":"2023-05-19T11:30:36.955Z","dependency_job_id":null,"html_url":"https://github.com/hailiang-wang/nnetcpp","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/hailiang-wang/nnetcpp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hailiang-wang%2Fnnetcpp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hailiang-wang%2Fnnetcpp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hailiang-wang%2Fnnetcpp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hailiang-wang%2Fnnetcpp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hailiang-wang","download_url":"https://codeload.github.com/hailiang-wang/nnetcpp/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hailiang-wang%2Fnnetcpp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34327690,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-14T02:00:07.365Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","deep-learning","lstm","rnn"],"created_at":"2024-10-11T07:39:13.787Z","updated_at":"2026-06-14T16:33:00.680Z","avatar_url":"https://github.com/hailiang-wang.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# [Neural networks in C++](https://github.com/steckdenis/nnetcpp)\n\nThis library provides components used for building advanced neural networks. It is built on Eigen (for speed and ease of development) and tries to be as flexible as possible. Some of its main features are :\n\n* Very flexible architecture: neural networks consist of a graph of nodes (cycles allowed), which allows an easy implementation of feed-forward deep networks, recurrent networks, autoencoders, complex networks that do several things in parallel and then merge results, etc.\n* Built on Eigen, known for its speed\n* Time-series can be trained in an efficient and incremental fashion (you can predict t(0)...t(n), then train t(n), then predict t(n+1) without having to re-predict t(0)...t(n), then train t(n+1), etc)\n* Fully unit-tested\n\n# Simple perceptron\n\nThe library consists of a `Network` class, that manages training and prediction, and several `AbstractNode` subclasses. Examples of usages of these classes can be found in the `tests/` directory, but here are some more detailed examples.\n\nCreating a neural network starts by instantiating a `Network` object, that will keep track of nodes :\n\n```cpp\nNetwork *net = new Network(num_inputs);\n```\n\nNow that the `Network` object has been created, `AbstractNode` subclasses can be instantiated. The most important subclasses are `Dense` (a fully-connected weighted dense layer, that learns its weights) and the different kinds of `Activation` nodes (`SigmoidActivation`, `TanhActivation`).\n\n```cpp\nDense *layer1 = new Dense(num_hidden, 0.005);\nSigmoidActivation *layer1_act = new SigmoidActivation;\nDense *layer2 = new Dense(num_outputs, 0.005);\nSigmoidActivation *layer2_act = new SigmoidActivation;\n```\n\nThe above code snippet creates the nodes required for the implementation of a single-hidden-layer feed-forward perceptron. The first dense node connects the `num_inputs` inputs of the network to the `num_hidden` neurons of the hidden layer. The second dense node connects the `num_hidden` neurons of the hidden layer to the `num_outputs` output neurons. The constructor of `Dense` takes as parameter the number of output neurons of the dense node. The number of input neurons will be inferred when the nodes are connected to each other (when `Dense` knows from what to take its input).\n\nThose two dense layers don't have any activation function by themselves (they use a linear activation), so two `SigmoidActivation` nodes will be used in order to add activations to them.\n\nNow that the nodes are created, they can be wired together. Each `AbstractNode` subclass exposes an *output port* (producing values and consuming error signals), and can have one or several input ports. In this simple example, all the nodes used have only one input port.\n\n```cpp\nlayer1-\u003esetInput(net-\u003einputPort());\nlayer1_act-\u003esetInput(layer1-\u003eoutput());\nlayer2-\u003esetInput(layer1_act-\u003eoutput());\nlayer2_act-\u003esetInput(layer2-\u003eoutput());\n```\n\nThe last step, that is simple when building feed-forward neural networks but can become tricky when building recurrent networks, consists of adding the layers to the network. The layers must be added in the order in which they have to forward their values. In this simple example, we will add `layer1`, then `layer1_act`, then `layer2` and finally `layer2_act`. If there are loops in the network graph, then an order has to be decided. It is usually the breadth-first order that is used.\n\n```cpp\nnet-\u003eaddNode(layer1);\nnet-\u003eaddNode(layer1_act);\nnet-\u003eaddNode(layer2);\nnet-\u003eaddNode(layer2_act);\n```\n\nThe first node added to `Network` is the input (it receives input vectors). The last one is the output and produces the value returned by `Network::predict()`.\n\nThis neural network is now complete and can be trained using `Network::train` (performing a single gradient update on a single example). The `tests/utils.h` file contains some utility functions that can be used in order to train a network on a batch of input/output samples.\n\n# Merge nodes\n\nMerge nodes are special types of nodes that take as many inputs as one wants, and merge them. Merging can be done either by adding the inputs (component-wise adding, so the first output neuron is the sum of the first neuron of all the inputs), or by multiplying them. Those nodes can be used to implement network that have gates: the output of a `Dense` node is multiplied by another one, that serves as a gate, which allows to design gated recurrent networks that can learn to forget, copy, or anything else. Those merge nodes are used internally by the GRU node.\n\nThe following snippet shows to to create a network that learns a function like `(ax + b)*(cx + d)`, with the a, b, c and d parameters learned by the `Dense` nodes.\n\n```cpp\nNetwork *net = new Network(1);\nDense *dense1 = new Dense(1, 0.05);\nDense *dense2 = new Dense(1, 0.05);\nMergeSum *product = new MergeProduct;\n\ndense1-\u003esetInput(net-\u003einputPort());\ndense2-\u003esetInput(net-\u003einputPort());\nproduct-\u003eaddInput(dense1-\u003eoutput());\nproduct-\u003eaddInput(dense2-\u003eoutput());\n\nnet-\u003eaddNode(dense1);\nnet-\u003eaddNode(dense2);\nnet-\u003eaddNode(product);\n```\n\n# Recurrent networks\n\nUsing recurrend nodes (`GRU` for instance, which works a bit like the famous `LSTM` networks but are sometimes a bit more efficient and stable) is easy. They are added to a network exactly like other nodes.\n\nBuilding recurrent networks piece by piece, so by assembling `Dense`, `Activation` and merge nodes, is a bit more complicated. The topology of the network first has to be designed (which can be done by looking at formulas or graphs explaining how the recurrent network works), then all the nodes have to be instantiated and wired in the correct way.\n\nOnce the wiring is correct, they have to be added to a `Network` in the right order. The main principle is that, during the forward pass, no node can be forwarded until all its dependencies have been forwarded. If all your nodes depend on `h(t-1)`, then ensure that you set the output of the node reprenting `h` to zero before starting to train a sequence, and add the networks so that the node (usually a `Dense`) connecting `h` to the input is forwarded first. The other nodes are usually easier to connect. Here is how the GRU unit has been connected :\n\n```cpp\n// Wire h(t-1) to what depends on it (but not on other things)\n_nodes.push_back(loop_output_to_updates);\n_nodes.push_back(loop_output_to_resets);\n\n// resets merges the h(t-1) (wired in the above step) and user-specified reset signals\n_nodes.push_back(resets);\n_nodes.push_back(reset_activation);\n_nodes.push_back(reset_times_output);  // depends on h(t-1) and reset_activation, okay to add here\n\n// Now that information can flow from resets to its activation to reset_times_output, reset_times_output can be used\n_nodes.push_back(loop_reset_times_output_to_inputs);\n\n// inputs depend on loop_reset_times_output_to_inputs, which has now been added to the network, and user-specific inputs.\n_nodes.push_back(inputs);\n_nodes.push_back(input_activation);\n\n// updates depends on loop_output_to_updates (h(t-1)) and user-specific updates.\n_nodes.push_back(updates);\n_nodes.push_back(update_activation);\n_nodes.push_back(oneminus_update_activation);\n_nodes.push_back(update_times_output);\n_nodes.push_back(oneminus_update_times_input);  // This node also depends on input, hence its addition near the end of this code snippet.\n\n// Now that everything has been computed using h(t-1), the new values can be forwarded to the output, h(t)\n_nodes.push_back(output);\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhailiang-wang%2Fnnetcpp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhailiang-wang%2Fnnetcpp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhailiang-wang%2Fnnetcpp/lists"}