{"id":13630230,"url":"https://github.com/bytedance/ns-x","last_synced_at":"2025-08-21T00:30:58.086Z","repository":{"id":45564639,"uuid":"395577706","full_name":"bytedance/ns-x","owner":"bytedance","description":"An easy-to-use, flexible network simulator library in Go.","archived":false,"fork":false,"pushed_at":"2024-01-22T12:04:46.000Z","size":253,"stargazers_count":141,"open_issues_count":0,"forks_count":3,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-12-10T07:43:42.926Z","etag":null,"topics":["network","ns-2","ns-3","simulation","simulator"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bytedance.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2021-08-13T08:38:14.000Z","updated_at":"2024-10-31T07:38:28.000Z","dependencies_parsed_at":"2024-02-25T03:33:54.601Z","dependency_job_id":null,"html_url":"https://github.com/bytedance/ns-x","commit_stats":{"total_commits":272,"total_committers":7,"mean_commits":"38.857142857142854","dds":"0.14338235294117652","last_synced_commit":"edcf0e3710cf3db6e7c1b98a98994b60ba9ebe71"},"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bytedance%2Fns-x","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bytedance%2Fns-x/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bytedance%2Fns-x/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bytedance%2Fns-x/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bytedance","download_url":"https://codeload.github.com/bytedance/ns-x/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230471175,"owners_count":18231193,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["network","ns-2","ns-3","simulation","simulator"],"created_at":"2024-08-01T22:01:34.825Z","updated_at":"2024-12-19T17:08:57.865Z","avatar_url":"https://github.com/bytedance.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# ns-x\n\n[![Go](https://github.com/bytedance/ns-x/actions/workflows/go.yml/badge.svg)](https://github.com/bytedance/ns-x/actions/workflows/go.yml)\n[![CodeQL](https://github.com/bytedance/ns-x/actions/workflows/codeql.yml/badge.svg)](https://github.com/bytedance/ns-x/actions/workflows/codeql.yml)\n\nAn easy-to-use, flexible **network simulator** library for Go.\n\n## Feature\n\n* Programmatically build customizable and scalable network topology from basic nodes.\n* Simulate packet loss, delay, etc. on any nodes, according to any parameters inputs in well-defined models.\n* Collect network and user-defined data in detail, from each and every node.\n* Cross-platform. High precision.\n\n## Introduction\n\n#### Concept\n\n* **Network**: a topological graph consist of **node**s, reflecting a real-world network for **packet** to transfer and route through.\n* **Node**: a physical or logical device in the **network** deciding what to do when a **packet** going through. A **node** usually **connect** to other **node**s.\n* **Event**: an action to be done at a given time point.\n* **Packet**: simulated data **packet**s **transfer**ring between **node**s.\n* **Transfer**: the behavior of **node** when a **packet** going through.\n\n#### Prerequisites\n\n- `go mod` must be supported and enabled.\n\n#### Usage\n\nUse in three steps: building network, starting network simulation, and collecting data.\n\n##### 1. Building network\n\nThe network is built by nodes and edges. Normally an edge connects only two nodes, each on one end. In some special cases, a node may connect to multiple (or none) incoming and/or outgoing nodes.\n\nWhile nodes are highly customizable, some typical nodes are pre-defined as follows:\n\n\u003c!-- note that mermaid compilation of GitHub action only supports code blocks with no indents --\u003e\n\n* Broadcast: a node transfers packet from one source to multiple targets.\n\n\u003c!-- generated by mermaid compile action - START --\u003e\n![~mermaid diagram 1~](/.resources/README-md-1.svg)\n\u003cdetails\u003e\n  \u003csummary\u003eMermaid markup\u003c/summary\u003e\n\n```mermaid\ngraph LR;\n    In --\u003e Broadcast --\u003e Out1;\n    Broadcast --\u003e Out2;\n    Broadcast --\u003e Out3;\n```\n\n\u003c/details\u003e\n\u003c!-- generated by mermaid compile action - END --\u003e\n\n* Channel: a node delays, losses or reorders packets passing by.\n\n\u003c!-- generated by mermaid compile action - START --\u003e\n![~mermaid diagram 2~](/.resources/README-md-2.svg)\n\u003cdetails\u003e\n  \u003csummary\u003eMermaid markup\u003c/summary\u003e\n\n```mermaid\ngraph LR;\n    In --\u003e Channel --\u003e|Loss \u0026 Delay \u0026 Reorder| Out;\n```\n\n\u003c/details\u003e\n\u003c!-- generated by mermaid compile action - END --\u003e\n\n* Endpoint: a node where to send and receive packets, usually acting as the endpoint of a chain.\n\n\u003c!-- generated by mermaid compile action - START --\u003e\n![~mermaid diagram 3~](/.resources/README-md-3.svg)\n\u003cdetails\u003e\n  \u003csummary\u003eMermaid markup\u003c/summary\u003e\n\n```mermaid\ngraph LR;\n   Nodes... --\u003e Endpoint;\n```\n\n\u003c/details\u003e\n\u003c!-- generated by mermaid compile action - END --\u003e\n\n* Gather: a node gathers packets from multiple sources to a single target.\n\n\u003c!-- generated by mermaid compile action - START --\u003e\n![~mermaid diagram 4~](/.resources/README-md-4.svg)\n\u003cdetails\u003e\n  \u003csummary\u003eMermaid markup\u003c/summary\u003e\n\n```mermaid\ngraph LR;\n    In1 --\u003e Gather ==\u003e Out;\n    In2 --\u003e Gather;\n    In3 --\u003e Gather;\n```\n\n\u003c/details\u003e\n\u003c!-- generated by mermaid compile action - END --\u003e\n\n* Restrict: a node limits pps or bps by dropping packets when its internal buffer overflows.\n\n\u003c!-- generated by mermaid compile action - START --\u003e\n![~mermaid diagram 5~](/.resources/README-md-5.svg)\n\u003cdetails\u003e\n  \u003csummary\u003eMermaid markup\u003c/summary\u003e\n\n```mermaid\ngraph LR;\n    In --\u003e Restrict --\u003e|Restricted Speed| Out;\n```\n\n\u003c/details\u003e\n\u003c!-- generated by mermaid compile action - END --\u003e\n\n* Scatter: a node selects which node the incoming packet should be route to according to a given rule.\n\n\u003c!-- generated by mermaid compile action - START --\u003e\n![~mermaid diagram 6~](/.resources/README-md-6.svg)\n\u003cdetails\u003e\n  \u003csummary\u003eMermaid markup\u003c/summary\u003e\n\n```mermaid\ngraph LR;\n    In --\u003e Scatter -.-\u003e Out1;\n    Scatter --\u003e|Selected Route| Out2;\n    Scatter -.-\u003e Out3;\n```\n\n\u003c/details\u003e\n\u003c!-- generated by mermaid compile action - END --\u003e\n\n\nAfter all necessary nodes created, connect them with code to build the network. To do so, just set the next node correctly for each node until all edges are defined as expected.\n\nIn addition, ns-x provides a builder to facilitate the process. Instead of connecting edges, it builds the network by connecting all paths in one line of code.\n\nA Path, *aka.* a chain, is similar to the path concept in graph theory, representing a route along the edges of a graph.\n\nMethods of the builder:\n\n* `Chain()`: saves current chain (path) and start to describe another chain.\n* `Node()`: appends a given node to current chain.\n* `NodeWithName()`: same as `Node()`, with a customizable name to refer to later.\n* `NodeByName()`: finds (refer to) a node with given name and appends it to current chain.\n* `NodeGroup()`: given a number of nodes, perform `Node()` operation on each of them in order.\n* `NodeGroupWithName()`: same as `NodeGroup()` with a customizable name.\n* `NodeGroupByName()`: finds a group with the given name, then perform `NodeGroup()` operation on it.\n* `Build()`: is the final trigger of the builder to really build the network. Note that in all nodes used in this builder line, all previously established connections will be overwritten.\n\n##### 2. Starting Network Simulation\n\nOnce the network built, start running it so packets can go through nodes.\n\n**Guaranteed behaviours of the simulation**\n\n* Order: if any event e at time point *t*, only generate events at time point not before *t*, then the handling order of two events at different time point is guaranteed, and the order of events at same time point is undetermined.\n* Accuracy: each event will be handled at the given time point exactly in simulate clock, and the difference between the simulator clock and real clock is as small as possible, usually some microseconds.\n\nSee comments in the code for additional node-specific guarantees.\n\n##### 3. Collecting Data\n\nData could be collected by callback function `node.OnTransferCallback()`. Also note that time-costing callbacks would slow down the simulation and lead to inaccuracy, so it is highly recommended only collecting data in the callbacks. Further analyses should be done after the simulation.\n\n#### Example\n\n##### 1. Basic Example\n\nSource code of this example can be found at main/basic.go\n\nFollowing is a basic example to show how to use the simulator, it's about a network with two entries, one endpoint and two chains.\n\n* Chain 1: entry1 -\u003e channel1(with `30% packet loss rate`) -\u003e restrict (`1 pps`, `1024 bps`, buffer limited to `4096 bytes` and `5 packets`) -\u003e endpoint\n* Chain 2: entry2 -\u003e channel2(with `10% packet loss rate`) -\u003e endpoint\n\n\u003cdetails\u003e\n\u003csummary\u003ecode of the basic example\u003c/summary\u003e\n\n```go\npackage main\n\n// Example of how to use the simulator basically\n\nimport (\n\t\"github.com/bytedance/ns-x/v2\"\n\t\"github.com/bytedance/ns-x/v2/base\"\n\t\"github.com/bytedance/ns-x/v2/math\"\n\t\"github.com/bytedance/ns-x/v2/node\"\n\t\"github.com/bytedance/ns-x/v2/tick\"\n\t\"go.uber.org/atomic\"\n\t\"math/rand\"\n\t\"time\"\n)\n\nfunc basic() {\n\tsource := rand.NewSource(0)\n\trandom := rand.New(source)\n\thelper := ns_x.NewBuilder()\n\tcallback := func(packet base.Packet, source, target base.Node, now time.Time) {\n\t\tprintln(\"emit packet\")\n\t}\n\tn1 := node.NewEndpointNode()\n\tt := time.Now()\n\tnetwork, nodes := helper.\n\t\tChain().\n\t\tNodeWithName(\"entry1\", n1).\n\t\tNode(node.NewChannelNode(node.WithTransferCallback(callback), node.WithLoss(math.NewRandomLoss(0.1, random)))).\n\t\tNode(node.NewRestrictNode(node.WithPPSLimit(1, 20))).\n\t\tNodeWithName(\"endpoint\", node.NewEndpointNode()).\n\t\tChain().\n\t\tNodeWithName(\"entry2\", node.NewEndpointNode()).\n\t\tNode(node.NewChannelNode(node.WithTransferCallback(callback), node.WithLoss(math.NewRandomLoss(0.1, random)))).\n\t\tNodeOfName(\"endpoint\").\n\t\tSummary().\n\t\tBuild()\n\tentry1 := nodes[\"entry1\"].(*node.EndpointNode)\n\tentry2 := nodes[\"entry2\"].(*node.EndpointNode)\n\tendpoint := nodes[\"endpoint\"].(*node.EndpointNode)\n\tcount := atomic.NewInt64(0)\n\tendpoint.Receive(func(packet base.Packet, now time.Time) []base.Event {\n\t\tif packet != nil {\n\t\t\tcount.Inc()\n\t\t\tprintln(\"receive packet at\", now.String())\n\t\t\tprintln(\"total\", count.Load(), \"packets received\")\n\t\t}\n\t\treturn nil\n\t})\n\ttotal := 20\n\tevents := make([]base.Event, 0, total*2)\n\tfor i := 0; i \u003c 20; i++ {\n\t\tevents = append(events, entry1.Send(base.RawPacket([]byte{0x01, 0x02}), t))\n\t}\n\tfor i := 0; i \u003c 20; i++ {\n\t\tevents = append(events, entry2.Send(base.RawPacket([]byte{0x01, 0x02}), t))\n\t}\n\tevent := base.NewPeriodicEvent(func(now time.Time) []base.Event {\n\t\tfor i := 0; i \u003c 10; i++ {\n\t\t\t_ = rand.Int()\n\t\t}\n\t\treturn nil\n\t}, time.Second, t)\n\tevents = append(events, event)\n\tnetwork.Run(events, tick.NewStepClock(t, time.Second), 300*time.Second)\n\tdefer network.Wait()\n}\n```\n\n\u003c/details\u003e\n\n##### 2. Echo Example\n\nSource code of this example can be found at main/echo.go\n\nFollowing is an example to show how to define a duplex network and set up an echo service, it's about an.\n\n\u003cdetails\u003e\n\u003csummary\u003ecode of the echo example\u003c/summary\u003e\n\n```go\npackage main\n\n// Example of a duplex network, where to endpoints echo to each other\n\nimport (\n\t\"github.com/bytedance/ns-x/v2\"\n\t\"github.com/bytedance/ns-x/v2/base\"\n\t\"github.com/bytedance/ns-x/v2/math\"\n\t\"github.com/bytedance/ns-x/v2/node\"\n\t\"github.com/bytedance/ns-x/v2/tick\"\n\t\"time\"\n)\n\nfunc echo() {\n\tnow := time.Now()\n\thelper := ns_x.NewBuilder()\n\tnetwork, nodes := helper.\n\t\tChain().\n\t\tNodeWithName(\"restrict 1\", node.NewRestrictNode(node.WithBPSLimit(1024*1024, 4*1024*1024))).\n\t\tNodeWithName(\"channel 1\", node.NewChannelNode(node.WithDelay(math.NewFixedDelay(150*time.Millisecond)))).\n\t\tChain().\n\t\tNodeWithName(\"restrict 2\", node.NewRestrictNode(node.WithPPSLimit(10, 50))).\n\t\tNodeWithName(\"channel 2\", node.NewChannelNode(node.WithDelay(math.NewFixedDelay(200*time.Millisecond)))).\n\t\tChain().\n\t\tNodeWithName(\"endpoint 1\", node.NewEndpointNode()).\n\t\tGroup(\"restrict 1\", \"channel 1\").\n\t\tNodeWithName(\"endpoint 2\", node.NewEndpointNode()).\n\t\tChain().\n\t\tNodeOfName(\"endpoint 2\").\n\t\tGroup(\"restrict 2\", \"channel 2\").\n\t\tNodeOfName(\"endpoint 1\").\n\t\tSummary().\n\t\tBuild()\n\tendpoint1 := nodes[\"endpoint 1\"].(*node.EndpointNode)\n\tendpoint2 := nodes[\"endpoint 2\"].(*node.EndpointNode)\n\tendpoint1.Receive(func(packet base.Packet, now time.Time) []base.Event {\n\t\tprintln(\"endpoint 1 receive:\", string(packet.(base.RawPacket)), \"at\", now.String())\n\t\treturn base.Aggregate(endpoint1.Send(packet, now))\n\t})\n\tendpoint2.Receive(func(packet base.Packet, now time.Time) []base.Event {\n\t\tprintln(\"endpoint 2 receive:\", string(packet.(base.RawPacket)), \"at\", now.String())\n\t\treturn base.Aggregate(endpoint2.Send(packet, now))\n\t})\n\tnetwork.Run([]base.Event{endpoint1.Send(base.RawPacket(\"hello world\"), now)}, tick.NewStepClock(now, time.Second), 30*time.Second)\n\tdefer network.Wait()\n}\n```\n\n\u003c/details\u003e\n\n##### 3. Route Example\n\nSource code of this example can be found at main/route.go\n\nFollowing is an advanced example to show how to customize route rules of the network.\n\nSuppose we have 1 client and 2 servers, route from client to server 1 has 200ms delay and no loss, route from client to server 2 has 300ms delay and no loss.\n\nWe assign ip address \"192.168.0.1\" to server 1, \"192.168.0.2\" to server 2, ports are not used in this example, but obviously route rules based on ip+port is similar to which based on ip.\n\nThis example only shows how to define route rules of a simplex network for simplicity, but it's easy to define route rules for duplex network.\n\n\u003cdetails\u003e\n\u003csummary\u003ecode of the route example\u003c/summary\u003e\n\n```go\npackage main\n\n// Example of how to customize route rule\n\nimport (\n\t\"github.com/bytedance/ns-x/v2\"\n\t\"github.com/bytedance/ns-x/v2/base\"\n\t\"github.com/bytedance/ns-x/v2/math\"\n\t\"github.com/bytedance/ns-x/v2/node\"\n\t\"github.com/bytedance/ns-x/v2/tick\"\n\t\"time\"\n)\n\nfunc route() {\n\thelper := ns_x.NewBuilder()\n\tt := time.Now()\n\trouteTable := make(map[base.Node]base.Node)\n\tipTable := make(map[string]base.Node)\n\tscatter := node.NewScatterNode(node.WithRouteSelector(func(packet base.Packet, nodes []base.Node) base.Node {\n\t\tif p, ok := packet.(*packetWithNode); ok {\n\t\t\treturn routeTable[p.destination]\n\t\t}\n\t\tpanic(\"no route to host\")\n\t}))\n\tclient := node.NewEndpointNode()\n\tnetwork, nodes := helper.\n\t\tChain().\n\t\tNode(client).\n\t\tNode(scatter).\n\t\tNodeWithName(\"route1\", node.NewChannelNode(node.WithDelay(math.NewFixedDelay(time.Millisecond*200)))).\n\t\tNodeWithName(\"server1\", node.NewEndpointNode()).\n\t\tChain().\n\t\tNode(client).\n\t\tNode(scatter).\n\t\tNodeWithName(\"route2\", node.NewChannelNode(node.WithDelay(math.NewFixedDelay(time.Millisecond*300)))).\n\t\tNodeWithName(\"server2\", node.NewEndpointNode()).\n\t\tBuild()\n\tserver1 := nodes[\"server1\"].(*node.EndpointNode)\n\tserver2 := nodes[\"server2\"].(*node.EndpointNode)\n\troute1 := nodes[\"route1\"]\n\troute2 := nodes[\"route2\"]\n\trouteTable[server1] = route1\n\trouteTable[server2] = route2\n\tipTable[\"192.168.0.1\"] = server1\n\tipTable[\"192.168.0.2\"] = server2\n\tserver1.Receive(react1) // server 1 should receive after 1-second send delay + 200 milliseconds channel delay\n\tserver2.Receive(react2) // server 2 should receive after 2-second send delay + 200 milliseconds channel delay\n\tsender := createSender(client, ipTable)\n\tevents := make([]base.Event, 0)\n\tevents = append(events, sender(base.RawPacket([]byte{}), \"192.168.0.1\", t.Add(time.Second*1))) // send to server1 after 1 second\n\tevents = append(events, sender(base.RawPacket([]byte{}), \"192.168.0.2\", t.Add(time.Second*2))) // send to server2 after 2 second\n\tnetwork.Run(events, tick.NewStepClock(t, time.Millisecond), 300*time.Second)\n\tdefer network.Wait()\n}\n\nfunc react1(packet base.Packet, now time.Time) []base.Event {\n\tprintln(\"server 1 receive at\", now.String())\n\treturn nil\n}\n\nfunc react2(packet base.Packet, now time.Time) []base.Event {\n\tprintln(\"server 2 receive at\", now.String())\n\treturn nil\n}\n\ntype packetWithNode struct {\n\tbase.Packet\n\tsource, destination base.Node\n}\n\ntype sender func(packet base.Packet, ip string, t time.Time) base.Event\n\nfunc createSender(client *node.EndpointNode, ipTable map[string]base.Node) sender {\n\treturn func(packet base.Packet, ip string, t time.Time) base.Event {\n\t\treturn client.Send(\u0026packetWithNode{packet, client, ipTable[ip]}, t)\n\t}\n}\n```\n\n\u003c/details\u003e\n\n## Design\n\n#### Architecture\n\nThe simulator is event driven, each event will be handled at the given time point, and generate subsequent events. Behaviors of nodes will be wrapped as events.\n\n#### Event Loop\n\nThe event loop maintains a thread-local event queue, in order to sort the events.\n\nBasic operation is dequeue an event from the queue, handle it and enqueue events generated each time. The event loop will drain the event queue until only events with time point after current time are left.\n\nOnce the event queue is drained, event loop will refresh current time from the clock, and redo operations above until no events left in the queue or reach lifetime of the simulation.\n\n#### Event Queue\n\nThe event queue is used to sort events to guarantee the handling order.\n\nAs observed, most of the events generated just with a short delay, which form an events cluster. For events cluster, bucket sort is used first to divide events into some buckets; other events are put into another bucket.\n\nFor each bucket, a heap sort is used to form the priority queue.\n\nSince buckets are created/destroyed frequently, but total count of buckets at the same time are usually within a bound. All the buckets are stored in a ring queue, to reduce the cost and avoid gc.\n\n## Contribution\n\n#### Future work\n\n* ~~parallelize main loop~~ (done)\n* ~~implement commonly used protocol stack as a new node type~~ (will be implemented as different packet type)\n* ~~separate send and pass to avoid cumulative error~~ (done)\n* ~~Buffer overflow determination of *restrict node* should have a more accurate way~~ (done)\n* ~~split event heap when size of heap is large enough~~ (done)\n* implement packets of commonly used protocol\n\n#### Contributors\n\n\u003ca href=\"https://github.com/bytedance/ns-x/graphs/contributors\"\u003e\n    \u003cimg src=\"https://contrib.rocks/image?repo=bytedance/ns-x\"  alt=\"contributors\"/\u003e\n\u003c/a\u003e\n\nMade with [contributors-img](https://contrib.rocks).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbytedance%2Fns-x","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbytedance%2Fns-x","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbytedance%2Fns-x/lists"}