{"id":20696847,"url":"https://github.com/araobp/nlan","last_synced_at":"2025-04-22T20:43:48.174Z","repository":{"id":145719240,"uuid":"43436060","full_name":"araobp/nlan","owner":"araobp","description":"Interactive SDN/IOT with tega db and Jupyter/IPython","archived":false,"fork":false,"pushed_at":"2017-07-22T08:11:45.000Z","size":11379,"stargazers_count":11,"open_issues_count":3,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-29T18:41:30.817Z","etag":null,"topics":["docker","jupyter","raspberry-pi","sdn","whitebox"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/araobp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2015-09-30T13:56:30.000Z","updated_at":"2021-02-03T09:33:16.000Z","dependencies_parsed_at":null,"dependency_job_id":"b20ad6bc-2f40-4031-85d3-b9931a032257","html_url":"https://github.com/araobp/nlan","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/araobp%2Fnlan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/araobp%2Fnlan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/araobp%2Fnlan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/araobp%2Fnlan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/araobp","download_url":"https://codeload.github.com/araobp/nlan/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250321408,"owners_count":21411569,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","jupyter","raspberry-pi","sdn","whitebox"],"created_at":"2024-11-17T00:15:37.347Z","updated_at":"2025-04-22T20:43:48.156Z","avatar_url":"https://github.com/araobp.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Interactive SDN/IOT with tega db and Jupyter/IPython\n\n![All in one](https://docs.google.com/drawings/d/1Q4Et0x4pe4XPdw8pGaFjUrOnA9Un6XUIJAvUs4DqsaI/pub?w=600\u0026h=400)\n\nThis project nlan (meaning \"new LAN\") unifies outputs from my two other projects \"[neutron-lan](https://github.com/araobp/neutron-lan)\" and \"[tega](https://github.com/araobp/tega)\".\n\n## Background and motivation\n- OpenDaylight MD-SAL is too heavy for networking Linux containers on my Raspberry Pi.\n- YANG is incompatible with Python dict, Golang map and so on: I just want JSON-centric MD-SAL.\n- As my hobby, I design a model-driven/event-driven architecture for networking Linux containers.\n- I think Jupyter/IPython is a wonderful IDE for SDN/IOT (and also for Deep Learning...).\n- [OCP networking](http://www.opencompute.org/wiki/Networking) is a wonderland!\n- If the computing power moves to the network edge, what you need is not VLAN but application-level logical seperation of network (SSL/TLS, WebSocket, RTP/RTCP, ...), that is, what you need is \"session\".\n\n## Architecture\nSort of \"immutable infrastructure\" for networking...\n\n![NLAN architecture](https://docs.google.com/drawings/d/1VauRM6d2A03gIPxFbaVYdZpP8Yadre8KqL53XnntqDI/pub?w=600\u0026h=400)\n\n![NLAN archiecture internal](https://docs.google.com/drawings/d/1y2YXolq8bpm8E2xTgcDPfNzRtJKjc19cBVYKYm9prFE/pub?w=600\u0026h=400)\n\n## Visualization and analytics\nI use Jupyter and IPython for visualization and analytics of NLAN.\n```\nimport networkx as nx\nget_ipython().magic('matplotlib inline')\n\nimport tega.driver\nd = tega.driver.Driver(host='192.168.57.133')\nsubnets = d.get(path='graph.subnets')\n\ng = nx.DiGraph(subnets['172.21.1.0/24'])\nnx.draw_spring(g, node_size=1000, with_labels=True, arrows=True, alpha=0.8)\n```\n![NLAN visualization](./doc/jupyter/output_2_0.png)\n\nDirectional graph of IP routing that Quagga and GoBGP have setup on the network\n\nNote: in case of OpenFlow Controller, the directional graph is calculated by the controller.\n```\nOpenFlow-based SDN: calculate directional graph and write the edges(flow entries) to the switches/routers.\nSDN with BGP/OSPF: write config/policy to the switches/routers, then each of the switches/routers calculates directional graph.\n```\n\n### Jupyter notebook examples\n- [PTN topology](./doc/jupyter/topo.md)\n- [Subnet graph set up by BGP at each router](./doc/jupyter/BGP_route_graph.md)\n- [Server-client graph from netstat at each router](./doc/jupyter/server_client.md)\n\nYou can find the notebooks [here](./ipynb).\n\n## NLAN services\n- PTN: Packet Transport Network (Layer 1 and Layer 2)\n- Vhosts: netns-based virtual hosts\n- Router: Quagga configuration\n\nTo be added:\n- Links: direct linking(veth/macvlan/tun/tap)\n- Bridges: non-distributed virtual switch(linux bridge per vlan)\n- DVR: Distributed Virtual Switch and Distributed Virtual Router (Layer 2 and Layer 3)\n\n## Target use cases\n\nUse case 1 has already been implemented, and use case 2 is being planned at the moment.\n\n### Use case 1: Network simulation\n\nThis use case makes use of NLAN's PTN, vHosts and Router services.\n![WAN simulation](https://docs.google.com/drawings/d/1VKfKlwnzWQ2-ImfXeB5uNegGBK0BnaGU_4lS8h4Qpcw/pub?w=640\u0026h=480)\n\n#### Declarative state representations:\n- [ptn-bgp.yaml](./etc/ptn-bgp.yaml)\n- [ptn-ospf.yaml](./etc/ptn-ospf.yaml)\n\n#### Data trees on tega db\n![NLAN data trees](https://docs.google.com/drawings/d/1JjByqUw7wvc9dKWcpEQF9F0_iQFdgl6X3o6rTb--12I/pub?w=480\u0026h=300)\n\n#### Running the simulated network on Raspberry Pi\nThis is sort of micro NFV(Network Function Virtualization) on a single Rapsberry Pi.\n- Nine virtual routers (Linux containers)\n- Sixteen virutal hosts (netns)\n\nYou can learn how routing protocols work on this simulated network.\n\n[Setting up the software on Raspberry Pi](./doc/RPI.md)\n\nLog in the virtual routers with ssh, and try \"ip\" or \"vtysh\" commands:\n- ip route\n- ip addr\n- ip link\n- ip netns\n- vtysh: show run\n- vtysh: show ip route\n- vtysh: show ip bgp\n     :\n\n#### Quagga and GoBGP:\nThis use case makes use of Quagga, but [gobgp](https://github.com/osrg/gobgp) may optionally be used as Route Reflector or Route Server on \"RR\" container in the fig above.\n- [gobgpd.conf](./etc/gobgpd.conf)\n\nYou can also launch gobgpd from NLAN agent by including \"EmbeddedBgp: true\" in your NLAN state file:\n```\n      Router:\n        Loopback: 10.1.1.5/32\n        EmbeddedBgp: true\n        Bgp:\n          - As: 100\n            Neighbors:\n              - Peer: 10.200.1.101\n                RemoteAs: 100\n                RouteReflectorClient: true\n              - Peer: 10.200.1.102\n                RemoteAs: 100\n                RouteReflectorClient: true\n              - Peer: 10.200.1.103\n                RemoteAs: 100\n                RouteReflectorClient: true\n              - Peer: 10.200.1.104\n                RemoteAs: 100\n                RouteReflectorClient: true\n```\n- [Using gobgp command](./doc/GOBGP.md)\n\n### Use case 2: SOHO NFV (Network Functions Virtualization)\n\nThis is the next use case I am going to work on... (as my hobby: not so practical)\n\n![SONO-NFV](https://docs.google.com/drawings/d/11fJUimZVrGxqAdq-hJK4abDu0ZThkfHGtbl_94zW0rQ/pub?w=640\u0026h=480)\n\n## Network simulation with Linux containers\nI use Linux containers as virtual routers, and this tool will set up virtual links (L0/L1) and virtual switches (L2) over the containers. Then I will run Quagga/Zebra(L3) daemons over the virtual routers to study how legacy routing protocols work.\n- [An example of such a network](https://camo.githubusercontent.com/3f15c9634b2491185ec680fa5bb7d19f6f01146b/68747470733a2f2f646f63732e676f6f676c652e636f6d2f64726177696e67732f642f31564b664b6c776e7a5751322d496d6658654235754e656747424b30426e6147555f346c53386834517063772f7075623f773d39363026683d373230)\n- [Working with Docker for network simulation](https://camo.githubusercontent.com/77cf473ea9499432e57b06a951f5f5248419f9e1/68747470733a2f2f646f63732e676f6f676c652e636f6d2f64726177696e67732f642f313631426e383077384a5a4b513742586d496f306272377851346b71456442635f585a3235347a754f5253552f7075623f773d36383026683d343030)\n\n# NLAN installation\n\n[Step 1] Make a Docker image named \"router\" following the instruction [here](./docker/SETUP.md).\n\n[Step 2] Install and start tega db:\n\nYou need to have Python3.5 installed on your Ubuntu/Debian.\n\n```\n$ go get github.com/araobp/tega/driver\n$ cd $GOPATH/src/github.com/araobp/tega\n$ python setup.py install\n$ pip install mako\n```\nFor Hypriot/RaspberryPi, you need to export this environment variable:\n```\n$ export SETUP_SCRIPT=setup_rpi.sh\n```\nFor Debian/Ubuntu, you do not need to export the variable above. \n\nThen start tega db:\n```\n$ cd scripts\n$ ./tegadb\n\n   __\n  / /____  ____ _____ _\n / __/ _ \\/ __ `/ __ `/\n/ /_/  __/ /_/ / /_/ /\n\\__/\\___/\\__, /\\__,_/\n        /____/\n\ntega_id: global, config: None, operational: None\n\nNamespace(config=None, extensions='/root/work/src/github.com/araobp/nlan/plugins/nlan', ghost=None, gport=None, logdir='./var', loglevel='INFO', maxlen=10, operational=None, port=8739, tegaid='global')\n\nINFO:2016-03-16 15:14:51,966:Reloading log from ./var...\nINFO:2016-03-16 15:14:51,972:Reloading done\nINFO:2016-03-16 15:14:52,675:plugin attached to idb: Hook\nINFO:2016-03-16 15:14:52,692:plugin attached to idb: Deployment\nINFO:2016-03-16 15:14:52,707:plugin attached to idb: Subnets\nINFO:2016-03-16 15:14:52,712:plugin attached to idb: Topo\nINFO:2016-03-16 15:14:52,739:plugin attached to idb: PtnBgp\nINFO:2016-03-16 15:14:52,765:plugin attached to idb: Workflow\nINFO:2016-03-16 15:14:52,782:plugin attached to idb: Fabric\nINFO:2016-03-16 15:14:52,800:plugin attached to idb: ServerClient\nINFO:2016-03-16 15:14:52,823:plugin attached to idb: IpAddressManagement\nINFO:2016-03-16 15:14:52,842:plugin attached to idb: Template\n```\n\n[Step 2]\nTry this at the tega CLI to put \"ptn-bgp\" state onto tega db: \n```\n[tega: 2] plugins.ptnbgp()\n```\nThe script sets up [this network](https://camo.githubusercontent.com/3f15c9634b2491185ec680fa5bb7d19f6f01146b/68747470733a2f2f646f63732e676f6f676c652e636f6d2f64726177696e67732f642f31564b664b6c776e7a5751322d496d6658654235754e656747424b30426e6147555f346c53386834517063772f7075623f773d39363026683d373230).\n\nYou may also try \"plugins.fabric()\" instead. It will setup L3 fabric simulating a data center network.\n\n[Step 3(option)]\nYou may take a snapshop of tega db to make tega db's start-up faster:\n```\n[tega: 3] ss \n```\n\n[Step 4] Execute the following command to build Docker image with NLAN agent embedded and to start the containers:\n\n```\n[tega: 4] plugins.deploy() \n```\n\nNLAN agent on each container connects to tega db to fetch NLAN state.\n\nIf you want to monitor the activities of each agents, subscribe(path=\"hosts\") on the CLI ([example](./doc/monitoring-activities.md)).\n\n[Step 5] Confirm that all the containers are running\n\n```\n[tega: 5] subscribers\nDeployment: [Deployment]\nIpAddressManagement: [IpAddressManagement]\nTemplate: [Template]\nTopo: [Topo, config-.*]\nce1: [ce1]\nce2: [ce2]\nce3: [ce3]\nce4: [ce4]\npe1: [pe1]\npe2: [pe2]\npe3: [pe3]\npe4: [pe4]\nrr: [rr]\n\n```\n\n[Step 6] Try raw commands to check the state of each container\n\n```\n[tega: 6] raw.ce1('ip route')\ndefault via 172.17.0.1 dev eth0\n10.1.1.1 via 10.201.11.1 dev int_br111  proto zebra\n10.1.1.2 via 10.202.11.1 dev int_br211  proto zebra\n10.1.1.3 via 10.201.11.1 dev int_br111  proto zebra\n10.1.2.2 via 10.201.11.1 dev int_br111  proto zebra\n10.1.2.3 via 10.201.11.1 dev int_br111  proto zebra\n10.1.2.4 via 10.201.11.1 dev int_br111  proto zebra\n10.10.10.0/24 dev eth0  proto kernel  scope link  src 10.10.10.6\n10.200.1.0/24 via 10.201.11.1 dev int_br111  proto zebra\n10.200.2.0/24 via 10.201.11.1 dev int_br111  proto zebra\n10.201.11.0/24 dev int_br111  proto kernel  scope link  src 10.201.11.2\n10.201.12.0/24 via 10.201.11.1 dev int_br111  proto zebra\n10.202.11.0/24 dev int_br211  proto kernel  scope link  src 10.202.11.2\n10.202.12.0/24 via 10.202.11.1 dev int_br211  proto zebra\n10.203.13.0/24 via 10.201.11.1 dev int_br111  proto zebra\n10.203.14.0/24 via 10.201.11.1 dev int_br111  proto zebra\n10.204.13.0/24 via 10.201.11.1 dev int_br111  proto zebra\n10.204.14.0/24 via 10.201.11.1 dev int_br111  proto zebra\n172.17.0.0/16 dev eth0  proto kernel  scope link  src 172.17.0.7\n172.21.1.0/24 dev br_172.21.1.1  proto kernel  scope link  src 172.21.1.1\n172.21.2.0/24 via 10.201.11.1 dev int_br111  proto zebra\n172.21.3.0/24 via 10.201.11.1 dev int_br111  proto zebra\n172.21.4.0/24 via 10.201.11.1 dev int_br111  proto zebra\n172.22.1.0/24 dev br_172.22.1.1  proto kernel  scope link  src 172.22.1.1\n172.22.2.0/24 via 10.201.11.1 dev int_br111  proto zebra\n172.22.3.0/24 via 10.201.11.1 dev int_br111  proto zebra\n172.22.4.0/24 via 10.201.11.1 dev int_br111  proto zebra\n\n[tega: 7] raw.ce2('ip route')\n               :\n               \n```\nYou may also start a ssh session to the containers:\n```\n$ cd scripts \n$ ./ssh.sh pe1\n       :\n$ ./ssh.sh ce1\n       :\n```\nThe password is \"root\".\n\nOr you may also use \"ip netns\" command to the containers:\n```\n$ ip netns exec pe1 ip route\n```\n\n[Step 8] Call hook functions to reflesh operational data trees\n```\n[tega: 8] plugins.hook() \n```\n\n[Step 9] Check the operational trees\n```\n[tega: 9] getr operational-(\\w*)\\.ip\noperational-ce1.ip:\n  groups:\n  - [ce1]\n  instance:\n    addr: {10.1.2.1: lo, 10.10.10.6: eth0, 10.201.11.2: int_br111, 10.202.11.2: int_br211,\n      127.0.0.1: lo, 172.17.0.7: eth0, 172.21.1.1: br_172.21.1.1, 172.22.1.1: br_172.22.1.1}\n    dev:\n      br_172.21.1.1: [172.21.1.1]\n      br_172.22.1.1: [172.22.1.1]\n      eth0: [172.17.0.7, 10.10.10.6]\n      int_br111: [10.201.11.2]\n      int_br211: [10.202.11.2]\n      lo: [127.0.0.1, 10.1.2.1]\n    hook: {addr: '%ce1.ipAddr', route: '%ce1.ipRoute'}\n    route:\n      10.1.1.1/32: {Dev: int_br111, Src: '', Via: 10.201.11.1}\n      10.1.1.2/32: {Dev: int_br211, Src: '', Via: 10.202.11.1}\n      10.1.1.3/32: {Dev: int_br111, Src: '', Via: 10.201.11.1}\n      10.1.1.4/32: {Dev: int_br111, Src: '', Via: 10.201.11.1}\n      10.1.2.2/32: {Dev: int_br111, Src: '', Via: 10.201.11.1}\n      10.1.2.3/32: {Dev: int_br111, Src: '', Via: 10.201.11.1}\n      10.1.2.4/32: {Dev: int_br211, Src: '', Via: 10.202.11.1}\n      10.10.10.0/24: {Dev: eth0, Src: 10.10.10.6, Via: ''}\n      10.200.1.0/24: {Dev: int_br111, Src: '', Via: 10.201.11.1}\n      10.200.2.0/24: {Dev: int_br111, Src: '', Via: 10.201.11.1}\n      10.201.11.0/24: {Dev: int_br111, Src: 10.201.11.2, Via: ''}\n      10.201.12.0/24: {Dev: int_br111, Src: '', Via: 10.201.11.1}\n      10.202.11.0/24: {Dev: int_br211, Src: 10.202.11.2, Via: ''}\n      10.202.12.0/24: {Dev: int_br211, Src: '', Via: 10.202.11.1}\n      10.203.13.0/24: {Dev: int_br111, Src: '', Via: 10.201.11.1}\n      10.203.14.0/24: {Dev: int_br111, Src: '', Via: 10.201.11.1}\n      10.204.13.0/24: {Dev: int_br111, Src: '', Via: 10.201.11.1}\n      10.204.14.0/24: {Dev: int_br111, Src: '', Via: 10.201.11.1}\n      172.17.0.0/16: {Dev: eth0, Src: 172.17.0.7, Via: ''}\n      172.21.1.0/24: {Dev: br_172.21.1.1, Src: 172.21.1.1, Via: ''}\n      172.21.2.0/24: {Dev: int_br111, Src: '', Via: 10.201.11.1}\n      172.21.3.0/24: {Dev: int_br111, Src: '', Via: 10.201.11.1}\n      172.21.4.0/24: {Dev: int_br211, Src: '', Via: 10.202.11.1}\n      172.22.1.0/24: {Dev: br_172.22.1.1, Src: 172.22.1.1, Via: ''}\n      172.22.2.0/24: {Dev: int_br111, Src: '', Via: 10.201.11.1}\n      172.22.3.0/24: {Dev: int_br111, Src: '', Via: 10.201.11.1}\n      172.22.4.0/24: {Dev: int_br211, Src: '', Via: 10.202.11.1}\n      default: {Dev: eth0, Src: '', Via: 172.17.0.1}\noperational-ce2.ip:\n  groups:\n  - [ce2]\n  instance:\n    addr: {10.1.2.2: lo, 10.10.10.7: eth0, 10.201.12.2: int_br112, 10.202.12.2: int_br212,\n      127.0.0.1: lo, 172.17.0.8: eth0, 172.21.2.1: br_172.21.2.1, 172.22.2.1: br_172.22.2.1}\n    dev:\n      br_172.21.2.1: [172.21.2.1]\n      br_172.22.2.1: [172.22.2.1]\n      eth0: [172.17.0.8, 10.10.10.7]\n      int_br112: [10.201.12.2]\n                 :\n```\n[Step 10] Start jupyter notebook and open the notebooks [here](./ipynb/).\n```\ncd to the project root directory, then:\n$ cd ipynb\n$ jupyter notebook\n```\n\nYou need to change the IP address to the one that tega db binds:\n```\nimport tega.driver\nd = tega.driver.Driver(host='192.168.57.133')  \u003c== MODIFY THIS!\n```\n\n# Development environment setup\n\n## Python3.5\n\n- Download the source code from [here](https://www.python.org/downloads/source/).\n- Build and install it.\n\n## IPython/Jupyter\nThe easiest way is to install Anaconda\n- https://www.continuum.io/downloads\n\nNote that Anaconda already includes Python3.5 and other packages used by this project as well.\n\n## Golang and protobuf\n- Go lang installation: https://golang.org/dl/\n- Protobuf build and installation: https://github.com/google/protobuf/blob/master/INSTALL.txt\n```\n$ ./autogen.sh\n$ ./configure\n$ make\n$ make install\n```\n- Add /usr/local/lib to LD_LIBRARY_PATH\n```\n$ export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARLY_PATH\n\n```\n### Go plugin for vim\n\nInstall [vim-go](https://github.com/fatih/vim-go) to your vim.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faraobp%2Fnlan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faraobp%2Fnlan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faraobp%2Fnlan/lists"}